You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "bbende (via GitHub)" <gi...@apache.org> on 2024/05/07 17:22:23 UTC

[PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

bbende opened a new pull request, #8765:
URL: https://github.com/apache/nifi/pull/8765

   …ing flows
   
   <!-- Licensed to the Apache Software Foundation (ASF) under one or more -->
   <!-- contributor license agreements.  See the NOTICE file distributed with -->
   <!-- this work for additional information regarding copyright ownership. -->
   <!-- The ASF licenses this file to You under the Apache License, Version 2.0 -->
   <!-- (the "License"); you may not use this file except in compliance with -->
   <!-- the License.  You may obtain a copy of the License at -->
   <!--     http://www.apache.org/licenses/LICENSE-2.0 -->
   <!-- Unless required by applicable law or agreed to in writing, software -->
   <!-- distributed under the License is distributed on an "AS IS" BASIS, -->
   <!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -->
   <!-- See the License for the specific language governing permissions and -->
   <!-- limitations under the License. -->
   
   # Summary
   
   [NIFI-13105](https://issues.apache.org/jira/browse/NIFI-13105)
   
   This PR adds a new `FlowRegistryClient` that uses `GitHub` API to version control flows. Currently the client is configured with the repo information, including a branch and optional path in the repo to use, otherwise the root of the repo is used. 
   
   Buckets are directories underneath the location in the repo. The client will check for a bucket named `Default`, and if it does not exist it will be created. Additional buckets can be created by adding additional directories to the repo outside of NiFi. 
   
   The branch is planned to become part of the version control screens when saving/importing which is tracked by. [NIFI-13127](https://issues.apache.org/jira/browse/NIFI-13127). Until then, the branch that is configured in the client will be used for all operations, so in order to work on different branches, it currently requires creating separate registry clients pointing at the same repo, but different branches.
   
   It is also possible to import flows from a public repo that you don't have permissions to by not providing an access token. This will allow import and change version, but attempting to save a change will result in a "not found" exception which is GitHub's behavior when you specify a branch or file that you don't have access to, even though it may be in the repo.
   
   # Tracking
   
   Please complete the following tracking steps prior to pull request creation.
   
   ### Issue Tracking
   
   - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created
   
   ### Pull Request Tracking
   
   - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-00000`
   - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-00000`
   
   ### Pull Request Formatting
   
   - [X] Pull Request based on current revision of the `main` branch
   - [X] Pull Request refers to a feature branch with one commit containing changes
   
   # Verification
   
   Please indicate the verification steps performed prior to pull request creation.
   
   ### Build
   
   - [X] Build completed using `mvn clean install -P contrib-check`
     - [X] JDK 21
   
   ### UI Contributions
   
   - [ ] NiFi is modernizing its UI. Any contributions that update the [current UI](https://github.com/apache/nifi/tree/main/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui) also need to be implemented in the [new UI](https://github.com/apache/nifi/tree/main/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi).  
   
   ### Licensing
   
   - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html)
   - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files
   
   ### Documentation
   
   - [ ] Documentation formatting appears as expected in rendered files
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594087217


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubRepositoryClient.java:
##########
@@ -0,0 +1,404 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+
+package org.apache.nifi.github;
+
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+import org.kohsuke.github.GHRef;
+import org.kohsuke.github.GHRepository;
+import org.kohsuke.github.GitHub;
+import org.kohsuke.github.GitHubBuilder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * Client to encapsulate access to a GitHub Repository through the Hub4j GitHub client.
+ */
+public class GitHubRepositoryClient {
+
+    private static final Logger LOGGER = LoggerFactory.getLogger(GitHubRepositoryClient.class);
+
+    private static final String BRANCH_REF_PATTERN = "refs/heads/%s";
+    private static final int COMMIT_PAGE_SIZE = 50;
+
+    private final GHRepository repository;
+    private final String repoPath;
+    private final GitHubAuthenticationType authenticationType;
+
+    private GitHubRepositoryClient(final Builder builder) throws IOException {
+        final GitHubBuilder gitHubBuilder = new GitHubBuilder().withEndpoint(builder.apiUrl);
+
+        authenticationType = builder.authenticationType;
+        switch (authenticationType) {
+            case PERSONAL_ACCESS_TOKEN -> gitHubBuilder.withOAuthToken(builder.personalAccessToken);
+            case APP_INSTALLATION_TOKEN -> gitHubBuilder.withAppInstallationToken(builder.appInstallationToken);
+        }
+
+        final GitHub gitHub = gitHubBuilder.build();
+        try {
+            repository = gitHub.getRepository(builder.repoOwner + "/" + builder.repoName);
+        } catch (final IOException e) {
+            LOGGER.error("Unable to access GitHub repository [{}]", builder.repoName, e);
+            throw e;
+        }
+        repoPath = builder.repoPath;
+    }
+
+    /**
+     * @return the authentication type this client is configured with
+     */
+    public GitHubAuthenticationType getAuthenticationType() {
+        return authenticationType;
+    }
+
+    /**
+     * Creates the content specified by the given builder.
+     *
+     * @param request the request for the content to create
+     * @return the update response
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public GHContentUpdateResponse createContent(final GitHubCreateContentRequest request) throws IOException, FlowRegistryException {
+        final String branch = request.getBranch();
+        final String resolvedPath = getResolvedPath(request.getPath());
+        LOGGER.debug("Creating content at path [{}] on branch [{}] in repo [{}] ", resolvedPath, branch, repository.getName());
+        return execute(() -> {
+            try {
+                return repository.createContent()
+                        .branch(branch)
+                        .path(resolvedPath)
+                        .content(request.getContent())
+                        .message(request.getMessage())
+                        .sha(request.getExistingContentSha())
+                        .commit();
+            } catch (final FileNotFoundException fnf) {
+                throwPathOrBranchNotFound(resolvedPath, branch);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the names of all the branches in the repo.
+     *
+     * @return the set of all branches in the repo
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public Set<String> getBranches() throws IOException, FlowRegistryException {
+        LOGGER.debug("Getting branches for repo [{}]", repository.getName());
+        return execute(() -> repository.getBranches().keySet());
+    }
+
+    /**
+     * Gets an InputStream to read the latest content of the given path from the given branch.
+     * The returned stream already contains the contents of the requested file.
+     *
+     * @param path the path to the content
+     * @param branch the branch
+     * @return an input stream containing the contents of the path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public InputStream getContentFromBranch(final String path, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting content for [{}] from branch [{}] in repo [{}] ", resolvedPath, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHContent ghContent = repository.getFileContent(resolvedPath, branchRef);
+                return ghContent.read();
+            } catch (final FileNotFoundException e) {
+                throwPathOrBranchNotFound(resolvedPath, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the content of the given path from the given commit.
+     * The returned stream already contains the contents of the requested file.
+     *
+     * @param path the path to the content
+     * @param commitSha the commit SHA
+     * @return an input stream containing the contents of the path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public InputStream getContentFromCommit(final String path, final String commitSha) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        LOGGER.debug("Getting content for [{}] from commit [{}] in repo [{}] ", resolvedPath, commitSha, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHContent ghContent = repository.getFileContent(resolvedPath, commitSha);
+                return ghContent.read();
+            } catch (final FileNotFoundException fnf) {
+                throw new FlowRegistryException("Path [" + resolvedPath + "] or Commit [" + commitSha + "] not found");
+            }
+        });
+    }
+
+    /**
+     * Gets the commits for a given path on a given branch.
+     *
+     * @param path the path
+     * @param branch the branch
+     * @return the list of commits for the given path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public List<GHCommit> getCommits(final String path, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting commits for [{}] from branch [{}] in repo [{}]", resolvedPath, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHRef branchGhRef = repository.getRef(branchRef);
+                return repository.queryCommits()
+                        .path(resolvedPath)
+                        .from(branchGhRef.getObject().getSha())
+                        .pageSize(COMMIT_PAGE_SIZE)

Review Comment:
   Yes the comments on `toList()` say:
   ```
   Eagerly walk {@link Iterable} and return the result in a list.
   ```
   So it will page through the results using page size and return all of them in a list.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on PR #8765:
URL: https://github.com/apache/nifi/pull/8765#issuecomment-2099186472

   Thanks @exceptionfactory ! Pushed updates, let me know of anything else


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on PR #8765:
URL: https://github.com/apache/nifi/pull/8765#issuecomment-2101392260

   @simonbence I pushed up some improvements based on your review, I think the permissions part is working much better now that I figured out how get the permissions that the client has to the repo, and can use them to set the bucket permissions.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "simonbence (via GitHub)" <gi...@apache.org>.
simonbence commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1593701552


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");

Review Comment:
   Looks like the `FlowAlreadyExistsException` would be a better fit for this situation



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);

Review Comment:
   If I get it right the `DEREGISTER_FLOW_COMMENT` will end up in the commit message. I think for making it easeier to follow it would make sense to add the flow id as well. (I did not try to run it at this point, it might be already implicitly included in the commit message)



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubRepositoryClient.java:
##########
@@ -0,0 +1,404 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+
+package org.apache.nifi.github;
+
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+import org.kohsuke.github.GHRef;
+import org.kohsuke.github.GHRepository;
+import org.kohsuke.github.GitHub;
+import org.kohsuke.github.GitHubBuilder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * Client to encapsulate access to a GitHub Repository through the Hub4j GitHub client.
+ */
+public class GitHubRepositoryClient {
+
+    private static final Logger LOGGER = LoggerFactory.getLogger(GitHubRepositoryClient.class);
+
+    private static final String BRANCH_REF_PATTERN = "refs/heads/%s";
+    private static final int COMMIT_PAGE_SIZE = 50;
+
+    private final GHRepository repository;
+    private final String repoPath;
+    private final GitHubAuthenticationType authenticationType;
+
+    private GitHubRepositoryClient(final Builder builder) throws IOException {
+        final GitHubBuilder gitHubBuilder = new GitHubBuilder().withEndpoint(builder.apiUrl);
+
+        authenticationType = builder.authenticationType;
+        switch (authenticationType) {
+            case PERSONAL_ACCESS_TOKEN -> gitHubBuilder.withOAuthToken(builder.personalAccessToken);
+            case APP_INSTALLATION_TOKEN -> gitHubBuilder.withAppInstallationToken(builder.appInstallationToken);
+        }
+
+        final GitHub gitHub = gitHubBuilder.build();
+        try {
+            repository = gitHub.getRepository(builder.repoOwner + "/" + builder.repoName);
+        } catch (final IOException e) {
+            LOGGER.error("Unable to access GitHub repository [{}]", builder.repoName, e);
+            throw e;
+        }
+        repoPath = builder.repoPath;
+    }
+
+    /**
+     * @return the authentication type this client is configured with
+     */
+    public GitHubAuthenticationType getAuthenticationType() {
+        return authenticationType;
+    }
+
+    /**
+     * Creates the content specified by the given builder.
+     *
+     * @param request the request for the content to create
+     * @return the update response
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public GHContentUpdateResponse createContent(final GitHubCreateContentRequest request) throws IOException, FlowRegistryException {
+        final String branch = request.getBranch();
+        final String resolvedPath = getResolvedPath(request.getPath());
+        LOGGER.debug("Creating content at path [{}] on branch [{}] in repo [{}] ", resolvedPath, branch, repository.getName());
+        return execute(() -> {
+            try {
+                return repository.createContent()
+                        .branch(branch)
+                        .path(resolvedPath)
+                        .content(request.getContent())
+                        .message(request.getMessage())
+                        .sha(request.getExistingContentSha())
+                        .commit();
+            } catch (final FileNotFoundException fnf) {
+                throwPathOrBranchNotFound(resolvedPath, branch);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the names of all the branches in the repo.
+     *
+     * @return the set of all branches in the repo
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public Set<String> getBranches() throws IOException, FlowRegistryException {
+        LOGGER.debug("Getting branches for repo [{}]", repository.getName());
+        return execute(() -> repository.getBranches().keySet());
+    }
+
+    /**
+     * Gets an InputStream to read the latest content of the given path from the given branch.
+     * The returned stream already contains the contents of the requested file.
+     *
+     * @param path the path to the content
+     * @param branch the branch
+     * @return an input stream containing the contents of the path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public InputStream getContentFromBranch(final String path, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting content for [{}] from branch [{}] in repo [{}] ", resolvedPath, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHContent ghContent = repository.getFileContent(resolvedPath, branchRef);
+                return ghContent.read();
+            } catch (final FileNotFoundException e) {
+                throwPathOrBranchNotFound(resolvedPath, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the content of the given path from the given commit.
+     * The returned stream already contains the contents of the requested file.
+     *
+     * @param path the path to the content
+     * @param commitSha the commit SHA
+     * @return an input stream containing the contents of the path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public InputStream getContentFromCommit(final String path, final String commitSha) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        LOGGER.debug("Getting content for [{}] from commit [{}] in repo [{}] ", resolvedPath, commitSha, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHContent ghContent = repository.getFileContent(resolvedPath, commitSha);
+                return ghContent.read();
+            } catch (final FileNotFoundException fnf) {
+                throw new FlowRegistryException("Path [" + resolvedPath + "] or Commit [" + commitSha + "] not found");
+            }
+        });
+    }
+
+    /**
+     * Gets the commits for a given path on a given branch.
+     *
+     * @param path the path
+     * @param branch the branch
+     * @return the list of commits for the given path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public List<GHCommit> getCommits(final String path, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting commits for [{}] from branch [{}] in repo [{}]", resolvedPath, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHRef branchGhRef = repository.getRef(branchRef);
+                return repository.queryCommits()
+                        .path(resolvedPath)
+                        .from(branchGhRef.getObject().getSha())
+                        .pageSize(COMMIT_PAGE_SIZE)

Review Comment:
   Will this return versions if there are more than 50 of them?



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);
+
+        final RegisteredFlowSnapshot deletedSnapshot = getSnapshot(deletedSnapshotContent.read());
+        updateBucketReferences(deletedSnapshot, flowLocation.getBucketId());
+        return deletedSnapshot.getFlow();
+    }
+
+    @Override
+    public RegisteredFlow getFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+        updateBucketReferences(existingSnapshot, flowLocation.getBucketId());
+
+        final RegisteredFlow registeredFlow = existingSnapshot.getFlow();
+        registeredFlow.setBranch(branch);
+        return registeredFlow;
+    }
+
+    @Override
+    public Set<RegisteredFlow> getFlows(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = bucketLocation.getBranch();
+        final String bucketId = bucketLocation.getBucketId();
+        final Set<RegisteredFlow> registeredFlows = new LinkedHashSet<>();
+
+        for (final String filename : repositoryClient.getFileNames(bucketId, branch)) {
+            if (!filename.endsWith(SNAPSHOT_FILE_EXTENSION)) {
+                continue;
+            }
+
+            final String flowId = filename.replace(SNAPSHOT_FILE_EXTENSION, "");
+            final RegisteredFlow registeredFlow = new RegisteredFlow();
+            registeredFlow.setIdentifier(flowId);
+            registeredFlow.setName(flowId);
+            registeredFlow.setBranch(branch);
+            registeredFlow.setBucketIdentifier(bucketId);
+            registeredFlow.setBucketName(bucketId);
+            registeredFlows.add(registeredFlow);
+        }
+
+        return registeredFlows;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot getFlowContents(final FlowRegistryClientConfigurationContext context, final FlowVersionLocation flowVersionLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String version = flowVersionLocation.getVersion();
+        final String filePath = getSnapshotFilePath(flowVersionLocation);
+
+        final InputStream inputStream = repositoryClient.getContentFromCommit(filePath, version);
+        final RegisteredFlowSnapshot flowSnapshot = getSnapshot(inputStream);
+        populateFlowAndSnapshotMetadata(flowSnapshot, flowVersionLocation);
+
+        // populate values that aren't store in GitHub
+        flowSnapshot.getSnapshotMetadata().setVersion(version);
+        flowSnapshot.getSnapshotMetadata().setBranch(flowVersionLocation.getBranch());
+        flowSnapshot.getFlow().setBranch(flowVersionLocation.getBranch());

Review Comment:
   I think it would be useful to have some tests in order to ensure this population/unpopulation (and the ones at `registerFlow` and `registerFlowSnapshot`) works correctly.
   
   With the current way of acquiring the `GitHubRepositoryClient` reference it does not look ideal for mocking, but if you do not want to funamentally change it, even with that `getRepositoryClient` might become a protected method and within a unit test, it can be overwritten to return a mock.
   
   This is not ideal, but other approaches (which I am completely open for) looks to come with much more work.



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubRepositoryClient.java:
##########
@@ -0,0 +1,404 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+
+package org.apache.nifi.github;
+
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+import org.kohsuke.github.GHRef;
+import org.kohsuke.github.GHRepository;
+import org.kohsuke.github.GitHub;
+import org.kohsuke.github.GitHubBuilder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * Client to encapsulate access to a GitHub Repository through the Hub4j GitHub client.
+ */
+public class GitHubRepositoryClient {
+
+    private static final Logger LOGGER = LoggerFactory.getLogger(GitHubRepositoryClient.class);
+
+    private static final String BRANCH_REF_PATTERN = "refs/heads/%s";
+    private static final int COMMIT_PAGE_SIZE = 50;
+
+    private final GHRepository repository;
+    private final String repoPath;
+    private final GitHubAuthenticationType authenticationType;
+
+    private GitHubRepositoryClient(final Builder builder) throws IOException {
+        final GitHubBuilder gitHubBuilder = new GitHubBuilder().withEndpoint(builder.apiUrl);
+
+        authenticationType = builder.authenticationType;
+        switch (authenticationType) {
+            case PERSONAL_ACCESS_TOKEN -> gitHubBuilder.withOAuthToken(builder.personalAccessToken);
+            case APP_INSTALLATION_TOKEN -> gitHubBuilder.withAppInstallationToken(builder.appInstallationToken);
+        }
+
+        final GitHub gitHub = gitHubBuilder.build();
+        try {
+            repository = gitHub.getRepository(builder.repoOwner + "/" + builder.repoName);
+        } catch (final IOException e) {
+            LOGGER.error("Unable to access GitHub repository [{}]", builder.repoName, e);
+            throw e;
+        }
+        repoPath = builder.repoPath;
+    }
+
+    /**
+     * @return the authentication type this client is configured with
+     */
+    public GitHubAuthenticationType getAuthenticationType() {
+        return authenticationType;
+    }
+
+    /**
+     * Creates the content specified by the given builder.
+     *
+     * @param request the request for the content to create
+     * @return the update response
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public GHContentUpdateResponse createContent(final GitHubCreateContentRequest request) throws IOException, FlowRegistryException {
+        final String branch = request.getBranch();
+        final String resolvedPath = getResolvedPath(request.getPath());
+        LOGGER.debug("Creating content at path [{}] on branch [{}] in repo [{}] ", resolvedPath, branch, repository.getName());
+        return execute(() -> {
+            try {
+                return repository.createContent()
+                        .branch(branch)
+                        .path(resolvedPath)
+                        .content(request.getContent())
+                        .message(request.getMessage())
+                        .sha(request.getExistingContentSha())
+                        .commit();
+            } catch (final FileNotFoundException fnf) {
+                throwPathOrBranchNotFound(resolvedPath, branch);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the names of all the branches in the repo.
+     *
+     * @return the set of all branches in the repo
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public Set<String> getBranches() throws IOException, FlowRegistryException {
+        LOGGER.debug("Getting branches for repo [{}]", repository.getName());
+        return execute(() -> repository.getBranches().keySet());
+    }
+
+    /**
+     * Gets an InputStream to read the latest content of the given path from the given branch.
+     * The returned stream already contains the contents of the requested file.
+     *
+     * @param path the path to the content
+     * @param branch the branch
+     * @return an input stream containing the contents of the path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public InputStream getContentFromBranch(final String path, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting content for [{}] from branch [{}] in repo [{}] ", resolvedPath, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHContent ghContent = repository.getFileContent(resolvedPath, branchRef);
+                return ghContent.read();
+            } catch (final FileNotFoundException e) {
+                throwPathOrBranchNotFound(resolvedPath, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the content of the given path from the given commit.
+     * The returned stream already contains the contents of the requested file.
+     *
+     * @param path the path to the content
+     * @param commitSha the commit SHA
+     * @return an input stream containing the contents of the path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public InputStream getContentFromCommit(final String path, final String commitSha) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        LOGGER.debug("Getting content for [{}] from commit [{}] in repo [{}] ", resolvedPath, commitSha, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHContent ghContent = repository.getFileContent(resolvedPath, commitSha);
+                return ghContent.read();
+            } catch (final FileNotFoundException fnf) {
+                throw new FlowRegistryException("Path [" + resolvedPath + "] or Commit [" + commitSha + "] not found");
+            }
+        });
+    }
+
+    /**
+     * Gets the commits for a given path on a given branch.
+     *
+     * @param path the path
+     * @param branch the branch
+     * @return the list of commits for the given path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public List<GHCommit> getCommits(final String path, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting commits for [{}] from branch [{}] in repo [{}]", resolvedPath, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHRef branchGhRef = repository.getRef(branchRef);
+                return repository.queryCommits()
+                        .path(resolvedPath)
+                        .from(branchGhRef.getObject().getSha())
+                        .pageSize(COMMIT_PAGE_SIZE)
+                        .list()
+                        .toList();
+            } catch (final FileNotFoundException fnf) {
+                throwPathOrBranchNotFound(resolvedPath, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the names of the directories contained within the given directory.
+     *
+     * @param directory the directory to list
+     * @param branch the branch
+     * @return the set of directory names
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public Set<String> getDirectoryNames(final String directory, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedDirectory = getResolvedPath(directory);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting directory names for [{}] from branch [{}] in repo [{}] ", resolvedDirectory, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                return repository.getDirectoryContent(resolvedDirectory, branchRef).stream()
+                        .filter(GHContent::isDirectory)
+                        .map(GHContent::getName)
+                        .collect(Collectors.toSet());
+            } catch (final FileNotFoundException e) {
+                throwPathOrBranchNotFound(resolvedDirectory, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the names of the directories container within the given directory.
+     *
+     * @param directory the directory to list
+     * @param branch the branch
+     * @return the set of file names
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public Set<String> getFileNames(final String directory, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedDirectory = getResolvedPath(directory);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting file names for [{}] from branch [{}] in repo [{}] ", resolvedDirectory, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                return repository.getDirectoryContent(resolvedDirectory, branchRef).stream()
+                        .filter(GHContent::isFile)
+                        .map(GHContent::getName)
+                        .collect(Collectors.toSet());
+            } catch (final FileNotFoundException e) {
+                throwPathOrBranchNotFound(resolvedDirectory, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the current SHA for the given path from the given branch.
+     *
+     * @param path the path to the content
+     * @param branch the branch
+     * @return current sha for the given file, or empty optional
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     */
+    public Optional<String> getContentSha(final String path, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting content SHA for [{}] from branch [{}] in repo [{}] ", resolvedPath, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHContent ghContent = repository.getFileContent(resolvedPath, branchRef);
+                return Optional.of(ghContent.getSha());
+            } catch (final FileNotFoundException e) {
+                LOGGER.warn("Unable to get content SHA for [{}] from branch [{}] because content does not exist", resolvedPath, branch);
+                return Optional.empty();
+            }
+        });
+    }
+
+    /**
+     * Deletes the contents for the given file on the given branch.
+     *
+     * @param filePath the file path to delete
+     * @param commitMessage the commit message for the delete commit
+     * @param branch the branch to delete from
+     * @return the deleted content
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public GHContent deleteContent(final String filePath, final String commitMessage, final String branch) throws FlowRegistryException, IOException {
+        final String resolvedPath = getResolvedPath(filePath);
+        LOGGER.debug("Deleting file [{}] in repo [{}] on branch [{}]", resolvedPath, repository.getName(), branch);
+        return execute(() -> {
+            try {
+                GHContent ghContent = repository.getFileContent(resolvedPath);
+                ghContent.delete(commitMessage, branch);
+                return ghContent;
+            } catch (final FileNotFoundException fnf) {
+                throwPathOrBranchNotFound(resolvedPath, branch);
+                return null;
+            }
+        });
+    }
+
+    private String getResolvedPath(final String path) {
+        return repoPath == null ? path : repoPath + "/" + path;
+    }
+
+    private void throwPathOrBranchNotFound(final String path, final String branch) throws FlowRegistryException {
+        throw new FlowRegistryException("Path [" + path + "] or Branch [" + branch + "] not found");
+    }
+
+    private <T> T execute(final GHRequest<T> action) throws FlowRegistryException, IOException {
+        try {
+            return action.execute();
+        } catch (final FlowRegistryException | IOException e) {
+            LOGGER.error(e.getMessage(), e);
+            throw e;
+        } catch (final Exception e) {
+            LOGGER.error(e.getMessage(), e);
+            throw new FlowRegistryException(e.getMessage(), e);
+        }
+    }
+
+    /**
+     * Functional interface for making a request to GitHub which may throw IOException.
+     *
+     * @param <T> the result of the request
+     */
+    private interface GHRequest<T> {
+
+        T execute() throws IOException, FlowRegistryException;
+
+    }
+
+    /**
+     * @return a new builder
+     */
+    public static Builder builder() {
+        return new Builder();
+    }
+
+    /**
+     * Builder for the repository client.
+     */
+    public static class Builder {
+
+        private String apiUrl;
+        private GitHubAuthenticationType authenticationType;
+        private String personalAccessToken;
+        private String appInstallationToken;
+        private String repoOwner;
+        private String repoName;
+        private String repoPath;
+
+        public Builder apiUrl(final String apiUrl) {
+            this.apiUrl = apiUrl;
+            return this;
+        }
+
+        public Builder authenticationType(final GitHubAuthenticationType authenticationType) {
+            this.authenticationType = authenticationType;
+            return this;
+        }
+
+        public Builder personalAccessToken(final String personalAccessToken) {
+            this.personalAccessToken = personalAccessToken;
+            return this;
+        }
+
+        public Builder appInstallationToken(final String appInstallationToken) {
+            this.appInstallationToken = appInstallationToken;
+            return this;
+        }
+
+        public Builder repoOwner(final String repoOwner) {
+            this.repoOwner = repoOwner;
+            return this;
+        }
+
+        public Builder repoName(final String repoName) {
+            this.repoName = repoName;
+            return this;
+        }
+
+        public Builder repoPath(final String repoPath) {
+            this.repoPath = repoPath;
+            return this;
+        }
+
+        public GitHubRepositoryClient build() throws IOException {

Review Comment:
   Minor: if we consider some attributes mandatory, this can be a good place to validate if they are set.



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());

Review Comment:
   If I get it right, this method creates a bucket DTO regardless of it's existence. I would prefer to see a check on it and throwing `NoSuchBucketException` if there is no such one



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);
+
+        final RegisteredFlowSnapshot deletedSnapshot = getSnapshot(deletedSnapshotContent.read());
+        updateBucketReferences(deletedSnapshot, flowLocation.getBucketId());
+        return deletedSnapshot.getFlow();
+    }
+
+    @Override
+    public RegisteredFlow getFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+        updateBucketReferences(existingSnapshot, flowLocation.getBucketId());
+
+        final RegisteredFlow registeredFlow = existingSnapshot.getFlow();
+        registeredFlow.setBranch(branch);
+        return registeredFlow;
+    }
+
+    @Override
+    public Set<RegisteredFlow> getFlows(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = bucketLocation.getBranch();
+        final String bucketId = bucketLocation.getBucketId();
+        final Set<RegisteredFlow> registeredFlows = new LinkedHashSet<>();
+
+        for (final String filename : repositoryClient.getFileNames(bucketId, branch)) {
+            if (!filename.endsWith(SNAPSHOT_FILE_EXTENSION)) {
+                continue;
+            }
+
+            final String flowId = filename.replace(SNAPSHOT_FILE_EXTENSION, "");
+            final RegisteredFlow registeredFlow = new RegisteredFlow();
+            registeredFlow.setIdentifier(flowId);
+            registeredFlow.setName(flowId);
+            registeredFlow.setBranch(branch);
+            registeredFlow.setBucketIdentifier(bucketId);
+            registeredFlow.setBucketName(bucketId);
+            registeredFlows.add(registeredFlow);
+        }
+
+        return registeredFlows;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot getFlowContents(final FlowRegistryClientConfigurationContext context, final FlowVersionLocation flowVersionLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String version = flowVersionLocation.getVersion();
+        final String filePath = getSnapshotFilePath(flowVersionLocation);
+
+        final InputStream inputStream = repositoryClient.getContentFromCommit(filePath, version);
+        final RegisteredFlowSnapshot flowSnapshot = getSnapshot(inputStream);
+        populateFlowAndSnapshotMetadata(flowSnapshot, flowVersionLocation);
+
+        // populate values that aren't store in GitHub
+        flowSnapshot.getSnapshotMetadata().setVersion(version);
+        flowSnapshot.getSnapshotMetadata().setBranch(flowVersionLocation.getBranch());
+        flowSnapshot.getFlow().setBranch(flowVersionLocation.getBranch());
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowVersionLocation.getBucketId());
+
+        // determine if the version is the "latest" version by comparing to the response of getLatestVersion
+        final String latestVersion = getLatestVersion(context, flowVersionLocation).orElse(null);
+        flowSnapshot.setLatest(version.equals(latestVersion));
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot registerFlowSnapshot(final FlowRegistryClientConfigurationContext context, final RegisteredFlowSnapshot flowSnapshot, final RegisterAction action)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = flowSnapshot.getSnapshotMetadata();
+
+        final String branch = snapshotMetadata.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(snapshotMetadata.getBranch(), snapshotMetadata.getBucketIdentifier(), snapshotMetadata.getFlowIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final String previousSha = repositoryClient.getContentSha(filePath, branch).orElse(null);
+
+        final String snapshotComments = snapshotMetadata.getComments();
+        final String commitMessage = StringUtils.isBlank(snapshotComments) ? DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE : snapshotComments;
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+
+        final RegisteredFlow existingFlow = existingSnapshot.getFlow();
+        existingFlow.setBranch(null);
+        flowSnapshot.setFlow(existingFlow);
+
+        // Clear values we don't want stored in the json in GitHub
+        flowSnapshot.setBucket(null);
+        flowSnapshot.getSnapshotMetadata().setBucketIdentifier(null);
+        flowSnapshot.getSnapshotMetadata().setBranch(null);
+        flowSnapshot.getSnapshotMetadata().setVersion(null);
+        flowSnapshot.getSnapshotMetadata().setComments(null);
+        flowSnapshot.getSnapshotMetadata().setTimestamp(0);

Review Comment:
   Is not timestemp we want to store? I think alternatively we can go with the timestamp of the commit itself, but as far as I found, this is set only in case of `getFlowVersions` not at the other methods where we try to acquire flow or version (every `RegisteredFlowSnapshot` contains the same metadata type as well)



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);
+
+        final RegisteredFlowSnapshot deletedSnapshot = getSnapshot(deletedSnapshotContent.read());
+        updateBucketReferences(deletedSnapshot, flowLocation.getBucketId());
+        return deletedSnapshot.getFlow();
+    }
+
+    @Override
+    public RegisteredFlow getFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+        updateBucketReferences(existingSnapshot, flowLocation.getBucketId());
+
+        final RegisteredFlow registeredFlow = existingSnapshot.getFlow();
+        registeredFlow.setBranch(branch);
+        return registeredFlow;
+    }
+
+    @Override
+    public Set<RegisteredFlow> getFlows(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = bucketLocation.getBranch();
+        final String bucketId = bucketLocation.getBucketId();
+        final Set<RegisteredFlow> registeredFlows = new LinkedHashSet<>();
+
+        for (final String filename : repositoryClient.getFileNames(bucketId, branch)) {
+            if (!filename.endsWith(SNAPSHOT_FILE_EXTENSION)) {
+                continue;
+            }
+
+            final String flowId = filename.replace(SNAPSHOT_FILE_EXTENSION, "");
+            final RegisteredFlow registeredFlow = new RegisteredFlow();
+            registeredFlow.setIdentifier(flowId);
+            registeredFlow.setName(flowId);
+            registeredFlow.setBranch(branch);
+            registeredFlow.setBucketIdentifier(bucketId);
+            registeredFlow.setBucketName(bucketId);
+            registeredFlows.add(registeredFlow);
+        }
+
+        return registeredFlows;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot getFlowContents(final FlowRegistryClientConfigurationContext context, final FlowVersionLocation flowVersionLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String version = flowVersionLocation.getVersion();
+        final String filePath = getSnapshotFilePath(flowVersionLocation);
+
+        final InputStream inputStream = repositoryClient.getContentFromCommit(filePath, version);
+        final RegisteredFlowSnapshot flowSnapshot = getSnapshot(inputStream);
+        populateFlowAndSnapshotMetadata(flowSnapshot, flowVersionLocation);
+
+        // populate values that aren't store in GitHub
+        flowSnapshot.getSnapshotMetadata().setVersion(version);
+        flowSnapshot.getSnapshotMetadata().setBranch(flowVersionLocation.getBranch());
+        flowSnapshot.getFlow().setBranch(flowVersionLocation.getBranch());
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowVersionLocation.getBucketId());
+
+        // determine if the version is the "latest" version by comparing to the response of getLatestVersion
+        final String latestVersion = getLatestVersion(context, flowVersionLocation).orElse(null);
+        flowSnapshot.setLatest(version.equals(latestVersion));
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot registerFlowSnapshot(final FlowRegistryClientConfigurationContext context, final RegisteredFlowSnapshot flowSnapshot, final RegisterAction action)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = flowSnapshot.getSnapshotMetadata();
+
+        final String branch = snapshotMetadata.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(snapshotMetadata.getBranch(), snapshotMetadata.getBucketIdentifier(), snapshotMetadata.getFlowIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final String previousSha = repositoryClient.getContentSha(filePath, branch).orElse(null);
+
+        final String snapshotComments = snapshotMetadata.getComments();
+        final String commitMessage = StringUtils.isBlank(snapshotComments) ? DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE : snapshotComments;
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+
+        final RegisteredFlow existingFlow = existingSnapshot.getFlow();
+        existingFlow.setBranch(null);
+        flowSnapshot.setFlow(existingFlow);
+
+        // Clear values we don't want stored in the json in GitHub
+        flowSnapshot.setBucket(null);
+        flowSnapshot.getSnapshotMetadata().setBucketIdentifier(null);
+        flowSnapshot.getSnapshotMetadata().setBranch(null);
+        flowSnapshot.getSnapshotMetadata().setVersion(null);
+        flowSnapshot.getSnapshotMetadata().setComments(null);
+        flowSnapshot.getSnapshotMetadata().setTimestamp(0);
+
+        // replace the id of the top level group and all of its references with a constant value prior to serializing to avoid
+        // unnecessary diffs when different instances of the same flow are imported and have different top-level PG ids
+        final String originalFlowContentsGroupId = replaceGroupId(flowSnapshot.getFlowContents(), FLOW_CONTENTS_GROUP_ID);
+        final Position originalFlowContentsPosition = replacePosition(flowSnapshot.getFlowContents(), new Position(0, 0));
+
+        final GitHubCreateContentRequest createContentRequest = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(commitMessage)
+                .existingContentSha(previousSha)
+                .build();
+
+        final GHContentUpdateResponse createContentResponse = repositoryClient.createContent(createContentRequest);
+        final String createContentCommitSha = createContentResponse.getCommit().getSha();
+
+        final VersionedFlowCoordinates versionedFlowCoordinates = new VersionedFlowCoordinates();
+        versionedFlowCoordinates.setRegistryId(getIdentifier());
+        versionedFlowCoordinates.setBranch(flowLocation.getBranch());
+        versionedFlowCoordinates.setBucketId(flowLocation.getBucketId());
+        versionedFlowCoordinates.setFlowId(flowLocation.getFlowId());
+        versionedFlowCoordinates.setVersion(createContentCommitSha);
+
+        flowSnapshot.getFlowContents().setVersionedFlowCoordinates(versionedFlowCoordinates);
+        flowSnapshot.getFlow().setBranch(branch);
+        flowSnapshot.getSnapshotMetadata().setBranch(branch);
+        flowSnapshot.getSnapshotMetadata().setVersion(createContentCommitSha);
+        flowSnapshot.setLatest(true);
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowLocation.getBucketId());
+
+        // set back to the original id so that the returned snapshot is has the correct values from what was passed in
+        replaceGroupId(flowSnapshot.getFlowContents(), originalFlowContentsGroupId);
+        replacePosition(flowSnapshot.getFlowContents(), originalFlowContentsPosition);
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public Set<RegisteredFlowSnapshotMetadata> getFlowVersions(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Set<RegisteredFlowSnapshotMetadata> snapshotMetadataSet = new LinkedHashSet<>();
+        for (final GHCommit ghCommit : repositoryClient.getCommits(filePath, branch)) {
+            final RegisteredFlowSnapshotMetadata snapshotMetadata = createSnapshotMetadata(ghCommit, flowLocation);
+            if (REGISTER_FLOW_COMMENT.equals(snapshotMetadata.getComments())) {
+                continue;
+            }
+            snapshotMetadataSet.add(snapshotMetadata);
+        }
+        return snapshotMetadataSet;
+    }
+
+    @Override
+    public Optional<String> getLatestVersion(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final List<GHCommit> commits = repositoryClient.getCommits(filePath, branch);
+        final String latestVersion = commits.isEmpty() ? null : commits.getFirst().getSHA1();
+        return Optional.ofNullable(latestVersion);
+    }
+
+    @Override
+    public String generateFlowId(final String flowName) {
+        return flowName.trim()
+                .replaceAll("\\s", "-") // replace whitespace with -
+                .replaceAll("[^a-zA-Z0-9-]", "") // replace all other invalid chars with empty string
+                .replaceAll("(-)\\1+", "$1"); // replace consecutive - with single -

Review Comment:
   Can any type of collusion happen here? For example: I want a flow with the name of `flow 1` and `flow-1`, but the ids will collude. Do we want to prevent this kind of behaviour?



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);
+
+        final RegisteredFlowSnapshot deletedSnapshot = getSnapshot(deletedSnapshotContent.read());
+        updateBucketReferences(deletedSnapshot, flowLocation.getBucketId());
+        return deletedSnapshot.getFlow();
+    }
+
+    @Override
+    public RegisteredFlow getFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);

Review Comment:
   The NiFiRegistry implemnetation utilizes a `NoSuchFlowException` for the cases the flow could not ne find. I think it would be beneficial to have something similar.



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;

Review Comment:
   With the current handling of this even with "false" value the `StandardFlowRegistryClientNode` will eventually try to look in this repository but the whole point of this method is to make it easier to the node to find the source of nested flows. Thus it would be good to have a "heuristical" implementation for this.



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);
+
+        final RegisteredFlowSnapshot deletedSnapshot = getSnapshot(deletedSnapshotContent.read());
+        updateBucketReferences(deletedSnapshot, flowLocation.getBucketId());
+        return deletedSnapshot.getFlow();
+    }
+
+    @Override
+    public RegisteredFlow getFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+        updateBucketReferences(existingSnapshot, flowLocation.getBucketId());
+
+        final RegisteredFlow registeredFlow = existingSnapshot.getFlow();
+        registeredFlow.setBranch(branch);
+        return registeredFlow;
+    }
+
+    @Override
+    public Set<RegisteredFlow> getFlows(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = bucketLocation.getBranch();
+        final String bucketId = bucketLocation.getBucketId();
+        final Set<RegisteredFlow> registeredFlows = new LinkedHashSet<>();
+
+        for (final String filename : repositoryClient.getFileNames(bucketId, branch)) {
+            if (!filename.endsWith(SNAPSHOT_FILE_EXTENSION)) {
+                continue;
+            }
+
+            final String flowId = filename.replace(SNAPSHOT_FILE_EXTENSION, "");
+            final RegisteredFlow registeredFlow = new RegisteredFlow();
+            registeredFlow.setIdentifier(flowId);
+            registeredFlow.setName(flowId);
+            registeredFlow.setBranch(branch);
+            registeredFlow.setBucketIdentifier(bucketId);
+            registeredFlow.setBucketName(bucketId);
+            registeredFlows.add(registeredFlow);
+        }
+
+        return registeredFlows;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot getFlowContents(final FlowRegistryClientConfigurationContext context, final FlowVersionLocation flowVersionLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String version = flowVersionLocation.getVersion();
+        final String filePath = getSnapshotFilePath(flowVersionLocation);
+
+        final InputStream inputStream = repositoryClient.getContentFromCommit(filePath, version);
+        final RegisteredFlowSnapshot flowSnapshot = getSnapshot(inputStream);
+        populateFlowAndSnapshotMetadata(flowSnapshot, flowVersionLocation);
+
+        // populate values that aren't store in GitHub
+        flowSnapshot.getSnapshotMetadata().setVersion(version);
+        flowSnapshot.getSnapshotMetadata().setBranch(flowVersionLocation.getBranch());
+        flowSnapshot.getFlow().setBranch(flowVersionLocation.getBranch());
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowVersionLocation.getBucketId());
+
+        // determine if the version is the "latest" version by comparing to the response of getLatestVersion
+        final String latestVersion = getLatestVersion(context, flowVersionLocation).orElse(null);
+        flowSnapshot.setLatest(version.equals(latestVersion));
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot registerFlowSnapshot(final FlowRegistryClientConfigurationContext context, final RegisteredFlowSnapshot flowSnapshot, final RegisterAction action)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = flowSnapshot.getSnapshotMetadata();
+
+        final String branch = snapshotMetadata.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(snapshotMetadata.getBranch(), snapshotMetadata.getBucketIdentifier(), snapshotMetadata.getFlowIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final String previousSha = repositoryClient.getContentSha(filePath, branch).orElse(null);
+
+        final String snapshotComments = snapshotMetadata.getComments();
+        final String commitMessage = StringUtils.isBlank(snapshotComments) ? DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE : snapshotComments;

Review Comment:
   Like above: in case it is not implicitly added, I suggest to include information about the snapshot/flow itself.



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);
+
+        final RegisteredFlowSnapshot deletedSnapshot = getSnapshot(deletedSnapshotContent.read());
+        updateBucketReferences(deletedSnapshot, flowLocation.getBucketId());
+        return deletedSnapshot.getFlow();
+    }
+
+    @Override
+    public RegisteredFlow getFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+        updateBucketReferences(existingSnapshot, flowLocation.getBucketId());
+
+        final RegisteredFlow registeredFlow = existingSnapshot.getFlow();
+        registeredFlow.setBranch(branch);
+        return registeredFlow;
+    }
+
+    @Override
+    public Set<RegisteredFlow> getFlows(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = bucketLocation.getBranch();
+        final String bucketId = bucketLocation.getBucketId();
+        final Set<RegisteredFlow> registeredFlows = new LinkedHashSet<>();
+
+        for (final String filename : repositoryClient.getFileNames(bucketId, branch)) {
+            if (!filename.endsWith(SNAPSHOT_FILE_EXTENSION)) {
+                continue;
+            }
+
+            final String flowId = filename.replace(SNAPSHOT_FILE_EXTENSION, "");
+            final RegisteredFlow registeredFlow = new RegisteredFlow();
+            registeredFlow.setIdentifier(flowId);
+            registeredFlow.setName(flowId);
+            registeredFlow.setBranch(branch);
+            registeredFlow.setBucketIdentifier(bucketId);
+            registeredFlow.setBucketName(bucketId);
+            registeredFlows.add(registeredFlow);
+        }
+
+        return registeredFlows;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot getFlowContents(final FlowRegistryClientConfigurationContext context, final FlowVersionLocation flowVersionLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String version = flowVersionLocation.getVersion();
+        final String filePath = getSnapshotFilePath(flowVersionLocation);
+
+        final InputStream inputStream = repositoryClient.getContentFromCommit(filePath, version);

Review Comment:
   I might be wrong, but I do not see how non-existing snasphots are handled. Please ignore if there is a way, if not, then I suggest throwing `NoSuchFlowVersionException`



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);

Review Comment:
   Should not be the bucket set back after the `creteContent` using `updateBucketReferences` or alike?



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubRepositoryClient.java:
##########
@@ -0,0 +1,404 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+
+package org.apache.nifi.github;
+
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+import org.kohsuke.github.GHRef;
+import org.kohsuke.github.GHRepository;
+import org.kohsuke.github.GitHub;
+import org.kohsuke.github.GitHubBuilder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * Client to encapsulate access to a GitHub Repository through the Hub4j GitHub client.
+ */
+public class GitHubRepositoryClient {
+
+    private static final Logger LOGGER = LoggerFactory.getLogger(GitHubRepositoryClient.class);
+
+    private static final String BRANCH_REF_PATTERN = "refs/heads/%s";
+    private static final int COMMIT_PAGE_SIZE = 50;
+
+    private final GHRepository repository;
+    private final String repoPath;
+    private final GitHubAuthenticationType authenticationType;
+
+    private GitHubRepositoryClient(final Builder builder) throws IOException {
+        final GitHubBuilder gitHubBuilder = new GitHubBuilder().withEndpoint(builder.apiUrl);
+
+        authenticationType = builder.authenticationType;
+        switch (authenticationType) {
+            case PERSONAL_ACCESS_TOKEN -> gitHubBuilder.withOAuthToken(builder.personalAccessToken);
+            case APP_INSTALLATION_TOKEN -> gitHubBuilder.withAppInstallationToken(builder.appInstallationToken);

Review Comment:
   Are these two might be used together?



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubRepositoryClient.java:
##########
@@ -0,0 +1,404 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+
+package org.apache.nifi.github;
+
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+import org.kohsuke.github.GHRef;
+import org.kohsuke.github.GHRepository;
+import org.kohsuke.github.GitHub;
+import org.kohsuke.github.GitHubBuilder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * Client to encapsulate access to a GitHub Repository through the Hub4j GitHub client.
+ */
+public class GitHubRepositoryClient {

Review Comment:
   I am not sure if it is ever in the horizont to change the underlying library or if we expect any change in the library`s API but in order to avoid rippling through these possible effects, I would suggest to fully wrap the dependencies from the external dependencies and ony accept/return our version (which is probably a subset of these classes both in number and in methods).



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);
+
+        final RegisteredFlowSnapshot deletedSnapshot = getSnapshot(deletedSnapshotContent.read());
+        updateBucketReferences(deletedSnapshot, flowLocation.getBucketId());
+        return deletedSnapshot.getFlow();
+    }
+
+    @Override
+    public RegisteredFlow getFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+        updateBucketReferences(existingSnapshot, flowLocation.getBucketId());
+
+        final RegisteredFlow registeredFlow = existingSnapshot.getFlow();
+        registeredFlow.setBranch(branch);
+        return registeredFlow;
+    }
+
+    @Override
+    public Set<RegisteredFlow> getFlows(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = bucketLocation.getBranch();
+        final String bucketId = bucketLocation.getBucketId();
+        final Set<RegisteredFlow> registeredFlows = new LinkedHashSet<>();
+
+        for (final String filename : repositoryClient.getFileNames(bucketId, branch)) {

Review Comment:
   Minor: it might be a bit more readable if a stream filter would be used, something like:
   ```
   repositoryClient.getFileNames(bucketId, branch).stream()
       .filter(fn -> filename.endsWith(SNAPSHOT_FILE_EXTENSION))
       .map(...)
       .collect(Collectors.toSet());
   ```
   
   Where the map could be a even method reference.



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);
+
+        final RegisteredFlowSnapshot deletedSnapshot = getSnapshot(deletedSnapshotContent.read());
+        updateBucketReferences(deletedSnapshot, flowLocation.getBucketId());
+        return deletedSnapshot.getFlow();
+    }
+
+    @Override
+    public RegisteredFlow getFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+        updateBucketReferences(existingSnapshot, flowLocation.getBucketId());
+
+        final RegisteredFlow registeredFlow = existingSnapshot.getFlow();
+        registeredFlow.setBranch(branch);
+        return registeredFlow;
+    }
+
+    @Override
+    public Set<RegisteredFlow> getFlows(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = bucketLocation.getBranch();
+        final String bucketId = bucketLocation.getBucketId();
+        final Set<RegisteredFlow> registeredFlows = new LinkedHashSet<>();
+
+        for (final String filename : repositoryClient.getFileNames(bucketId, branch)) {
+            if (!filename.endsWith(SNAPSHOT_FILE_EXTENSION)) {
+                continue;
+            }
+
+            final String flowId = filename.replace(SNAPSHOT_FILE_EXTENSION, "");
+            final RegisteredFlow registeredFlow = new RegisteredFlow();
+            registeredFlow.setIdentifier(flowId);
+            registeredFlow.setName(flowId);
+            registeredFlow.setBranch(branch);
+            registeredFlow.setBucketIdentifier(bucketId);
+            registeredFlow.setBucketName(bucketId);
+            registeredFlows.add(registeredFlow);
+        }
+
+        return registeredFlows;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot getFlowContents(final FlowRegistryClientConfigurationContext context, final FlowVersionLocation flowVersionLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String version = flowVersionLocation.getVersion();
+        final String filePath = getSnapshotFilePath(flowVersionLocation);
+
+        final InputStream inputStream = repositoryClient.getContentFromCommit(filePath, version);
+        final RegisteredFlowSnapshot flowSnapshot = getSnapshot(inputStream);
+        populateFlowAndSnapshotMetadata(flowSnapshot, flowVersionLocation);
+
+        // populate values that aren't store in GitHub
+        flowSnapshot.getSnapshotMetadata().setVersion(version);
+        flowSnapshot.getSnapshotMetadata().setBranch(flowVersionLocation.getBranch());
+        flowSnapshot.getFlow().setBranch(flowVersionLocation.getBranch());
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowVersionLocation.getBucketId());
+
+        // determine if the version is the "latest" version by comparing to the response of getLatestVersion
+        final String latestVersion = getLatestVersion(context, flowVersionLocation).orElse(null);
+        flowSnapshot.setLatest(version.equals(latestVersion));
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot registerFlowSnapshot(final FlowRegistryClientConfigurationContext context, final RegisteredFlowSnapshot flowSnapshot, final RegisterAction action)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = flowSnapshot.getSnapshotMetadata();
+
+        final String branch = snapshotMetadata.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(snapshotMetadata.getBranch(), snapshotMetadata.getBucketIdentifier(), snapshotMetadata.getFlowIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final String previousSha = repositoryClient.getContentSha(filePath, branch).orElse(null);
+
+        final String snapshotComments = snapshotMetadata.getComments();
+        final String commitMessage = StringUtils.isBlank(snapshotComments) ? DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE : snapshotComments;
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+
+        final RegisteredFlow existingFlow = existingSnapshot.getFlow();
+        existingFlow.setBranch(null);
+        flowSnapshot.setFlow(existingFlow);
+
+        // Clear values we don't want stored in the json in GitHub
+        flowSnapshot.setBucket(null);
+        flowSnapshot.getSnapshotMetadata().setBucketIdentifier(null);
+        flowSnapshot.getSnapshotMetadata().setBranch(null);
+        flowSnapshot.getSnapshotMetadata().setVersion(null);
+        flowSnapshot.getSnapshotMetadata().setComments(null);
+        flowSnapshot.getSnapshotMetadata().setTimestamp(0);
+
+        // replace the id of the top level group and all of its references with a constant value prior to serializing to avoid
+        // unnecessary diffs when different instances of the same flow are imported and have different top-level PG ids
+        final String originalFlowContentsGroupId = replaceGroupId(flowSnapshot.getFlowContents(), FLOW_CONTENTS_GROUP_ID);
+        final Position originalFlowContentsPosition = replacePosition(flowSnapshot.getFlowContents(), new Position(0, 0));
+
+        final GitHubCreateContentRequest createContentRequest = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(commitMessage)
+                .existingContentSha(previousSha)
+                .build();
+
+        final GHContentUpdateResponse createContentResponse = repositoryClient.createContent(createContentRequest);
+        final String createContentCommitSha = createContentResponse.getCommit().getSha();
+
+        final VersionedFlowCoordinates versionedFlowCoordinates = new VersionedFlowCoordinates();
+        versionedFlowCoordinates.setRegistryId(getIdentifier());
+        versionedFlowCoordinates.setBranch(flowLocation.getBranch());
+        versionedFlowCoordinates.setBucketId(flowLocation.getBucketId());
+        versionedFlowCoordinates.setFlowId(flowLocation.getFlowId());
+        versionedFlowCoordinates.setVersion(createContentCommitSha);
+
+        flowSnapshot.getFlowContents().setVersionedFlowCoordinates(versionedFlowCoordinates);
+        flowSnapshot.getFlow().setBranch(branch);
+        flowSnapshot.getSnapshotMetadata().setBranch(branch);
+        flowSnapshot.getSnapshotMetadata().setVersion(createContentCommitSha);
+        flowSnapshot.setLatest(true);
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowLocation.getBucketId());
+
+        // set back to the original id so that the returned snapshot is has the correct values from what was passed in
+        replaceGroupId(flowSnapshot.getFlowContents(), originalFlowContentsGroupId);
+        replacePosition(flowSnapshot.getFlowContents(), originalFlowContentsPosition);
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public Set<RegisteredFlowSnapshotMetadata> getFlowVersions(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Set<RegisteredFlowSnapshotMetadata> snapshotMetadataSet = new LinkedHashSet<>();
+        for (final GHCommit ghCommit : repositoryClient.getCommits(filePath, branch)) {
+            final RegisteredFlowSnapshotMetadata snapshotMetadata = createSnapshotMetadata(ghCommit, flowLocation);
+            if (REGISTER_FLOW_COMMENT.equals(snapshotMetadata.getComments())) {
+                continue;
+            }
+            snapshotMetadataSet.add(snapshotMetadata);
+        }
+        return snapshotMetadataSet;
+    }
+
+    @Override
+    public Optional<String> getLatestVersion(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final List<GHCommit> commits = repositoryClient.getCommits(filePath, branch);
+        final String latestVersion = commits.isEmpty() ? null : commits.getFirst().getSHA1();
+        return Optional.ofNullable(latestVersion);
+    }
+
+    @Override
+    public String generateFlowId(final String flowName) {
+        return flowName.trim()
+                .replaceAll("\\s", "-") // replace whitespace with -
+                .replaceAll("[^a-zA-Z0-9-]", "") // replace all other invalid chars with empty string
+                .replaceAll("(-)\\1+", "$1"); // replace consecutive - with single -
+    }
+
+    private FlowRegistryBucket createFlowRegistryBucket(final String name) {
+        final FlowRegistryPermissions bucketPermissions = new FlowRegistryPermissions();
+        bucketPermissions.setCanRead(true);

Review Comment:
   Is not there any permission control with the repository? I am aware of that the access is covered by the usage of the AUTHENTICATION_TYPE but if I get it correctly it might happen that within Github the user has no rights to write or delete but here we return that he has. As far as I remember the UI works based on this and lets you take action based on this settings which might result a case where the UI allows an action which migh fail due to lack of privileges.



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubRepositoryClient.java:
##########
@@ -0,0 +1,404 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+
+package org.apache.nifi.github;
+
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+import org.kohsuke.github.GHRef;
+import org.kohsuke.github.GHRepository;
+import org.kohsuke.github.GitHub;
+import org.kohsuke.github.GitHubBuilder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * Client to encapsulate access to a GitHub Repository through the Hub4j GitHub client.
+ */
+public class GitHubRepositoryClient {
+
+    private static final Logger LOGGER = LoggerFactory.getLogger(GitHubRepositoryClient.class);
+
+    private static final String BRANCH_REF_PATTERN = "refs/heads/%s";
+    private static final int COMMIT_PAGE_SIZE = 50;
+
+    private final GHRepository repository;
+    private final String repoPath;
+    private final GitHubAuthenticationType authenticationType;
+
+    private GitHubRepositoryClient(final Builder builder) throws IOException {
+        final GitHubBuilder gitHubBuilder = new GitHubBuilder().withEndpoint(builder.apiUrl);
+
+        authenticationType = builder.authenticationType;
+        switch (authenticationType) {
+            case PERSONAL_ACCESS_TOKEN -> gitHubBuilder.withOAuthToken(builder.personalAccessToken);
+            case APP_INSTALLATION_TOKEN -> gitHubBuilder.withAppInstallationToken(builder.appInstallationToken);
+        }
+
+        final GitHub gitHub = gitHubBuilder.build();
+        try {
+            repository = gitHub.getRepository(builder.repoOwner + "/" + builder.repoName);
+        } catch (final IOException e) {
+            LOGGER.error("Unable to access GitHub repository [{}]", builder.repoName, e);
+            throw e;
+        }
+        repoPath = builder.repoPath;
+    }
+
+    /**
+     * @return the authentication type this client is configured with
+     */
+    public GitHubAuthenticationType getAuthenticationType() {
+        return authenticationType;
+    }
+
+    /**
+     * Creates the content specified by the given builder.
+     *
+     * @param request the request for the content to create
+     * @return the update response
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public GHContentUpdateResponse createContent(final GitHubCreateContentRequest request) throws IOException, FlowRegistryException {
+        final String branch = request.getBranch();
+        final String resolvedPath = getResolvedPath(request.getPath());
+        LOGGER.debug("Creating content at path [{}] on branch [{}] in repo [{}] ", resolvedPath, branch, repository.getName());
+        return execute(() -> {
+            try {
+                return repository.createContent()
+                        .branch(branch)
+                        .path(resolvedPath)
+                        .content(request.getContent())
+                        .message(request.getMessage())
+                        .sha(request.getExistingContentSha())
+                        .commit();
+            } catch (final FileNotFoundException fnf) {
+                throwPathOrBranchNotFound(resolvedPath, branch);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the names of all the branches in the repo.
+     *
+     * @return the set of all branches in the repo
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public Set<String> getBranches() throws IOException, FlowRegistryException {
+        LOGGER.debug("Getting branches for repo [{}]", repository.getName());
+        return execute(() -> repository.getBranches().keySet());
+    }
+
+    /**
+     * Gets an InputStream to read the latest content of the given path from the given branch.
+     * The returned stream already contains the contents of the requested file.
+     *
+     * @param path the path to the content
+     * @param branch the branch
+     * @return an input stream containing the contents of the path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public InputStream getContentFromBranch(final String path, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting content for [{}] from branch [{}] in repo [{}] ", resolvedPath, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHContent ghContent = repository.getFileContent(resolvedPath, branchRef);
+                return ghContent.read();
+            } catch (final FileNotFoundException e) {
+                throwPathOrBranchNotFound(resolvedPath, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the content of the given path from the given commit.
+     * The returned stream already contains the contents of the requested file.
+     *
+     * @param path the path to the content
+     * @param commitSha the commit SHA
+     * @return an input stream containing the contents of the path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public InputStream getContentFromCommit(final String path, final String commitSha) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        LOGGER.debug("Getting content for [{}] from commit [{}] in repo [{}] ", resolvedPath, commitSha, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHContent ghContent = repository.getFileContent(resolvedPath, commitSha);
+                return ghContent.read();
+            } catch (final FileNotFoundException fnf) {
+                throw new FlowRegistryException("Path [" + resolvedPath + "] or Commit [" + commitSha + "] not found");
+            }
+        });
+    }
+
+    /**
+     * Gets the commits for a given path on a given branch.
+     *
+     * @param path the path
+     * @param branch the branch
+     * @return the list of commits for the given path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public List<GHCommit> getCommits(final String path, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting commits for [{}] from branch [{}] in repo [{}]", resolvedPath, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHRef branchGhRef = repository.getRef(branchRef);
+                return repository.queryCommits()
+                        .path(resolvedPath)
+                        .from(branchGhRef.getObject().getSha())
+                        .pageSize(COMMIT_PAGE_SIZE)
+                        .list()
+                        .toList();
+            } catch (final FileNotFoundException fnf) {
+                throwPathOrBranchNotFound(resolvedPath, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the names of the directories contained within the given directory.
+     *
+     * @param directory the directory to list
+     * @param branch the branch
+     * @return the set of directory names
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public Set<String> getDirectoryNames(final String directory, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedDirectory = getResolvedPath(directory);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting directory names for [{}] from branch [{}] in repo [{}] ", resolvedDirectory, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                return repository.getDirectoryContent(resolvedDirectory, branchRef).stream()
+                        .filter(GHContent::isDirectory)
+                        .map(GHContent::getName)
+                        .collect(Collectors.toSet());
+            } catch (final FileNotFoundException e) {
+                throwPathOrBranchNotFound(resolvedDirectory, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the names of the directories container within the given directory.
+     *
+     * @param directory the directory to list
+     * @param branch the branch
+     * @return the set of file names
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public Set<String> getFileNames(final String directory, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedDirectory = getResolvedPath(directory);

Review Comment:
   It looks to be the same as `getDirectoryNames` expect the filter. It might worth to merge them into a private method and the API methods might call that with different filters. Something like `Set<String> getFilteredContent(final String directory, final String branch, final Filter<GHContent> filter)`



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);
+
+        final RegisteredFlowSnapshot deletedSnapshot = getSnapshot(deletedSnapshotContent.read());
+        updateBucketReferences(deletedSnapshot, flowLocation.getBucketId());
+        return deletedSnapshot.getFlow();
+    }
+
+    @Override
+    public RegisteredFlow getFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+        updateBucketReferences(existingSnapshot, flowLocation.getBucketId());
+
+        final RegisteredFlow registeredFlow = existingSnapshot.getFlow();
+        registeredFlow.setBranch(branch);
+        return registeredFlow;
+    }
+
+    @Override
+    public Set<RegisteredFlow> getFlows(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = bucketLocation.getBranch();
+        final String bucketId = bucketLocation.getBucketId();
+        final Set<RegisteredFlow> registeredFlows = new LinkedHashSet<>();
+
+        for (final String filename : repositoryClient.getFileNames(bucketId, branch)) {
+            if (!filename.endsWith(SNAPSHOT_FILE_EXTENSION)) {
+                continue;
+            }
+
+            final String flowId = filename.replace(SNAPSHOT_FILE_EXTENSION, "");
+            final RegisteredFlow registeredFlow = new RegisteredFlow();
+            registeredFlow.setIdentifier(flowId);
+            registeredFlow.setName(flowId);
+            registeredFlow.setBranch(branch);
+            registeredFlow.setBucketIdentifier(bucketId);
+            registeredFlow.setBucketName(bucketId);
+            registeredFlows.add(registeredFlow);
+        }
+
+        return registeredFlows;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot getFlowContents(final FlowRegistryClientConfigurationContext context, final FlowVersionLocation flowVersionLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String version = flowVersionLocation.getVersion();
+        final String filePath = getSnapshotFilePath(flowVersionLocation);
+
+        final InputStream inputStream = repositoryClient.getContentFromCommit(filePath, version);
+        final RegisteredFlowSnapshot flowSnapshot = getSnapshot(inputStream);
+        populateFlowAndSnapshotMetadata(flowSnapshot, flowVersionLocation);
+
+        // populate values that aren't store in GitHub
+        flowSnapshot.getSnapshotMetadata().setVersion(version);
+        flowSnapshot.getSnapshotMetadata().setBranch(flowVersionLocation.getBranch());
+        flowSnapshot.getFlow().setBranch(flowVersionLocation.getBranch());
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowVersionLocation.getBucketId());
+
+        // determine if the version is the "latest" version by comparing to the response of getLatestVersion
+        final String latestVersion = getLatestVersion(context, flowVersionLocation).orElse(null);
+        flowSnapshot.setLatest(version.equals(latestVersion));
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot registerFlowSnapshot(final FlowRegistryClientConfigurationContext context, final RegisteredFlowSnapshot flowSnapshot, final RegisterAction action)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = flowSnapshot.getSnapshotMetadata();
+
+        final String branch = snapshotMetadata.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(snapshotMetadata.getBranch(), snapshotMetadata.getBucketIdentifier(), snapshotMetadata.getFlowIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final String previousSha = repositoryClient.getContentSha(filePath, branch).orElse(null);
+
+        final String snapshotComments = snapshotMetadata.getComments();
+        final String commitMessage = StringUtils.isBlank(snapshotComments) ? DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE : snapshotComments;
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+
+        final RegisteredFlow existingFlow = existingSnapshot.getFlow();
+        existingFlow.setBranch(null);
+        flowSnapshot.setFlow(existingFlow);
+
+        // Clear values we don't want stored in the json in GitHub
+        flowSnapshot.setBucket(null);
+        flowSnapshot.getSnapshotMetadata().setBucketIdentifier(null);
+        flowSnapshot.getSnapshotMetadata().setBranch(null);
+        flowSnapshot.getSnapshotMetadata().setVersion(null);
+        flowSnapshot.getSnapshotMetadata().setComments(null);
+        flowSnapshot.getSnapshotMetadata().setTimestamp(0);
+
+        // replace the id of the top level group and all of its references with a constant value prior to serializing to avoid
+        // unnecessary diffs when different instances of the same flow are imported and have different top-level PG ids
+        final String originalFlowContentsGroupId = replaceGroupId(flowSnapshot.getFlowContents(), FLOW_CONTENTS_GROUP_ID);
+        final Position originalFlowContentsPosition = replacePosition(flowSnapshot.getFlowContents(), new Position(0, 0));
+
+        final GitHubCreateContentRequest createContentRequest = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(commitMessage)
+                .existingContentSha(previousSha)
+                .build();
+
+        final GHContentUpdateResponse createContentResponse = repositoryClient.createContent(createContentRequest);
+        final String createContentCommitSha = createContentResponse.getCommit().getSha();
+
+        final VersionedFlowCoordinates versionedFlowCoordinates = new VersionedFlowCoordinates();
+        versionedFlowCoordinates.setRegistryId(getIdentifier());
+        versionedFlowCoordinates.setBranch(flowLocation.getBranch());
+        versionedFlowCoordinates.setBucketId(flowLocation.getBucketId());
+        versionedFlowCoordinates.setFlowId(flowLocation.getFlowId());
+        versionedFlowCoordinates.setVersion(createContentCommitSha);
+
+        flowSnapshot.getFlowContents().setVersionedFlowCoordinates(versionedFlowCoordinates);
+        flowSnapshot.getFlow().setBranch(branch);
+        flowSnapshot.getSnapshotMetadata().setBranch(branch);
+        flowSnapshot.getSnapshotMetadata().setVersion(createContentCommitSha);
+        flowSnapshot.setLatest(true);
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowLocation.getBucketId());
+
+        // set back to the original id so that the returned snapshot is has the correct values from what was passed in
+        replaceGroupId(flowSnapshot.getFlowContents(), originalFlowContentsGroupId);
+        replacePosition(flowSnapshot.getFlowContents(), originalFlowContentsPosition);
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public Set<RegisteredFlowSnapshotMetadata> getFlowVersions(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Set<RegisteredFlowSnapshotMetadata> snapshotMetadataSet = new LinkedHashSet<>();
+        for (final GHCommit ghCommit : repositoryClient.getCommits(filePath, branch)) {
+            final RegisteredFlowSnapshotMetadata snapshotMetadata = createSnapshotMetadata(ghCommit, flowLocation);
+            if (REGISTER_FLOW_COMMENT.equals(snapshotMetadata.getComments())) {
+                continue;
+            }
+            snapshotMetadataSet.add(snapshotMetadata);
+        }
+        return snapshotMetadataSet;
+    }
+
+    @Override
+    public Optional<String> getLatestVersion(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final List<GHCommit> commits = repositoryClient.getCommits(filePath, branch);
+        final String latestVersion = commits.isEmpty() ? null : commits.getFirst().getSHA1();
+        return Optional.ofNullable(latestVersion);
+    }
+
+    @Override
+    public String generateFlowId(final String flowName) {
+        return flowName.trim()
+                .replaceAll("\\s", "-") // replace whitespace with -
+                .replaceAll("[^a-zA-Z0-9-]", "") // replace all other invalid chars with empty string
+                .replaceAll("(-)\\1+", "$1"); // replace consecutive - with single -
+    }
+
+    private FlowRegistryBucket createFlowRegistryBucket(final String name) {
+        final FlowRegistryPermissions bucketPermissions = new FlowRegistryPermissions();
+        bucketPermissions.setCanRead(true);
+        bucketPermissions.setCanWrite(true);
+        bucketPermissions.setCanDelete(true);
+
+        final FlowRegistryBucket bucket = new FlowRegistryBucket();
+        bucket.setIdentifier(name);
+        bucket.setName(name);
+        bucket.setPermissions(bucketPermissions);
+        return bucket;
+    }
+
+    private RegisteredFlowSnapshotMetadata createSnapshotMetadata(final GHCommit ghCommit, final FlowLocation flowLocation) throws IOException {
+        final GHCommit.ShortInfo shortInfo = ghCommit.getCommitShortInfo();
+
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = new RegisteredFlowSnapshotMetadata();
+        snapshotMetadata.setBranch(flowLocation.getBranch());
+        snapshotMetadata.setBucketIdentifier(flowLocation.getBucketId());
+        snapshotMetadata.setFlowIdentifier(flowLocation.getFlowId());
+        snapshotMetadata.setVersion(ghCommit.getSHA1());
+        snapshotMetadata.setAuthor(ghCommit.getAuthor().getLogin());
+        snapshotMetadata.setComments(shortInfo.getMessage());
+        snapshotMetadata.setTimestamp(shortInfo.getCommitDate().getTime());
+        return snapshotMetadata;
+    }
+
+    private String getSnapshotFilePath(final FlowLocation flowLocation) {
+        return SNAPSHOT_FILE_PATH_FORMAT.formatted(flowLocation.getBucketId(), flowLocation.getFlowId());
+    }
+
+    private RegisteredFlowSnapshot getSnapshot(final String filePath, final String branch) throws IOException, FlowRegistryException {
+        try (final InputStream contentInputStream = repositoryClient.getContentFromBranch(filePath, branch)) {
+            return OBJECT_MAPPER.readValue(contentInputStream, RegisteredFlowSnapshot.class);
+        }
+    }
+
+    private RegisteredFlowSnapshot getSnapshot(final InputStream inputStream) throws IOException {
+        try {
+            return OBJECT_MAPPER.readValue(inputStream, RegisteredFlowSnapshot.class);
+        } finally {
+            IOUtils.closeQuietly(inputStream);
+        }
+    }
+
+    private Position replacePosition(final VersionedProcessGroup group, final Position newPosition) {
+        final Position originalPosition = group.getPosition();
+        group.setPosition(newPosition);
+        return originalPosition;
+    }
+
+    private String replaceGroupId(final VersionedProcessGroup group, final String newGroupId) {
+        final String originalGroupId = group.getIdentifier();
+        group.setIdentifier(newGroupId);
+
+        replaceGroupId(group.getProcessGroups(), newGroupId);
+        replaceGroupId(group.getRemoteProcessGroups(), newGroupId);
+        replaceGroupId(group.getProcessors(), newGroupId);
+        replaceGroupId(group.getFunnels(), newGroupId);
+        replaceGroupId(group.getLabels(), newGroupId);
+        replaceGroupId(group.getInputPorts(), newGroupId);
+        replaceGroupId(group.getOutputPorts(), newGroupId);
+        replaceGroupId(group.getControllerServices(), newGroupId);
+        replaceGroupId(group.getConnections(), newGroupId);
+
+        if (group.getConnections() != null) {
+            for (final VersionedConnection connection : group.getConnections()) {
+                replaceGroupId(connection.getSource(), originalGroupId, newGroupId);
+                replaceGroupId(connection.getDestination(), originalGroupId, newGroupId);
+            }
+        }
+
+        return originalGroupId;
+    }
+
+    private <T extends VersionedComponent> void replaceGroupId(final Collection<T> components, final String newGroupIdentifier) {
+        if (components == null) {
+            return;
+        }
+        components.forEach(c -> c.setGroupIdentifier(newGroupIdentifier));
+    }
+
+    private void replaceGroupId(final ConnectableComponent connectableComponent, final String originalGroupId, final String newGroupId) {
+        if (connectableComponent == null) {
+            return;
+        }
+
+        if (originalGroupId.equals(connectableComponent.getGroupId())) {
+            connectableComponent.setGroupId(newGroupId);
+        }
+    }
+
+    private void updateBucketReferences(final RegisteredFlowSnapshot flowSnapshot, final String bucketId) {
+        final FlowRegistryBucket bucket = createFlowRegistryBucket(bucketId);
+        flowSnapshot.setBucket(bucket);
+
+        final RegisteredFlow flow = flowSnapshot.getFlow();
+        flow.setBucketName(bucketId);
+        flow.setBucketIdentifier(bucketId);
+
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = flowSnapshot.getSnapshotMetadata();
+        snapshotMetadata.setBucketIdentifier(bucketId);
+    }
+
+    // Ensures the snapshot has non-null flow and metadata fields, which would only be null if taking a flow from "Download Flow Definition" and adding directly to GitHub
+    private void populateFlowAndSnapshotMetadata(final RegisteredFlowSnapshot flowSnapshot, final FlowLocation flowLocation) {
+        if (flowSnapshot.getFlow() == null) {
+            final RegisteredFlow registeredFlow = new RegisteredFlow();
+            registeredFlow.setName(flowLocation.getFlowId());
+            registeredFlow.setIdentifier(flowLocation.getFlowId());
+            flowSnapshot.setFlow(registeredFlow);
+        }
+        if (flowSnapshot.getSnapshotMetadata() == null) {
+            final RegisteredFlowSnapshotMetadata snapshotMetadata = new RegisteredFlowSnapshotMetadata();
+            snapshotMetadata.setFlowIdentifier(flowLocation.getFlowId());
+            flowSnapshot.setSnapshotMetadata(snapshotMetadata);
+        }
+    }
+
+    private synchronized GitHubRepositoryClient getRepositoryClient(final FlowRegistryClientConfigurationContext context) throws IOException, FlowRegistryException {
+        if (!initialized.get()) {
+            getLogger().info("Initializing GitHub repository client");
+            repositoryClient = GitHubRepositoryClient.builder()
+                    .apiUrl(context.getProperty(GITHUB_API_URL).getValue())
+                    .authenticationType(GitHubAuthenticationType.valueOf(context.getProperty(AUTHENTICATION_TYPE).getValue()))
+                    .personalAccessToken(context.getProperty(PERSONAL_ACCESS_TOKEN).getValue())
+                    .appInstallationToken(context.getProperty(APP_INSTALLATION_TOKEN).getValue())
+                    .repoOwner(context.getProperty(REPOSITORY_OWNER).getValue())
+                    .repoName(context.getProperty(REPOSITORY_NAME).getValue())
+                    .repoPath(context.getProperty(REPOSITORY_PATH).getValue())
+                    .build();
+            initialized.set(true);
+
+            // If the client is configured to authenticate, then ensure the directory for the default bucket is present and if not create it,
+            // otherwise the client can only be used to import flows from the repo and won't be able to set up the default bucket
+            if (repositoryClient.getAuthenticationType() != GitHubAuthenticationType.NONE) {
+                final String branch = context.getProperty(REPOSITORY_BRANCH).getValue();
+                final Optional<String> defaultBucketKeepFileSha = repositoryClient.getContentSha(DEFAULT_BUCKET_KEEP_FILE_PATH, branch);
+                if (defaultBucketKeepFileSha.isPresent()) {
+                    getLogger().info("Found default bucket with SHA {}", defaultBucketKeepFileSha);
+                } else {
+                    getLogger().info("Creating default bucket");
+                    repositoryClient.createContent(

Review Comment:
   I am not sure that I would make this a default behaviour. This looks to be a hidden writing operation into an external source might be unexpected. It can even happen that the given user has no right for the operation and the client should only read.
   
   I do not doubt the usefullness of this in some cases but I think it should be guarded by a property, something like "create default branch if not found" where the default value for the property is false.
   
   (I see that it is handled using a log for some cases but I think this might cause more consfusion than expected)



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubRepositoryClient.java:
##########
@@ -0,0 +1,404 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+
+package org.apache.nifi.github;
+
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+import org.kohsuke.github.GHRef;
+import org.kohsuke.github.GHRepository;
+import org.kohsuke.github.GitHub;
+import org.kohsuke.github.GitHubBuilder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * Client to encapsulate access to a GitHub Repository through the Hub4j GitHub client.
+ */
+public class GitHubRepositoryClient {
+
+    private static final Logger LOGGER = LoggerFactory.getLogger(GitHubRepositoryClient.class);
+
+    private static final String BRANCH_REF_PATTERN = "refs/heads/%s";
+    private static final int COMMIT_PAGE_SIZE = 50;
+
+    private final GHRepository repository;
+    private final String repoPath;
+    private final GitHubAuthenticationType authenticationType;
+
+    private GitHubRepositoryClient(final Builder builder) throws IOException {
+        final GitHubBuilder gitHubBuilder = new GitHubBuilder().withEndpoint(builder.apiUrl);
+
+        authenticationType = builder.authenticationType;
+        switch (authenticationType) {
+            case PERSONAL_ACCESS_TOKEN -> gitHubBuilder.withOAuthToken(builder.personalAccessToken);
+            case APP_INSTALLATION_TOKEN -> gitHubBuilder.withAppInstallationToken(builder.appInstallationToken);
+        }
+
+        final GitHub gitHub = gitHubBuilder.build();
+        try {
+            repository = gitHub.getRepository(builder.repoOwner + "/" + builder.repoName);
+        } catch (final IOException e) {
+            LOGGER.error("Unable to access GitHub repository [{}]", builder.repoName, e);
+            throw e;
+        }
+        repoPath = builder.repoPath;
+    }
+
+    /**
+     * @return the authentication type this client is configured with
+     */
+    public GitHubAuthenticationType getAuthenticationType() {
+        return authenticationType;
+    }
+
+    /**
+     * Creates the content specified by the given builder.
+     *
+     * @param request the request for the content to create
+     * @return the update response
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public GHContentUpdateResponse createContent(final GitHubCreateContentRequest request) throws IOException, FlowRegistryException {
+        final String branch = request.getBranch();
+        final String resolvedPath = getResolvedPath(request.getPath());
+        LOGGER.debug("Creating content at path [{}] on branch [{}] in repo [{}] ", resolvedPath, branch, repository.getName());
+        return execute(() -> {
+            try {
+                return repository.createContent()
+                        .branch(branch)
+                        .path(resolvedPath)
+                        .content(request.getContent())
+                        .message(request.getMessage())
+                        .sha(request.getExistingContentSha())
+                        .commit();
+            } catch (final FileNotFoundException fnf) {
+                throwPathOrBranchNotFound(resolvedPath, branch);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the names of all the branches in the repo.
+     *
+     * @return the set of all branches in the repo
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public Set<String> getBranches() throws IOException, FlowRegistryException {
+        LOGGER.debug("Getting branches for repo [{}]", repository.getName());
+        return execute(() -> repository.getBranches().keySet());
+    }
+
+    /**
+     * Gets an InputStream to read the latest content of the given path from the given branch.
+     * The returned stream already contains the contents of the requested file.
+     *
+     * @param path the path to the content
+     * @param branch the branch
+     * @return an input stream containing the contents of the path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public InputStream getContentFromBranch(final String path, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting content for [{}] from branch [{}] in repo [{}] ", resolvedPath, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHContent ghContent = repository.getFileContent(resolvedPath, branchRef);
+                return ghContent.read();
+            } catch (final FileNotFoundException e) {
+                throwPathOrBranchNotFound(resolvedPath, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the content of the given path from the given commit.
+     * The returned stream already contains the contents of the requested file.
+     *
+     * @param path the path to the content
+     * @param commitSha the commit SHA
+     * @return an input stream containing the contents of the path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public InputStream getContentFromCommit(final String path, final String commitSha) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        LOGGER.debug("Getting content for [{}] from commit [{}] in repo [{}] ", resolvedPath, commitSha, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHContent ghContent = repository.getFileContent(resolvedPath, commitSha);
+                return ghContent.read();
+            } catch (final FileNotFoundException fnf) {
+                throw new FlowRegistryException("Path [" + resolvedPath + "] or Commit [" + commitSha + "] not found");
+            }
+        });
+    }
+
+    /**
+     * Gets the commits for a given path on a given branch.
+     *
+     * @param path the path
+     * @param branch the branch
+     * @return the list of commits for the given path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public List<GHCommit> getCommits(final String path, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting commits for [{}] from branch [{}] in repo [{}]", resolvedPath, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHRef branchGhRef = repository.getRef(branchRef);
+                return repository.queryCommits()
+                        .path(resolvedPath)
+                        .from(branchGhRef.getObject().getSha())
+                        .pageSize(COMMIT_PAGE_SIZE)
+                        .list()
+                        .toList();
+            } catch (final FileNotFoundException fnf) {
+                throwPathOrBranchNotFound(resolvedPath, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the names of the directories contained within the given directory.
+     *
+     * @param directory the directory to list
+     * @param branch the branch
+     * @return the set of directory names
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public Set<String> getDirectoryNames(final String directory, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedDirectory = getResolvedPath(directory);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting directory names for [{}] from branch [{}] in repo [{}] ", resolvedDirectory, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                return repository.getDirectoryContent(resolvedDirectory, branchRef).stream()
+                        .filter(GHContent::isDirectory)
+                        .map(GHContent::getName)
+                        .collect(Collectors.toSet());
+            } catch (final FileNotFoundException e) {
+                throwPathOrBranchNotFound(resolvedDirectory, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the names of the directories container within the given directory.
+     *
+     * @param directory the directory to list
+     * @param branch the branch
+     * @return the set of file names
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public Set<String> getFileNames(final String directory, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedDirectory = getResolvedPath(directory);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting file names for [{}] from branch [{}] in repo [{}] ", resolvedDirectory, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                return repository.getDirectoryContent(resolvedDirectory, branchRef).stream()
+                        .filter(GHContent::isFile)
+                        .map(GHContent::getName)
+                        .collect(Collectors.toSet());
+            } catch (final FileNotFoundException e) {
+                throwPathOrBranchNotFound(resolvedDirectory, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the current SHA for the given path from the given branch.
+     *
+     * @param path the path to the content
+     * @param branch the branch
+     * @return current sha for the given file, or empty optional
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     */
+    public Optional<String> getContentSha(final String path, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting content SHA for [{}] from branch [{}] in repo [{}] ", resolvedPath, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHContent ghContent = repository.getFileContent(resolvedPath, branchRef);
+                return Optional.of(ghContent.getSha());
+            } catch (final FileNotFoundException e) {
+                LOGGER.warn("Unable to get content SHA for [{}] from branch [{}] because content does not exist", resolvedPath, branch);
+                return Optional.empty();
+            }
+        });
+    }
+
+    /**
+     * Deletes the contents for the given file on the given branch.
+     *
+     * @param filePath the file path to delete
+     * @param commitMessage the commit message for the delete commit
+     * @param branch the branch to delete from
+     * @return the deleted content
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public GHContent deleteContent(final String filePath, final String commitMessage, final String branch) throws FlowRegistryException, IOException {
+        final String resolvedPath = getResolvedPath(filePath);
+        LOGGER.debug("Deleting file [{}] in repo [{}] on branch [{}]", resolvedPath, repository.getName(), branch);
+        return execute(() -> {
+            try {
+                GHContent ghContent = repository.getFileContent(resolvedPath);
+                ghContent.delete(commitMessage, branch);
+                return ghContent;
+            } catch (final FileNotFoundException fnf) {
+                throwPathOrBranchNotFound(resolvedPath, branch);
+                return null;
+            }
+        });
+    }
+
+    private String getResolvedPath(final String path) {
+        return repoPath == null ? path : repoPath + "/" + path;
+    }
+
+    private void throwPathOrBranchNotFound(final String path, final String branch) throws FlowRegistryException {
+        throw new FlowRegistryException("Path [" + path + "] or Branch [" + branch + "] not found");

Review Comment:
   Depending on which part of the path is not existing (bucket, flow, version), there are specific `FlowRegistryException` subclasses for the case. Also as the new concept "Branch" was added, it might worth to add a specific exception for that as well.



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);
+
+        final RegisteredFlowSnapshot deletedSnapshot = getSnapshot(deletedSnapshotContent.read());
+        updateBucketReferences(deletedSnapshot, flowLocation.getBucketId());
+        return deletedSnapshot.getFlow();
+    }
+
+    @Override
+    public RegisteredFlow getFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+        updateBucketReferences(existingSnapshot, flowLocation.getBucketId());
+
+        final RegisteredFlow registeredFlow = existingSnapshot.getFlow();
+        registeredFlow.setBranch(branch);
+        return registeredFlow;
+    }
+
+    @Override
+    public Set<RegisteredFlow> getFlows(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = bucketLocation.getBranch();
+        final String bucketId = bucketLocation.getBucketId();
+        final Set<RegisteredFlow> registeredFlows = new LinkedHashSet<>();
+
+        for (final String filename : repositoryClient.getFileNames(bucketId, branch)) {
+            if (!filename.endsWith(SNAPSHOT_FILE_EXTENSION)) {
+                continue;
+            }
+
+            final String flowId = filename.replace(SNAPSHOT_FILE_EXTENSION, "");
+            final RegisteredFlow registeredFlow = new RegisteredFlow();
+            registeredFlow.setIdentifier(flowId);
+            registeredFlow.setName(flowId);
+            registeredFlow.setBranch(branch);
+            registeredFlow.setBucketIdentifier(bucketId);
+            registeredFlow.setBucketName(bucketId);
+            registeredFlows.add(registeredFlow);
+        }
+
+        return registeredFlows;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot getFlowContents(final FlowRegistryClientConfigurationContext context, final FlowVersionLocation flowVersionLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String version = flowVersionLocation.getVersion();
+        final String filePath = getSnapshotFilePath(flowVersionLocation);
+
+        final InputStream inputStream = repositoryClient.getContentFromCommit(filePath, version);
+        final RegisteredFlowSnapshot flowSnapshot = getSnapshot(inputStream);
+        populateFlowAndSnapshotMetadata(flowSnapshot, flowVersionLocation);
+
+        // populate values that aren't store in GitHub
+        flowSnapshot.getSnapshotMetadata().setVersion(version);
+        flowSnapshot.getSnapshotMetadata().setBranch(flowVersionLocation.getBranch());
+        flowSnapshot.getFlow().setBranch(flowVersionLocation.getBranch());
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowVersionLocation.getBucketId());
+
+        // determine if the version is the "latest" version by comparing to the response of getLatestVersion
+        final String latestVersion = getLatestVersion(context, flowVersionLocation).orElse(null);
+        flowSnapshot.setLatest(version.equals(latestVersion));
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot registerFlowSnapshot(final FlowRegistryClientConfigurationContext context, final RegisteredFlowSnapshot flowSnapshot, final RegisterAction action)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = flowSnapshot.getSnapshotMetadata();
+
+        final String branch = snapshotMetadata.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(snapshotMetadata.getBranch(), snapshotMetadata.getBucketIdentifier(), snapshotMetadata.getFlowIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final String previousSha = repositoryClient.getContentSha(filePath, branch).orElse(null);
+
+        final String snapshotComments = snapshotMetadata.getComments();
+        final String commitMessage = StringUtils.isBlank(snapshotComments) ? DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE : snapshotComments;
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+
+        final RegisteredFlow existingFlow = existingSnapshot.getFlow();
+        existingFlow.setBranch(null);
+        flowSnapshot.setFlow(existingFlow);
+
+        // Clear values we don't want stored in the json in GitHub
+        flowSnapshot.setBucket(null);
+        flowSnapshot.getSnapshotMetadata().setBucketIdentifier(null);
+        flowSnapshot.getSnapshotMetadata().setBranch(null);
+        flowSnapshot.getSnapshotMetadata().setVersion(null);
+        flowSnapshot.getSnapshotMetadata().setComments(null);
+        flowSnapshot.getSnapshotMetadata().setTimestamp(0);
+
+        // replace the id of the top level group and all of its references with a constant value prior to serializing to avoid
+        // unnecessary diffs when different instances of the same flow are imported and have different top-level PG ids
+        final String originalFlowContentsGroupId = replaceGroupId(flowSnapshot.getFlowContents(), FLOW_CONTENTS_GROUP_ID);
+        final Position originalFlowContentsPosition = replacePosition(flowSnapshot.getFlowContents(), new Position(0, 0));
+
+        final GitHubCreateContentRequest createContentRequest = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(commitMessage)
+                .existingContentSha(previousSha)
+                .build();
+
+        final GHContentUpdateResponse createContentResponse = repositoryClient.createContent(createContentRequest);
+        final String createContentCommitSha = createContentResponse.getCommit().getSha();
+
+        final VersionedFlowCoordinates versionedFlowCoordinates = new VersionedFlowCoordinates();
+        versionedFlowCoordinates.setRegistryId(getIdentifier());
+        versionedFlowCoordinates.setBranch(flowLocation.getBranch());
+        versionedFlowCoordinates.setBucketId(flowLocation.getBucketId());
+        versionedFlowCoordinates.setFlowId(flowLocation.getFlowId());
+        versionedFlowCoordinates.setVersion(createContentCommitSha);
+
+        flowSnapshot.getFlowContents().setVersionedFlowCoordinates(versionedFlowCoordinates);
+        flowSnapshot.getFlow().setBranch(branch);
+        flowSnapshot.getSnapshotMetadata().setBranch(branch);
+        flowSnapshot.getSnapshotMetadata().setVersion(createContentCommitSha);
+        flowSnapshot.setLatest(true);
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowLocation.getBucketId());
+
+        // set back to the original id so that the returned snapshot is has the correct values from what was passed in
+        replaceGroupId(flowSnapshot.getFlowContents(), originalFlowContentsGroupId);
+        replacePosition(flowSnapshot.getFlowContents(), originalFlowContentsPosition);
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public Set<RegisteredFlowSnapshotMetadata> getFlowVersions(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Set<RegisteredFlowSnapshotMetadata> snapshotMetadataSet = new LinkedHashSet<>();
+        for (final GHCommit ghCommit : repositoryClient.getCommits(filePath, branch)) {
+            final RegisteredFlowSnapshotMetadata snapshotMetadata = createSnapshotMetadata(ghCommit, flowLocation);
+            if (REGISTER_FLOW_COMMENT.equals(snapshotMetadata.getComments())) {
+                continue;
+            }
+            snapshotMetadataSet.add(snapshotMetadata);
+        }
+        return snapshotMetadataSet;
+    }
+
+    @Override
+    public Optional<String> getLatestVersion(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final List<GHCommit> commits = repositoryClient.getCommits(filePath, branch);
+        final String latestVersion = commits.isEmpty() ? null : commits.getFirst().getSHA1();
+        return Optional.ofNullable(latestVersion);
+    }
+
+    @Override
+    public String generateFlowId(final String flowName) {
+        return flowName.trim()
+                .replaceAll("\\s", "-") // replace whitespace with -
+                .replaceAll("[^a-zA-Z0-9-]", "") // replace all other invalid chars with empty string
+                .replaceAll("(-)\\1+", "$1"); // replace consecutive - with single -
+    }
+
+    private FlowRegistryBucket createFlowRegistryBucket(final String name) {
+        final FlowRegistryPermissions bucketPermissions = new FlowRegistryPermissions();
+        bucketPermissions.setCanRead(true);
+        bucketPermissions.setCanWrite(true);
+        bucketPermissions.setCanDelete(true);
+
+        final FlowRegistryBucket bucket = new FlowRegistryBucket();
+        bucket.setIdentifier(name);
+        bucket.setName(name);
+        bucket.setPermissions(bucketPermissions);
+        return bucket;
+    }
+
+    private RegisteredFlowSnapshotMetadata createSnapshotMetadata(final GHCommit ghCommit, final FlowLocation flowLocation) throws IOException {
+        final GHCommit.ShortInfo shortInfo = ghCommit.getCommitShortInfo();
+
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = new RegisteredFlowSnapshotMetadata();
+        snapshotMetadata.setBranch(flowLocation.getBranch());
+        snapshotMetadata.setBucketIdentifier(flowLocation.getBucketId());
+        snapshotMetadata.setFlowIdentifier(flowLocation.getFlowId());
+        snapshotMetadata.setVersion(ghCommit.getSHA1());
+        snapshotMetadata.setAuthor(ghCommit.getAuthor().getLogin());
+        snapshotMetadata.setComments(shortInfo.getMessage());
+        snapshotMetadata.setTimestamp(shortInfo.getCommitDate().getTime());
+        return snapshotMetadata;
+    }
+
+    private String getSnapshotFilePath(final FlowLocation flowLocation) {
+        return SNAPSHOT_FILE_PATH_FORMAT.formatted(flowLocation.getBucketId(), flowLocation.getFlowId());
+    }
+
+    private RegisteredFlowSnapshot getSnapshot(final String filePath, final String branch) throws IOException, FlowRegistryException {
+        try (final InputStream contentInputStream = repositoryClient.getContentFromBranch(filePath, branch)) {
+            return OBJECT_MAPPER.readValue(contentInputStream, RegisteredFlowSnapshot.class);
+        }
+    }
+
+    private RegisteredFlowSnapshot getSnapshot(final InputStream inputStream) throws IOException {
+        try {
+            return OBJECT_MAPPER.readValue(inputStream, RegisteredFlowSnapshot.class);
+        } finally {
+            IOUtils.closeQuietly(inputStream);

Review Comment:
   I think some logging would be useful for debugging purposes



##########
nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java:
##########
@@ -5357,6 +5355,8 @@ public RegisteredFlow registerVersionedFlow(final String registryId, final Regis
         }
 
         try {
+            final String generatedId = registry.generateFlowId(flow.getName());

Review Comment:
   This might be something I would prefer not to expose outside of the node. This should be an internal implementation detail for the registry node (or implementation). Is there any particular reason this leaves the boundaries of a registry?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594115130


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);
+
+        final RegisteredFlowSnapshot deletedSnapshot = getSnapshot(deletedSnapshotContent.read());
+        updateBucketReferences(deletedSnapshot, flowLocation.getBucketId());
+        return deletedSnapshot.getFlow();
+    }
+
+    @Override
+    public RegisteredFlow getFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+        updateBucketReferences(existingSnapshot, flowLocation.getBucketId());
+
+        final RegisteredFlow registeredFlow = existingSnapshot.getFlow();
+        registeredFlow.setBranch(branch);
+        return registeredFlow;
+    }
+
+    @Override
+    public Set<RegisteredFlow> getFlows(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = bucketLocation.getBranch();
+        final String bucketId = bucketLocation.getBucketId();
+        final Set<RegisteredFlow> registeredFlows = new LinkedHashSet<>();
+
+        for (final String filename : repositoryClient.getFileNames(bucketId, branch)) {
+            if (!filename.endsWith(SNAPSHOT_FILE_EXTENSION)) {
+                continue;
+            }
+
+            final String flowId = filename.replace(SNAPSHOT_FILE_EXTENSION, "");
+            final RegisteredFlow registeredFlow = new RegisteredFlow();
+            registeredFlow.setIdentifier(flowId);
+            registeredFlow.setName(flowId);
+            registeredFlow.setBranch(branch);
+            registeredFlow.setBucketIdentifier(bucketId);
+            registeredFlow.setBucketName(bucketId);
+            registeredFlows.add(registeredFlow);
+        }
+
+        return registeredFlows;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot getFlowContents(final FlowRegistryClientConfigurationContext context, final FlowVersionLocation flowVersionLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String version = flowVersionLocation.getVersion();
+        final String filePath = getSnapshotFilePath(flowVersionLocation);
+
+        final InputStream inputStream = repositoryClient.getContentFromCommit(filePath, version);
+        final RegisteredFlowSnapshot flowSnapshot = getSnapshot(inputStream);
+        populateFlowAndSnapshotMetadata(flowSnapshot, flowVersionLocation);
+
+        // populate values that aren't store in GitHub
+        flowSnapshot.getSnapshotMetadata().setVersion(version);
+        flowSnapshot.getSnapshotMetadata().setBranch(flowVersionLocation.getBranch());
+        flowSnapshot.getFlow().setBranch(flowVersionLocation.getBranch());
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowVersionLocation.getBucketId());
+
+        // determine if the version is the "latest" version by comparing to the response of getLatestVersion
+        final String latestVersion = getLatestVersion(context, flowVersionLocation).orElse(null);
+        flowSnapshot.setLatest(version.equals(latestVersion));
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot registerFlowSnapshot(final FlowRegistryClientConfigurationContext context, final RegisteredFlowSnapshot flowSnapshot, final RegisterAction action)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = flowSnapshot.getSnapshotMetadata();
+
+        final String branch = snapshotMetadata.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(snapshotMetadata.getBranch(), snapshotMetadata.getBucketIdentifier(), snapshotMetadata.getFlowIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final String previousSha = repositoryClient.getContentSha(filePath, branch).orElse(null);
+
+        final String snapshotComments = snapshotMetadata.getComments();
+        final String commitMessage = StringUtils.isBlank(snapshotComments) ? DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE : snapshotComments;
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+
+        final RegisteredFlow existingFlow = existingSnapshot.getFlow();
+        existingFlow.setBranch(null);
+        flowSnapshot.setFlow(existingFlow);
+
+        // Clear values we don't want stored in the json in GitHub
+        flowSnapshot.setBucket(null);
+        flowSnapshot.getSnapshotMetadata().setBucketIdentifier(null);
+        flowSnapshot.getSnapshotMetadata().setBranch(null);
+        flowSnapshot.getSnapshotMetadata().setVersion(null);
+        flowSnapshot.getSnapshotMetadata().setComments(null);
+        flowSnapshot.getSnapshotMetadata().setTimestamp(0);
+
+        // replace the id of the top level group and all of its references with a constant value prior to serializing to avoid
+        // unnecessary diffs when different instances of the same flow are imported and have different top-level PG ids
+        final String originalFlowContentsGroupId = replaceGroupId(flowSnapshot.getFlowContents(), FLOW_CONTENTS_GROUP_ID);
+        final Position originalFlowContentsPosition = replacePosition(flowSnapshot.getFlowContents(), new Position(0, 0));
+
+        final GitHubCreateContentRequest createContentRequest = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(commitMessage)
+                .existingContentSha(previousSha)
+                .build();
+
+        final GHContentUpdateResponse createContentResponse = repositoryClient.createContent(createContentRequest);
+        final String createContentCommitSha = createContentResponse.getCommit().getSha();
+
+        final VersionedFlowCoordinates versionedFlowCoordinates = new VersionedFlowCoordinates();
+        versionedFlowCoordinates.setRegistryId(getIdentifier());
+        versionedFlowCoordinates.setBranch(flowLocation.getBranch());
+        versionedFlowCoordinates.setBucketId(flowLocation.getBucketId());
+        versionedFlowCoordinates.setFlowId(flowLocation.getFlowId());
+        versionedFlowCoordinates.setVersion(createContentCommitSha);
+
+        flowSnapshot.getFlowContents().setVersionedFlowCoordinates(versionedFlowCoordinates);
+        flowSnapshot.getFlow().setBranch(branch);
+        flowSnapshot.getSnapshotMetadata().setBranch(branch);
+        flowSnapshot.getSnapshotMetadata().setVersion(createContentCommitSha);
+        flowSnapshot.setLatest(true);
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowLocation.getBucketId());
+
+        // set back to the original id so that the returned snapshot is has the correct values from what was passed in
+        replaceGroupId(flowSnapshot.getFlowContents(), originalFlowContentsGroupId);
+        replacePosition(flowSnapshot.getFlowContents(), originalFlowContentsPosition);
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public Set<RegisteredFlowSnapshotMetadata> getFlowVersions(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Set<RegisteredFlowSnapshotMetadata> snapshotMetadataSet = new LinkedHashSet<>();
+        for (final GHCommit ghCommit : repositoryClient.getCommits(filePath, branch)) {
+            final RegisteredFlowSnapshotMetadata snapshotMetadata = createSnapshotMetadata(ghCommit, flowLocation);
+            if (REGISTER_FLOW_COMMENT.equals(snapshotMetadata.getComments())) {
+                continue;
+            }
+            snapshotMetadataSet.add(snapshotMetadata);
+        }
+        return snapshotMetadataSet;
+    }
+
+    @Override
+    public Optional<String> getLatestVersion(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final List<GHCommit> commits = repositoryClient.getCommits(filePath, branch);
+        final String latestVersion = commits.isEmpty() ? null : commits.getFirst().getSHA1();
+        return Optional.ofNullable(latestVersion);
+    }
+
+    @Override
+    public String generateFlowId(final String flowName) {
+        return flowName.trim()
+                .replaceAll("\\s", "-") // replace whitespace with -
+                .replaceAll("[^a-zA-Z0-9-]", "") // replace all other invalid chars with empty string
+                .replaceAll("(-)\\1+", "$1"); // replace consecutive - with single -
+    }
+
+    private FlowRegistryBucket createFlowRegistryBucket(final String name) {
+        final FlowRegistryPermissions bucketPermissions = new FlowRegistryPermissions();
+        bucketPermissions.setCanRead(true);
+        bucketPermissions.setCanWrite(true);
+        bucketPermissions.setCanDelete(true);
+
+        final FlowRegistryBucket bucket = new FlowRegistryBucket();
+        bucket.setIdentifier(name);
+        bucket.setName(name);
+        bucket.setPermissions(bucketPermissions);
+        return bucket;
+    }
+
+    private RegisteredFlowSnapshotMetadata createSnapshotMetadata(final GHCommit ghCommit, final FlowLocation flowLocation) throws IOException {
+        final GHCommit.ShortInfo shortInfo = ghCommit.getCommitShortInfo();
+
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = new RegisteredFlowSnapshotMetadata();
+        snapshotMetadata.setBranch(flowLocation.getBranch());
+        snapshotMetadata.setBucketIdentifier(flowLocation.getBucketId());
+        snapshotMetadata.setFlowIdentifier(flowLocation.getFlowId());
+        snapshotMetadata.setVersion(ghCommit.getSHA1());
+        snapshotMetadata.setAuthor(ghCommit.getAuthor().getLogin());
+        snapshotMetadata.setComments(shortInfo.getMessage());
+        snapshotMetadata.setTimestamp(shortInfo.getCommitDate().getTime());
+        return snapshotMetadata;
+    }
+
+    private String getSnapshotFilePath(final FlowLocation flowLocation) {
+        return SNAPSHOT_FILE_PATH_FORMAT.formatted(flowLocation.getBucketId(), flowLocation.getFlowId());
+    }
+
+    private RegisteredFlowSnapshot getSnapshot(final String filePath, final String branch) throws IOException, FlowRegistryException {
+        try (final InputStream contentInputStream = repositoryClient.getContentFromBranch(filePath, branch)) {
+            return OBJECT_MAPPER.readValue(contentInputStream, RegisteredFlowSnapshot.class);
+        }
+    }
+
+    private RegisteredFlowSnapshot getSnapshot(final InputStream inputStream) throws IOException {
+        try {
+            return OBJECT_MAPPER.readValue(inputStream, RegisteredFlowSnapshot.class);
+        } finally {
+            IOUtils.closeQuietly(inputStream);
+        }
+    }
+
+    private Position replacePosition(final VersionedProcessGroup group, final Position newPosition) {
+        final Position originalPosition = group.getPosition();
+        group.setPosition(newPosition);
+        return originalPosition;
+    }
+
+    private String replaceGroupId(final VersionedProcessGroup group, final String newGroupId) {
+        final String originalGroupId = group.getIdentifier();
+        group.setIdentifier(newGroupId);
+
+        replaceGroupId(group.getProcessGroups(), newGroupId);
+        replaceGroupId(group.getRemoteProcessGroups(), newGroupId);
+        replaceGroupId(group.getProcessors(), newGroupId);
+        replaceGroupId(group.getFunnels(), newGroupId);
+        replaceGroupId(group.getLabels(), newGroupId);
+        replaceGroupId(group.getInputPorts(), newGroupId);
+        replaceGroupId(group.getOutputPorts(), newGroupId);
+        replaceGroupId(group.getControllerServices(), newGroupId);
+        replaceGroupId(group.getConnections(), newGroupId);
+
+        if (group.getConnections() != null) {
+            for (final VersionedConnection connection : group.getConnections()) {
+                replaceGroupId(connection.getSource(), originalGroupId, newGroupId);
+                replaceGroupId(connection.getDestination(), originalGroupId, newGroupId);
+            }
+        }
+
+        return originalGroupId;
+    }
+
+    private <T extends VersionedComponent> void replaceGroupId(final Collection<T> components, final String newGroupIdentifier) {
+        if (components == null) {
+            return;
+        }
+        components.forEach(c -> c.setGroupIdentifier(newGroupIdentifier));
+    }
+
+    private void replaceGroupId(final ConnectableComponent connectableComponent, final String originalGroupId, final String newGroupId) {
+        if (connectableComponent == null) {
+            return;
+        }
+
+        if (originalGroupId.equals(connectableComponent.getGroupId())) {
+            connectableComponent.setGroupId(newGroupId);
+        }
+    }
+
+    private void updateBucketReferences(final RegisteredFlowSnapshot flowSnapshot, final String bucketId) {
+        final FlowRegistryBucket bucket = createFlowRegistryBucket(bucketId);
+        flowSnapshot.setBucket(bucket);
+
+        final RegisteredFlow flow = flowSnapshot.getFlow();
+        flow.setBucketName(bucketId);
+        flow.setBucketIdentifier(bucketId);
+
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = flowSnapshot.getSnapshotMetadata();
+        snapshotMetadata.setBucketIdentifier(bucketId);
+    }
+
+    // Ensures the snapshot has non-null flow and metadata fields, which would only be null if taking a flow from "Download Flow Definition" and adding directly to GitHub
+    private void populateFlowAndSnapshotMetadata(final RegisteredFlowSnapshot flowSnapshot, final FlowLocation flowLocation) {
+        if (flowSnapshot.getFlow() == null) {
+            final RegisteredFlow registeredFlow = new RegisteredFlow();
+            registeredFlow.setName(flowLocation.getFlowId());
+            registeredFlow.setIdentifier(flowLocation.getFlowId());
+            flowSnapshot.setFlow(registeredFlow);
+        }
+        if (flowSnapshot.getSnapshotMetadata() == null) {
+            final RegisteredFlowSnapshotMetadata snapshotMetadata = new RegisteredFlowSnapshotMetadata();
+            snapshotMetadata.setFlowIdentifier(flowLocation.getFlowId());
+            flowSnapshot.setSnapshotMetadata(snapshotMetadata);
+        }
+    }
+
+    private synchronized GitHubRepositoryClient getRepositoryClient(final FlowRegistryClientConfigurationContext context) throws IOException, FlowRegistryException {
+        if (!initialized.get()) {
+            getLogger().info("Initializing GitHub repository client");
+            repositoryClient = GitHubRepositoryClient.builder()
+                    .apiUrl(context.getProperty(GITHUB_API_URL).getValue())
+                    .authenticationType(GitHubAuthenticationType.valueOf(context.getProperty(AUTHENTICATION_TYPE).getValue()))
+                    .personalAccessToken(context.getProperty(PERSONAL_ACCESS_TOKEN).getValue())
+                    .appInstallationToken(context.getProperty(APP_INSTALLATION_TOKEN).getValue())
+                    .repoOwner(context.getProperty(REPOSITORY_OWNER).getValue())
+                    .repoName(context.getProperty(REPOSITORY_NAME).getValue())
+                    .repoPath(context.getProperty(REPOSITORY_PATH).getValue())
+                    .build();
+            initialized.set(true);
+
+            // If the client is configured to authenticate, then ensure the directory for the default bucket is present and if not create it,
+            // otherwise the client can only be used to import flows from the repo and won't be able to set up the default bucket
+            if (repositoryClient.getAuthenticationType() != GitHubAuthenticationType.NONE) {
+                final String branch = context.getProperty(REPOSITORY_BRANCH).getValue();
+                final Optional<String> defaultBucketKeepFileSha = repositoryClient.getContentSha(DEFAULT_BUCKET_KEEP_FILE_PATH, branch);
+                if (defaultBucketKeepFileSha.isPresent()) {
+                    getLogger().info("Found default bucket with SHA {}", defaultBucketKeepFileSha);
+                } else {
+                    getLogger().info("Creating default bucket");
+                    repositoryClient.createContent(

Review Comment:
   I'm not sure if I fully agree. I think this is the easiest way for people to get things working. Otherwise they are going to create the client and go to try and use it from NiFi and see no buckets in the list and be confused about what they should do.
   
   The call to create the bucket is only done if the client is configured with authentication, which in most cases would mean they have access to the repo. If you were connecting to a public repo for importing only, then you wouldn't provide any authentication and this call wouldn't happen. I suppose there is an in between case where its a private repo with read-only access, but not sure. Let me think more about this.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594911047


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);
+
+        final RegisteredFlowSnapshot deletedSnapshot = getSnapshot(deletedSnapshotContent.read());
+        updateBucketReferences(deletedSnapshot, flowLocation.getBucketId());
+        return deletedSnapshot.getFlow();
+    }
+
+    @Override
+    public RegisteredFlow getFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+        updateBucketReferences(existingSnapshot, flowLocation.getBucketId());
+
+        final RegisteredFlow registeredFlow = existingSnapshot.getFlow();
+        registeredFlow.setBranch(branch);
+        return registeredFlow;
+    }
+
+    @Override
+    public Set<RegisteredFlow> getFlows(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = bucketLocation.getBranch();
+        final String bucketId = bucketLocation.getBucketId();
+        final Set<RegisteredFlow> registeredFlows = new LinkedHashSet<>();
+
+        for (final String filename : repositoryClient.getFileNames(bucketId, branch)) {
+            if (!filename.endsWith(SNAPSHOT_FILE_EXTENSION)) {
+                continue;
+            }
+
+            final String flowId = filename.replace(SNAPSHOT_FILE_EXTENSION, "");
+            final RegisteredFlow registeredFlow = new RegisteredFlow();
+            registeredFlow.setIdentifier(flowId);
+            registeredFlow.setName(flowId);
+            registeredFlow.setBranch(branch);
+            registeredFlow.setBucketIdentifier(bucketId);
+            registeredFlow.setBucketName(bucketId);
+            registeredFlows.add(registeredFlow);
+        }
+
+        return registeredFlows;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot getFlowContents(final FlowRegistryClientConfigurationContext context, final FlowVersionLocation flowVersionLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String version = flowVersionLocation.getVersion();
+        final String filePath = getSnapshotFilePath(flowVersionLocation);
+
+        final InputStream inputStream = repositoryClient.getContentFromCommit(filePath, version);
+        final RegisteredFlowSnapshot flowSnapshot = getSnapshot(inputStream);
+        populateFlowAndSnapshotMetadata(flowSnapshot, flowVersionLocation);
+
+        // populate values that aren't store in GitHub
+        flowSnapshot.getSnapshotMetadata().setVersion(version);
+        flowSnapshot.getSnapshotMetadata().setBranch(flowVersionLocation.getBranch());
+        flowSnapshot.getFlow().setBranch(flowVersionLocation.getBranch());
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowVersionLocation.getBucketId());
+
+        // determine if the version is the "latest" version by comparing to the response of getLatestVersion
+        final String latestVersion = getLatestVersion(context, flowVersionLocation).orElse(null);
+        flowSnapshot.setLatest(version.equals(latestVersion));
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot registerFlowSnapshot(final FlowRegistryClientConfigurationContext context, final RegisteredFlowSnapshot flowSnapshot, final RegisterAction action)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = flowSnapshot.getSnapshotMetadata();
+
+        final String branch = snapshotMetadata.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(snapshotMetadata.getBranch(), snapshotMetadata.getBucketIdentifier(), snapshotMetadata.getFlowIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final String previousSha = repositoryClient.getContentSha(filePath, branch).orElse(null);
+
+        final String snapshotComments = snapshotMetadata.getComments();
+        final String commitMessage = StringUtils.isBlank(snapshotComments) ? DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE : snapshotComments;
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+
+        final RegisteredFlow existingFlow = existingSnapshot.getFlow();
+        existingFlow.setBranch(null);
+        flowSnapshot.setFlow(existingFlow);
+
+        // Clear values we don't want stored in the json in GitHub
+        flowSnapshot.setBucket(null);
+        flowSnapshot.getSnapshotMetadata().setBucketIdentifier(null);
+        flowSnapshot.getSnapshotMetadata().setBranch(null);
+        flowSnapshot.getSnapshotMetadata().setVersion(null);
+        flowSnapshot.getSnapshotMetadata().setComments(null);
+        flowSnapshot.getSnapshotMetadata().setTimestamp(0);
+
+        // replace the id of the top level group and all of its references with a constant value prior to serializing to avoid
+        // unnecessary diffs when different instances of the same flow are imported and have different top-level PG ids
+        final String originalFlowContentsGroupId = replaceGroupId(flowSnapshot.getFlowContents(), FLOW_CONTENTS_GROUP_ID);
+        final Position originalFlowContentsPosition = replacePosition(flowSnapshot.getFlowContents(), new Position(0, 0));
+
+        final GitHubCreateContentRequest createContentRequest = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(commitMessage)
+                .existingContentSha(previousSha)
+                .build();
+
+        final GHContentUpdateResponse createContentResponse = repositoryClient.createContent(createContentRequest);
+        final String createContentCommitSha = createContentResponse.getCommit().getSha();
+
+        final VersionedFlowCoordinates versionedFlowCoordinates = new VersionedFlowCoordinates();
+        versionedFlowCoordinates.setRegistryId(getIdentifier());
+        versionedFlowCoordinates.setBranch(flowLocation.getBranch());
+        versionedFlowCoordinates.setBucketId(flowLocation.getBucketId());
+        versionedFlowCoordinates.setFlowId(flowLocation.getFlowId());
+        versionedFlowCoordinates.setVersion(createContentCommitSha);
+
+        flowSnapshot.getFlowContents().setVersionedFlowCoordinates(versionedFlowCoordinates);
+        flowSnapshot.getFlow().setBranch(branch);
+        flowSnapshot.getSnapshotMetadata().setBranch(branch);
+        flowSnapshot.getSnapshotMetadata().setVersion(createContentCommitSha);
+        flowSnapshot.setLatest(true);
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowLocation.getBucketId());
+
+        // set back to the original id so that the returned snapshot is has the correct values from what was passed in
+        replaceGroupId(flowSnapshot.getFlowContents(), originalFlowContentsGroupId);
+        replacePosition(flowSnapshot.getFlowContents(), originalFlowContentsPosition);
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public Set<RegisteredFlowSnapshotMetadata> getFlowVersions(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Set<RegisteredFlowSnapshotMetadata> snapshotMetadataSet = new LinkedHashSet<>();
+        for (final GHCommit ghCommit : repositoryClient.getCommits(filePath, branch)) {
+            final RegisteredFlowSnapshotMetadata snapshotMetadata = createSnapshotMetadata(ghCommit, flowLocation);
+            if (REGISTER_FLOW_COMMENT.equals(snapshotMetadata.getComments())) {
+                continue;
+            }
+            snapshotMetadataSet.add(snapshotMetadata);
+        }
+        return snapshotMetadataSet;
+    }
+
+    @Override
+    public Optional<String> getLatestVersion(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final List<GHCommit> commits = repositoryClient.getCommits(filePath, branch);
+        final String latestVersion = commits.isEmpty() ? null : commits.getFirst().getSHA1();
+        return Optional.ofNullable(latestVersion);
+    }
+
+    @Override
+    public String generateFlowId(final String flowName) {
+        return flowName.trim()
+                .replaceAll("\\s", "-") // replace whitespace with -
+                .replaceAll("[^a-zA-Z0-9-]", "") // replace all other invalid chars with empty string
+                .replaceAll("(-)\\1+", "$1"); // replace consecutive - with single -
+    }
+
+    private FlowRegistryBucket createFlowRegistryBucket(final String name) {
+        final FlowRegistryPermissions bucketPermissions = new FlowRegistryPermissions();
+        bucketPermissions.setCanRead(true);

Review Comment:
   Bucket permissions are now set based on the permissions the client has to the repo



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594262547


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);
+
+        final RegisteredFlowSnapshot deletedSnapshot = getSnapshot(deletedSnapshotContent.read());
+        updateBucketReferences(deletedSnapshot, flowLocation.getBucketId());
+        return deletedSnapshot.getFlow();
+    }
+
+    @Override
+    public RegisteredFlow getFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+        updateBucketReferences(existingSnapshot, flowLocation.getBucketId());
+
+        final RegisteredFlow registeredFlow = existingSnapshot.getFlow();
+        registeredFlow.setBranch(branch);
+        return registeredFlow;
+    }
+
+    @Override
+    public Set<RegisteredFlow> getFlows(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = bucketLocation.getBranch();
+        final String bucketId = bucketLocation.getBucketId();
+        final Set<RegisteredFlow> registeredFlows = new LinkedHashSet<>();
+
+        for (final String filename : repositoryClient.getFileNames(bucketId, branch)) {
+            if (!filename.endsWith(SNAPSHOT_FILE_EXTENSION)) {
+                continue;
+            }
+
+            final String flowId = filename.replace(SNAPSHOT_FILE_EXTENSION, "");
+            final RegisteredFlow registeredFlow = new RegisteredFlow();
+            registeredFlow.setIdentifier(flowId);
+            registeredFlow.setName(flowId);
+            registeredFlow.setBranch(branch);
+            registeredFlow.setBucketIdentifier(bucketId);
+            registeredFlow.setBucketName(bucketId);
+            registeredFlows.add(registeredFlow);
+        }
+
+        return registeredFlows;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot getFlowContents(final FlowRegistryClientConfigurationContext context, final FlowVersionLocation flowVersionLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String version = flowVersionLocation.getVersion();
+        final String filePath = getSnapshotFilePath(flowVersionLocation);
+
+        final InputStream inputStream = repositoryClient.getContentFromCommit(filePath, version);
+        final RegisteredFlowSnapshot flowSnapshot = getSnapshot(inputStream);
+        populateFlowAndSnapshotMetadata(flowSnapshot, flowVersionLocation);
+
+        // populate values that aren't store in GitHub
+        flowSnapshot.getSnapshotMetadata().setVersion(version);
+        flowSnapshot.getSnapshotMetadata().setBranch(flowVersionLocation.getBranch());
+        flowSnapshot.getFlow().setBranch(flowVersionLocation.getBranch());
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowVersionLocation.getBucketId());
+
+        // determine if the version is the "latest" version by comparing to the response of getLatestVersion
+        final String latestVersion = getLatestVersion(context, flowVersionLocation).orElse(null);
+        flowSnapshot.setLatest(version.equals(latestVersion));
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot registerFlowSnapshot(final FlowRegistryClientConfigurationContext context, final RegisteredFlowSnapshot flowSnapshot, final RegisterAction action)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = flowSnapshot.getSnapshotMetadata();
+
+        final String branch = snapshotMetadata.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(snapshotMetadata.getBranch(), snapshotMetadata.getBucketIdentifier(), snapshotMetadata.getFlowIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final String previousSha = repositoryClient.getContentSha(filePath, branch).orElse(null);
+
+        final String snapshotComments = snapshotMetadata.getComments();
+        final String commitMessage = StringUtils.isBlank(snapshotComments) ? DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE : snapshotComments;
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+
+        final RegisteredFlow existingFlow = existingSnapshot.getFlow();
+        existingFlow.setBranch(null);
+        flowSnapshot.setFlow(existingFlow);
+
+        // Clear values we don't want stored in the json in GitHub
+        flowSnapshot.setBucket(null);
+        flowSnapshot.getSnapshotMetadata().setBucketIdentifier(null);
+        flowSnapshot.getSnapshotMetadata().setBranch(null);
+        flowSnapshot.getSnapshotMetadata().setVersion(null);
+        flowSnapshot.getSnapshotMetadata().setComments(null);
+        flowSnapshot.getSnapshotMetadata().setTimestamp(0);
+
+        // replace the id of the top level group and all of its references with a constant value prior to serializing to avoid
+        // unnecessary diffs when different instances of the same flow are imported and have different top-level PG ids
+        final String originalFlowContentsGroupId = replaceGroupId(flowSnapshot.getFlowContents(), FLOW_CONTENTS_GROUP_ID);
+        final Position originalFlowContentsPosition = replacePosition(flowSnapshot.getFlowContents(), new Position(0, 0));
+
+        final GitHubCreateContentRequest createContentRequest = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(commitMessage)
+                .existingContentSha(previousSha)
+                .build();
+
+        final GHContentUpdateResponse createContentResponse = repositoryClient.createContent(createContentRequest);
+        final String createContentCommitSha = createContentResponse.getCommit().getSha();
+
+        final VersionedFlowCoordinates versionedFlowCoordinates = new VersionedFlowCoordinates();
+        versionedFlowCoordinates.setRegistryId(getIdentifier());
+        versionedFlowCoordinates.setBranch(flowLocation.getBranch());
+        versionedFlowCoordinates.setBucketId(flowLocation.getBucketId());
+        versionedFlowCoordinates.setFlowId(flowLocation.getFlowId());
+        versionedFlowCoordinates.setVersion(createContentCommitSha);
+
+        flowSnapshot.getFlowContents().setVersionedFlowCoordinates(versionedFlowCoordinates);
+        flowSnapshot.getFlow().setBranch(branch);
+        flowSnapshot.getSnapshotMetadata().setBranch(branch);
+        flowSnapshot.getSnapshotMetadata().setVersion(createContentCommitSha);
+        flowSnapshot.setLatest(true);
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowLocation.getBucketId());
+
+        // set back to the original id so that the returned snapshot is has the correct values from what was passed in
+        replaceGroupId(flowSnapshot.getFlowContents(), originalFlowContentsGroupId);
+        replacePosition(flowSnapshot.getFlowContents(), originalFlowContentsPosition);
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public Set<RegisteredFlowSnapshotMetadata> getFlowVersions(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Set<RegisteredFlowSnapshotMetadata> snapshotMetadataSet = new LinkedHashSet<>();
+        for (final GHCommit ghCommit : repositoryClient.getCommits(filePath, branch)) {
+            final RegisteredFlowSnapshotMetadata snapshotMetadata = createSnapshotMetadata(ghCommit, flowLocation);
+            if (REGISTER_FLOW_COMMENT.equals(snapshotMetadata.getComments())) {
+                continue;
+            }
+            snapshotMetadataSet.add(snapshotMetadata);
+        }
+        return snapshotMetadataSet;
+    }
+
+    @Override
+    public Optional<String> getLatestVersion(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final List<GHCommit> commits = repositoryClient.getCommits(filePath, branch);
+        final String latestVersion = commits.isEmpty() ? null : commits.getFirst().getSHA1();
+        return Optional.ofNullable(latestVersion);
+    }
+
+    @Override
+    public String generateFlowId(final String flowName) {
+        return flowName.trim()
+                .replaceAll("\\s", "-") // replace whitespace with -
+                .replaceAll("[^a-zA-Z0-9-]", "") // replace all other invalid chars with empty string
+                .replaceAll("(-)\\1+", "$1"); // replace consecutive - with single -
+    }
+
+    private FlowRegistryBucket createFlowRegistryBucket(final String name) {
+        final FlowRegistryPermissions bucketPermissions = new FlowRegistryPermissions();
+        bucketPermissions.setCanRead(true);
+        bucketPermissions.setCanWrite(true);
+        bucketPermissions.setCanDelete(true);
+
+        final FlowRegistryBucket bucket = new FlowRegistryBucket();
+        bucket.setIdentifier(name);
+        bucket.setName(name);
+        bucket.setPermissions(bucketPermissions);
+        return bucket;
+    }
+
+    private RegisteredFlowSnapshotMetadata createSnapshotMetadata(final GHCommit ghCommit, final FlowLocation flowLocation) throws IOException {
+        final GHCommit.ShortInfo shortInfo = ghCommit.getCommitShortInfo();
+
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = new RegisteredFlowSnapshotMetadata();
+        snapshotMetadata.setBranch(flowLocation.getBranch());
+        snapshotMetadata.setBucketIdentifier(flowLocation.getBucketId());
+        snapshotMetadata.setFlowIdentifier(flowLocation.getFlowId());
+        snapshotMetadata.setVersion(ghCommit.getSHA1());
+        snapshotMetadata.setAuthor(ghCommit.getAuthor().getLogin());
+        snapshotMetadata.setComments(shortInfo.getMessage());
+        snapshotMetadata.setTimestamp(shortInfo.getCommitDate().getTime());
+        return snapshotMetadata;
+    }
+
+    private String getSnapshotFilePath(final FlowLocation flowLocation) {
+        return SNAPSHOT_FILE_PATH_FORMAT.formatted(flowLocation.getBucketId(), flowLocation.getFlowId());
+    }
+
+    private RegisteredFlowSnapshot getSnapshot(final String filePath, final String branch) throws IOException, FlowRegistryException {
+        try (final InputStream contentInputStream = repositoryClient.getContentFromBranch(filePath, branch)) {
+            return OBJECT_MAPPER.readValue(contentInputStream, RegisteredFlowSnapshot.class);
+        }
+    }
+
+    private RegisteredFlowSnapshot getSnapshot(final InputStream inputStream) throws IOException {
+        try {
+            return OBJECT_MAPPER.readValue(inputStream, RegisteredFlowSnapshot.class);
+        } finally {
+            IOUtils.closeQuietly(inputStream);

Review Comment:
   Not sure there is anything to log here, if an exception happened it would throw IOException out of this method and propagate. Failing to close the stream isn't very meaningful.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594075886


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubRepositoryClient.java:
##########
@@ -0,0 +1,404 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+
+package org.apache.nifi.github;
+
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+import org.kohsuke.github.GHRef;
+import org.kohsuke.github.GHRepository;
+import org.kohsuke.github.GitHub;
+import org.kohsuke.github.GitHubBuilder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * Client to encapsulate access to a GitHub Repository through the Hub4j GitHub client.
+ */
+public class GitHubRepositoryClient {
+
+    private static final Logger LOGGER = LoggerFactory.getLogger(GitHubRepositoryClient.class);
+
+    private static final String BRANCH_REF_PATTERN = "refs/heads/%s";
+    private static final int COMMIT_PAGE_SIZE = 50;
+
+    private final GHRepository repository;
+    private final String repoPath;
+    private final GitHubAuthenticationType authenticationType;
+
+    private GitHubRepositoryClient(final Builder builder) throws IOException {
+        final GitHubBuilder gitHubBuilder = new GitHubBuilder().withEndpoint(builder.apiUrl);
+
+        authenticationType = builder.authenticationType;
+        switch (authenticationType) {
+            case PERSONAL_ACCESS_TOKEN -> gitHubBuilder.withOAuthToken(builder.personalAccessToken);
+            case APP_INSTALLATION_TOKEN -> gitHubBuilder.withAppInstallationToken(builder.appInstallationToken);
+        }
+
+        final GitHub gitHub = gitHubBuilder.build();
+        try {
+            repository = gitHub.getRepository(builder.repoOwner + "/" + builder.repoName);
+        } catch (final IOException e) {
+            LOGGER.error("Unable to access GitHub repository [{}]", builder.repoName, e);
+            throw e;
+        }
+        repoPath = builder.repoPath;
+    }
+
+    /**
+     * @return the authentication type this client is configured with
+     */
+    public GitHubAuthenticationType getAuthenticationType() {
+        return authenticationType;
+    }
+
+    /**
+     * Creates the content specified by the given builder.
+     *
+     * @param request the request for the content to create
+     * @return the update response
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public GHContentUpdateResponse createContent(final GitHubCreateContentRequest request) throws IOException, FlowRegistryException {
+        final String branch = request.getBranch();
+        final String resolvedPath = getResolvedPath(request.getPath());
+        LOGGER.debug("Creating content at path [{}] on branch [{}] in repo [{}] ", resolvedPath, branch, repository.getName());
+        return execute(() -> {
+            try {
+                return repository.createContent()
+                        .branch(branch)
+                        .path(resolvedPath)
+                        .content(request.getContent())
+                        .message(request.getMessage())
+                        .sha(request.getExistingContentSha())
+                        .commit();
+            } catch (final FileNotFoundException fnf) {
+                throwPathOrBranchNotFound(resolvedPath, branch);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the names of all the branches in the repo.
+     *
+     * @return the set of all branches in the repo
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public Set<String> getBranches() throws IOException, FlowRegistryException {
+        LOGGER.debug("Getting branches for repo [{}]", repository.getName());
+        return execute(() -> repository.getBranches().keySet());
+    }
+
+    /**
+     * Gets an InputStream to read the latest content of the given path from the given branch.
+     * The returned stream already contains the contents of the requested file.
+     *
+     * @param path the path to the content
+     * @param branch the branch
+     * @return an input stream containing the contents of the path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public InputStream getContentFromBranch(final String path, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting content for [{}] from branch [{}] in repo [{}] ", resolvedPath, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHContent ghContent = repository.getFileContent(resolvedPath, branchRef);
+                return ghContent.read();
+            } catch (final FileNotFoundException e) {
+                throwPathOrBranchNotFound(resolvedPath, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the content of the given path from the given commit.
+     * The returned stream already contains the contents of the requested file.
+     *
+     * @param path the path to the content
+     * @param commitSha the commit SHA
+     * @return an input stream containing the contents of the path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public InputStream getContentFromCommit(final String path, final String commitSha) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        LOGGER.debug("Getting content for [{}] from commit [{}] in repo [{}] ", resolvedPath, commitSha, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHContent ghContent = repository.getFileContent(resolvedPath, commitSha);
+                return ghContent.read();
+            } catch (final FileNotFoundException fnf) {
+                throw new FlowRegistryException("Path [" + resolvedPath + "] or Commit [" + commitSha + "] not found");
+            }
+        });
+    }
+
+    /**
+     * Gets the commits for a given path on a given branch.
+     *
+     * @param path the path
+     * @param branch the branch
+     * @return the list of commits for the given path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public List<GHCommit> getCommits(final String path, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting commits for [{}] from branch [{}] in repo [{}]", resolvedPath, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHRef branchGhRef = repository.getRef(branchRef);
+                return repository.queryCommits()
+                        .path(resolvedPath)
+                        .from(branchGhRef.getObject().getSha())
+                        .pageSize(COMMIT_PAGE_SIZE)
+                        .list()
+                        .toList();
+            } catch (final FileNotFoundException fnf) {
+                throwPathOrBranchNotFound(resolvedPath, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the names of the directories contained within the given directory.
+     *
+     * @param directory the directory to list
+     * @param branch the branch
+     * @return the set of directory names
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public Set<String> getDirectoryNames(final String directory, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedDirectory = getResolvedPath(directory);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting directory names for [{}] from branch [{}] in repo [{}] ", resolvedDirectory, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                return repository.getDirectoryContent(resolvedDirectory, branchRef).stream()
+                        .filter(GHContent::isDirectory)
+                        .map(GHContent::getName)
+                        .collect(Collectors.toSet());
+            } catch (final FileNotFoundException e) {
+                throwPathOrBranchNotFound(resolvedDirectory, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the names of the directories container within the given directory.
+     *
+     * @param directory the directory to list
+     * @param branch the branch
+     * @return the set of file names
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public Set<String> getFileNames(final String directory, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedDirectory = getResolvedPath(directory);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting file names for [{}] from branch [{}] in repo [{}] ", resolvedDirectory, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                return repository.getDirectoryContent(resolvedDirectory, branchRef).stream()
+                        .filter(GHContent::isFile)
+                        .map(GHContent::getName)
+                        .collect(Collectors.toSet());
+            } catch (final FileNotFoundException e) {
+                throwPathOrBranchNotFound(resolvedDirectory, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the current SHA for the given path from the given branch.
+     *
+     * @param path the path to the content
+     * @param branch the branch
+     * @return current sha for the given file, or empty optional
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     */
+    public Optional<String> getContentSha(final String path, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting content SHA for [{}] from branch [{}] in repo [{}] ", resolvedPath, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHContent ghContent = repository.getFileContent(resolvedPath, branchRef);
+                return Optional.of(ghContent.getSha());
+            } catch (final FileNotFoundException e) {
+                LOGGER.warn("Unable to get content SHA for [{}] from branch [{}] because content does not exist", resolvedPath, branch);
+                return Optional.empty();
+            }
+        });
+    }
+
+    /**
+     * Deletes the contents for the given file on the given branch.
+     *
+     * @param filePath the file path to delete
+     * @param commitMessage the commit message for the delete commit
+     * @param branch the branch to delete from
+     * @return the deleted content
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public GHContent deleteContent(final String filePath, final String commitMessage, final String branch) throws FlowRegistryException, IOException {
+        final String resolvedPath = getResolvedPath(filePath);
+        LOGGER.debug("Deleting file [{}] in repo [{}] on branch [{}]", resolvedPath, repository.getName(), branch);
+        return execute(() -> {
+            try {
+                GHContent ghContent = repository.getFileContent(resolvedPath);
+                ghContent.delete(commitMessage, branch);
+                return ghContent;
+            } catch (final FileNotFoundException fnf) {
+                throwPathOrBranchNotFound(resolvedPath, branch);
+                return null;
+            }
+        });
+    }
+
+    private String getResolvedPath(final String path) {
+        return repoPath == null ? path : repoPath + "/" + path;
+    }
+
+    private void throwPathOrBranchNotFound(final String path, final String branch) throws FlowRegistryException {
+        throw new FlowRegistryException("Path [" + path + "] or Branch [" + branch + "] not found");
+    }
+
+    private <T> T execute(final GHRequest<T> action) throws FlowRegistryException, IOException {
+        try {
+            return action.execute();
+        } catch (final FlowRegistryException | IOException e) {
+            LOGGER.error(e.getMessage(), e);
+            throw e;
+        } catch (final Exception e) {
+            LOGGER.error(e.getMessage(), e);
+            throw new FlowRegistryException(e.getMessage(), e);
+        }
+    }
+
+    /**
+     * Functional interface for making a request to GitHub which may throw IOException.
+     *
+     * @param <T> the result of the request
+     */
+    private interface GHRequest<T> {
+
+        T execute() throws IOException, FlowRegistryException;
+
+    }
+
+    /**
+     * @return a new builder
+     */
+    public static Builder builder() {
+        return new Builder();
+    }
+
+    /**
+     * Builder for the repository client.
+     */
+    public static class Builder {
+
+        private String apiUrl;
+        private GitHubAuthenticationType authenticationType;
+        private String personalAccessToken;
+        private String appInstallationToken;
+        private String repoOwner;
+        private String repoName;
+        private String repoPath;
+
+        public Builder apiUrl(final String apiUrl) {
+            this.apiUrl = apiUrl;
+            return this;
+        }
+
+        public Builder authenticationType(final GitHubAuthenticationType authenticationType) {
+            this.authenticationType = authenticationType;
+            return this;
+        }
+
+        public Builder personalAccessToken(final String personalAccessToken) {
+            this.personalAccessToken = personalAccessToken;
+            return this;
+        }
+
+        public Builder appInstallationToken(final String appInstallationToken) {
+            this.appInstallationToken = appInstallationToken;
+            return this;
+        }
+
+        public Builder repoOwner(final String repoOwner) {
+            this.repoOwner = repoOwner;
+            return this;
+        }
+
+        public Builder repoName(final String repoName) {
+            this.repoName = repoName;
+            return this;
+        }
+
+        public Builder repoPath(final String repoPath) {
+            this.repoPath = repoPath;
+            return this;
+        }
+
+        public GitHubRepositoryClient build() throws IOException {

Review Comment:
   I don't think there is anything to validate here, the overall `GitHubFlowRegistryClient` should not pass validation if the required properties are not set. This lower level client will only be created when all the required valid properties are set.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594142663


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);
+
+        final RegisteredFlowSnapshot deletedSnapshot = getSnapshot(deletedSnapshotContent.read());
+        updateBucketReferences(deletedSnapshot, flowLocation.getBucketId());
+        return deletedSnapshot.getFlow();
+    }
+
+    @Override
+    public RegisteredFlow getFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+        updateBucketReferences(existingSnapshot, flowLocation.getBucketId());
+
+        final RegisteredFlow registeredFlow = existingSnapshot.getFlow();
+        registeredFlow.setBranch(branch);
+        return registeredFlow;
+    }
+
+    @Override
+    public Set<RegisteredFlow> getFlows(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = bucketLocation.getBranch();
+        final String bucketId = bucketLocation.getBucketId();
+        final Set<RegisteredFlow> registeredFlows = new LinkedHashSet<>();
+
+        for (final String filename : repositoryClient.getFileNames(bucketId, branch)) {
+            if (!filename.endsWith(SNAPSHOT_FILE_EXTENSION)) {
+                continue;
+            }
+
+            final String flowId = filename.replace(SNAPSHOT_FILE_EXTENSION, "");
+            final RegisteredFlow registeredFlow = new RegisteredFlow();
+            registeredFlow.setIdentifier(flowId);
+            registeredFlow.setName(flowId);
+            registeredFlow.setBranch(branch);
+            registeredFlow.setBucketIdentifier(bucketId);
+            registeredFlow.setBucketName(bucketId);
+            registeredFlows.add(registeredFlow);
+        }
+
+        return registeredFlows;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot getFlowContents(final FlowRegistryClientConfigurationContext context, final FlowVersionLocation flowVersionLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String version = flowVersionLocation.getVersion();
+        final String filePath = getSnapshotFilePath(flowVersionLocation);
+
+        final InputStream inputStream = repositoryClient.getContentFromCommit(filePath, version);
+        final RegisteredFlowSnapshot flowSnapshot = getSnapshot(inputStream);
+        populateFlowAndSnapshotMetadata(flowSnapshot, flowVersionLocation);
+
+        // populate values that aren't store in GitHub
+        flowSnapshot.getSnapshotMetadata().setVersion(version);
+        flowSnapshot.getSnapshotMetadata().setBranch(flowVersionLocation.getBranch());
+        flowSnapshot.getFlow().setBranch(flowVersionLocation.getBranch());
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowVersionLocation.getBucketId());
+
+        // determine if the version is the "latest" version by comparing to the response of getLatestVersion
+        final String latestVersion = getLatestVersion(context, flowVersionLocation).orElse(null);
+        flowSnapshot.setLatest(version.equals(latestVersion));
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot registerFlowSnapshot(final FlowRegistryClientConfigurationContext context, final RegisteredFlowSnapshot flowSnapshot, final RegisterAction action)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = flowSnapshot.getSnapshotMetadata();
+
+        final String branch = snapshotMetadata.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(snapshotMetadata.getBranch(), snapshotMetadata.getBucketIdentifier(), snapshotMetadata.getFlowIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final String previousSha = repositoryClient.getContentSha(filePath, branch).orElse(null);
+
+        final String snapshotComments = snapshotMetadata.getComments();
+        final String commitMessage = StringUtils.isBlank(snapshotComments) ? DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE : snapshotComments;
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+
+        final RegisteredFlow existingFlow = existingSnapshot.getFlow();
+        existingFlow.setBranch(null);
+        flowSnapshot.setFlow(existingFlow);
+
+        // Clear values we don't want stored in the json in GitHub
+        flowSnapshot.setBucket(null);
+        flowSnapshot.getSnapshotMetadata().setBucketIdentifier(null);
+        flowSnapshot.getSnapshotMetadata().setBranch(null);
+        flowSnapshot.getSnapshotMetadata().setVersion(null);
+        flowSnapshot.getSnapshotMetadata().setComments(null);
+        flowSnapshot.getSnapshotMetadata().setTimestamp(0);

Review Comment:
   The reason for clearing these values is to reduce the differences someone would see in a diff for a pull request. It seemed meaningless to see the timestamp change in a pull request, so I was trying to narrow it down to only the real changes to the flow. Also, if you were looking at the file in GitHub you can see the timestamps of the commits, no need to see it in the json. 
   
   NiFi does not use the timestamp inside the overall snapshot or the flow object for anything. It only uses the timestamps when listings versions. We could try to populate it on the way out, but from a quick look it will require making another request to GitHub, which I'm trying to limit.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594103148


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);
+
+        final RegisteredFlowSnapshot deletedSnapshot = getSnapshot(deletedSnapshotContent.read());
+        updateBucketReferences(deletedSnapshot, flowLocation.getBucketId());
+        return deletedSnapshot.getFlow();
+    }
+
+    @Override
+    public RegisteredFlow getFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+        updateBucketReferences(existingSnapshot, flowLocation.getBucketId());
+
+        final RegisteredFlow registeredFlow = existingSnapshot.getFlow();
+        registeredFlow.setBranch(branch);
+        return registeredFlow;
+    }
+
+    @Override
+    public Set<RegisteredFlow> getFlows(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = bucketLocation.getBranch();
+        final String bucketId = bucketLocation.getBucketId();
+        final Set<RegisteredFlow> registeredFlows = new LinkedHashSet<>();
+
+        for (final String filename : repositoryClient.getFileNames(bucketId, branch)) {
+            if (!filename.endsWith(SNAPSHOT_FILE_EXTENSION)) {
+                continue;
+            }
+
+            final String flowId = filename.replace(SNAPSHOT_FILE_EXTENSION, "");
+            final RegisteredFlow registeredFlow = new RegisteredFlow();
+            registeredFlow.setIdentifier(flowId);
+            registeredFlow.setName(flowId);
+            registeredFlow.setBranch(branch);
+            registeredFlow.setBucketIdentifier(bucketId);
+            registeredFlow.setBucketName(bucketId);
+            registeredFlows.add(registeredFlow);
+        }
+
+        return registeredFlows;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot getFlowContents(final FlowRegistryClientConfigurationContext context, final FlowVersionLocation flowVersionLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String version = flowVersionLocation.getVersion();
+        final String filePath = getSnapshotFilePath(flowVersionLocation);
+
+        final InputStream inputStream = repositoryClient.getContentFromCommit(filePath, version);

Review Comment:
   It will go into the `getContentFromCommit` and if the commit or path isn't found it will throw an exception:
   ```
   try {
                   final GHContent ghContent = repository.getFileContent(resolvedPath, commitSha);
                   return ghContent.read();
               } catch (final FileNotFoundException fnf) {
                   throw new FlowRegistryException("Path [" + resolvedPath + "] or Commit [" + commitSha + "] not found");
               }
   ```
   Now that I am aware of the additional exception classes, I will consider if it makes sense to use them, but also I don't see NiFi doing anything with the additional exceptions, there aren't any special exception mappers or special handling, so the end result for the user is the same regardless of whether we throw `FlowRegistryException` or one of the sub-classes.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594911232


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);
+
+        final RegisteredFlowSnapshot deletedSnapshot = getSnapshot(deletedSnapshotContent.read());
+        updateBucketReferences(deletedSnapshot, flowLocation.getBucketId());
+        return deletedSnapshot.getFlow();
+    }
+
+    @Override
+    public RegisteredFlow getFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+        updateBucketReferences(existingSnapshot, flowLocation.getBucketId());
+
+        final RegisteredFlow registeredFlow = existingSnapshot.getFlow();
+        registeredFlow.setBranch(branch);
+        return registeredFlow;
+    }
+
+    @Override
+    public Set<RegisteredFlow> getFlows(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = bucketLocation.getBranch();
+        final String bucketId = bucketLocation.getBucketId();
+        final Set<RegisteredFlow> registeredFlows = new LinkedHashSet<>();
+
+        for (final String filename : repositoryClient.getFileNames(bucketId, branch)) {
+            if (!filename.endsWith(SNAPSHOT_FILE_EXTENSION)) {
+                continue;
+            }
+
+            final String flowId = filename.replace(SNAPSHOT_FILE_EXTENSION, "");
+            final RegisteredFlow registeredFlow = new RegisteredFlow();
+            registeredFlow.setIdentifier(flowId);
+            registeredFlow.setName(flowId);
+            registeredFlow.setBranch(branch);
+            registeredFlow.setBucketIdentifier(bucketId);
+            registeredFlow.setBucketName(bucketId);
+            registeredFlows.add(registeredFlow);
+        }
+
+        return registeredFlows;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot getFlowContents(final FlowRegistryClientConfigurationContext context, final FlowVersionLocation flowVersionLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String version = flowVersionLocation.getVersion();
+        final String filePath = getSnapshotFilePath(flowVersionLocation);
+
+        final InputStream inputStream = repositoryClient.getContentFromCommit(filePath, version);
+        final RegisteredFlowSnapshot flowSnapshot = getSnapshot(inputStream);
+        populateFlowAndSnapshotMetadata(flowSnapshot, flowVersionLocation);
+
+        // populate values that aren't store in GitHub
+        flowSnapshot.getSnapshotMetadata().setVersion(version);
+        flowSnapshot.getSnapshotMetadata().setBranch(flowVersionLocation.getBranch());
+        flowSnapshot.getFlow().setBranch(flowVersionLocation.getBranch());

Review Comment:
   Added a basic unit test for the two register methods, I think more can be added later.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594910869


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;

Review Comment:
   This should be resolved now



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594119285


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);

Review Comment:
   In this method the snapshot is not returned from the method, only the flow is, so there isn't a need to re-populate the snapshot, in fact we probably could remove this line since the new snapshot already has a null bucket.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594106906


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());

Review Comment:
   The method `createFlowRegistryBucket` may sound misleading, but it is not creating a bucket in GitHub, it is just creating a bucket object to represent whatever the requested bucket name was. This is because buckets in GitHub aren't really something that needs to exist before hand... if you perform a commit with a path like `Bucket1/MyFlow.json` and `Bucket1` didn't exist, it is created during the commit. So it ends up not mattering whether the bucket is in the repo or not, its just a directory.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594097376


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubRepositoryClient.java:
##########
@@ -0,0 +1,404 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+
+package org.apache.nifi.github;
+
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+import org.kohsuke.github.GHRef;
+import org.kohsuke.github.GHRepository;
+import org.kohsuke.github.GitHub;
+import org.kohsuke.github.GitHubBuilder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * Client to encapsulate access to a GitHub Repository through the Hub4j GitHub client.
+ */
+public class GitHubRepositoryClient {

Review Comment:
   It is only two classes that would be impacted if we changed the underlying library, this class and the actual `GitHubFlowRegistryClient` that uses it, it would all be internal implementation changes. I'm not saying it wouldn't be beneficial, but for the amount of additional work to get this PR in, I'm not sure it is really necessary.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594170696


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);
+
+        final RegisteredFlowSnapshot deletedSnapshot = getSnapshot(deletedSnapshotContent.read());
+        updateBucketReferences(deletedSnapshot, flowLocation.getBucketId());
+        return deletedSnapshot.getFlow();
+    }
+
+    @Override
+    public RegisteredFlow getFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+        updateBucketReferences(existingSnapshot, flowLocation.getBucketId());
+
+        final RegisteredFlow registeredFlow = existingSnapshot.getFlow();
+        registeredFlow.setBranch(branch);
+        return registeredFlow;
+    }
+
+    @Override
+    public Set<RegisteredFlow> getFlows(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = bucketLocation.getBranch();
+        final String bucketId = bucketLocation.getBucketId();
+        final Set<RegisteredFlow> registeredFlows = new LinkedHashSet<>();
+
+        for (final String filename : repositoryClient.getFileNames(bucketId, branch)) {
+            if (!filename.endsWith(SNAPSHOT_FILE_EXTENSION)) {
+                continue;
+            }
+
+            final String flowId = filename.replace(SNAPSHOT_FILE_EXTENSION, "");
+            final RegisteredFlow registeredFlow = new RegisteredFlow();
+            registeredFlow.setIdentifier(flowId);
+            registeredFlow.setName(flowId);
+            registeredFlow.setBranch(branch);
+            registeredFlow.setBucketIdentifier(bucketId);
+            registeredFlow.setBucketName(bucketId);
+            registeredFlows.add(registeredFlow);
+        }
+
+        return registeredFlows;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot getFlowContents(final FlowRegistryClientConfigurationContext context, final FlowVersionLocation flowVersionLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String version = flowVersionLocation.getVersion();
+        final String filePath = getSnapshotFilePath(flowVersionLocation);
+
+        final InputStream inputStream = repositoryClient.getContentFromCommit(filePath, version);
+        final RegisteredFlowSnapshot flowSnapshot = getSnapshot(inputStream);
+        populateFlowAndSnapshotMetadata(flowSnapshot, flowVersionLocation);
+
+        // populate values that aren't store in GitHub
+        flowSnapshot.getSnapshotMetadata().setVersion(version);
+        flowSnapshot.getSnapshotMetadata().setBranch(flowVersionLocation.getBranch());
+        flowSnapshot.getFlow().setBranch(flowVersionLocation.getBranch());
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowVersionLocation.getBucketId());
+
+        // determine if the version is the "latest" version by comparing to the response of getLatestVersion
+        final String latestVersion = getLatestVersion(context, flowVersionLocation).orElse(null);
+        flowSnapshot.setLatest(version.equals(latestVersion));
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot registerFlowSnapshot(final FlowRegistryClientConfigurationContext context, final RegisteredFlowSnapshot flowSnapshot, final RegisterAction action)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = flowSnapshot.getSnapshotMetadata();
+
+        final String branch = snapshotMetadata.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(snapshotMetadata.getBranch(), snapshotMetadata.getBucketIdentifier(), snapshotMetadata.getFlowIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final String previousSha = repositoryClient.getContentSha(filePath, branch).orElse(null);
+
+        final String snapshotComments = snapshotMetadata.getComments();
+        final String commitMessage = StringUtils.isBlank(snapshotComments) ? DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE : snapshotComments;
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+
+        final RegisteredFlow existingFlow = existingSnapshot.getFlow();
+        existingFlow.setBranch(null);
+        flowSnapshot.setFlow(existingFlow);
+
+        // Clear values we don't want stored in the json in GitHub
+        flowSnapshot.setBucket(null);
+        flowSnapshot.getSnapshotMetadata().setBucketIdentifier(null);
+        flowSnapshot.getSnapshotMetadata().setBranch(null);
+        flowSnapshot.getSnapshotMetadata().setVersion(null);
+        flowSnapshot.getSnapshotMetadata().setComments(null);
+        flowSnapshot.getSnapshotMetadata().setTimestamp(0);
+
+        // replace the id of the top level group and all of its references with a constant value prior to serializing to avoid
+        // unnecessary diffs when different instances of the same flow are imported and have different top-level PG ids
+        final String originalFlowContentsGroupId = replaceGroupId(flowSnapshot.getFlowContents(), FLOW_CONTENTS_GROUP_ID);
+        final Position originalFlowContentsPosition = replacePosition(flowSnapshot.getFlowContents(), new Position(0, 0));
+
+        final GitHubCreateContentRequest createContentRequest = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(commitMessage)
+                .existingContentSha(previousSha)
+                .build();
+
+        final GHContentUpdateResponse createContentResponse = repositoryClient.createContent(createContentRequest);
+        final String createContentCommitSha = createContentResponse.getCommit().getSha();
+
+        final VersionedFlowCoordinates versionedFlowCoordinates = new VersionedFlowCoordinates();
+        versionedFlowCoordinates.setRegistryId(getIdentifier());
+        versionedFlowCoordinates.setBranch(flowLocation.getBranch());
+        versionedFlowCoordinates.setBucketId(flowLocation.getBucketId());
+        versionedFlowCoordinates.setFlowId(flowLocation.getFlowId());
+        versionedFlowCoordinates.setVersion(createContentCommitSha);
+
+        flowSnapshot.getFlowContents().setVersionedFlowCoordinates(versionedFlowCoordinates);
+        flowSnapshot.getFlow().setBranch(branch);
+        flowSnapshot.getSnapshotMetadata().setBranch(branch);
+        flowSnapshot.getSnapshotMetadata().setVersion(createContentCommitSha);
+        flowSnapshot.setLatest(true);
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowLocation.getBucketId());
+
+        // set back to the original id so that the returned snapshot is has the correct values from what was passed in
+        replaceGroupId(flowSnapshot.getFlowContents(), originalFlowContentsGroupId);
+        replacePosition(flowSnapshot.getFlowContents(), originalFlowContentsPosition);
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public Set<RegisteredFlowSnapshotMetadata> getFlowVersions(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Set<RegisteredFlowSnapshotMetadata> snapshotMetadataSet = new LinkedHashSet<>();
+        for (final GHCommit ghCommit : repositoryClient.getCommits(filePath, branch)) {
+            final RegisteredFlowSnapshotMetadata snapshotMetadata = createSnapshotMetadata(ghCommit, flowLocation);
+            if (REGISTER_FLOW_COMMENT.equals(snapshotMetadata.getComments())) {
+                continue;
+            }
+            snapshotMetadataSet.add(snapshotMetadata);
+        }
+        return snapshotMetadataSet;
+    }
+
+    @Override
+    public Optional<String> getLatestVersion(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final List<GHCommit> commits = repositoryClient.getCommits(filePath, branch);
+        final String latestVersion = commits.isEmpty() ? null : commits.getFirst().getSHA1();
+        return Optional.ofNullable(latestVersion);
+    }
+
+    @Override
+    public String generateFlowId(final String flowName) {
+        return flowName.trim()
+                .replaceAll("\\s", "-") // replace whitespace with -
+                .replaceAll("[^a-zA-Z0-9-]", "") // replace all other invalid chars with empty string
+                .replaceAll("(-)\\1+", "$1"); // replace consecutive - with single -
+    }
+
+    private FlowRegistryBucket createFlowRegistryBucket(final String name) {
+        final FlowRegistryPermissions bucketPermissions = new FlowRegistryPermissions();
+        bucketPermissions.setCanRead(true);

Review Comment:
   Good question. First thing to keep in mind here is that buckets in GitHub are directories, which themselves don't have permissions. If you have write access to the repo you have write access to all buckets, etc. This concept of bucket permissions came from NiFi Registry where the bucket can have permissions, and NiFi Registry can tell NiFi what the permissions are.
   
   I think what we really need is permissions on the registry client, because for GitHub its more of an all or nothing. If the client has a token with WRITE access to the repo, then the client has WRITE for all buckets, etc. We currently don't have a concept of permissions on registry clients, and the current authorization is extremely restrictive:
   https://issues.apache.org/jira/browse/NIFI-12482



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594910362


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);
+
+        final RegisteredFlowSnapshot deletedSnapshot = getSnapshot(deletedSnapshotContent.read());
+        updateBucketReferences(deletedSnapshot, flowLocation.getBucketId());
+        return deletedSnapshot.getFlow();
+    }
+
+    @Override
+    public RegisteredFlow getFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+        updateBucketReferences(existingSnapshot, flowLocation.getBucketId());
+
+        final RegisteredFlow registeredFlow = existingSnapshot.getFlow();
+        registeredFlow.setBranch(branch);
+        return registeredFlow;
+    }
+
+    @Override
+    public Set<RegisteredFlow> getFlows(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = bucketLocation.getBranch();
+        final String bucketId = bucketLocation.getBucketId();
+        final Set<RegisteredFlow> registeredFlows = new LinkedHashSet<>();
+
+        for (final String filename : repositoryClient.getFileNames(bucketId, branch)) {
+            if (!filename.endsWith(SNAPSHOT_FILE_EXTENSION)) {
+                continue;
+            }
+
+            final String flowId = filename.replace(SNAPSHOT_FILE_EXTENSION, "");
+            final RegisteredFlow registeredFlow = new RegisteredFlow();
+            registeredFlow.setIdentifier(flowId);
+            registeredFlow.setName(flowId);
+            registeredFlow.setBranch(branch);
+            registeredFlow.setBucketIdentifier(bucketId);
+            registeredFlow.setBucketName(bucketId);
+            registeredFlows.add(registeredFlow);
+        }
+
+        return registeredFlows;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot getFlowContents(final FlowRegistryClientConfigurationContext context, final FlowVersionLocation flowVersionLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String version = flowVersionLocation.getVersion();
+        final String filePath = getSnapshotFilePath(flowVersionLocation);
+
+        final InputStream inputStream = repositoryClient.getContentFromCommit(filePath, version);
+        final RegisteredFlowSnapshot flowSnapshot = getSnapshot(inputStream);
+        populateFlowAndSnapshotMetadata(flowSnapshot, flowVersionLocation);
+
+        // populate values that aren't store in GitHub
+        flowSnapshot.getSnapshotMetadata().setVersion(version);
+        flowSnapshot.getSnapshotMetadata().setBranch(flowVersionLocation.getBranch());
+        flowSnapshot.getFlow().setBranch(flowVersionLocation.getBranch());
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowVersionLocation.getBucketId());
+
+        // determine if the version is the "latest" version by comparing to the response of getLatestVersion
+        final String latestVersion = getLatestVersion(context, flowVersionLocation).orElse(null);
+        flowSnapshot.setLatest(version.equals(latestVersion));
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot registerFlowSnapshot(final FlowRegistryClientConfigurationContext context, final RegisteredFlowSnapshot flowSnapshot, final RegisterAction action)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = flowSnapshot.getSnapshotMetadata();
+
+        final String branch = snapshotMetadata.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(snapshotMetadata.getBranch(), snapshotMetadata.getBucketIdentifier(), snapshotMetadata.getFlowIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final String previousSha = repositoryClient.getContentSha(filePath, branch).orElse(null);
+
+        final String snapshotComments = snapshotMetadata.getComments();
+        final String commitMessage = StringUtils.isBlank(snapshotComments) ? DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE : snapshotComments;
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+
+        final RegisteredFlow existingFlow = existingSnapshot.getFlow();
+        existingFlow.setBranch(null);
+        flowSnapshot.setFlow(existingFlow);
+
+        // Clear values we don't want stored in the json in GitHub
+        flowSnapshot.setBucket(null);
+        flowSnapshot.getSnapshotMetadata().setBucketIdentifier(null);
+        flowSnapshot.getSnapshotMetadata().setBranch(null);
+        flowSnapshot.getSnapshotMetadata().setVersion(null);
+        flowSnapshot.getSnapshotMetadata().setComments(null);
+        flowSnapshot.getSnapshotMetadata().setTimestamp(0);
+
+        // replace the id of the top level group and all of its references with a constant value prior to serializing to avoid
+        // unnecessary diffs when different instances of the same flow are imported and have different top-level PG ids
+        final String originalFlowContentsGroupId = replaceGroupId(flowSnapshot.getFlowContents(), FLOW_CONTENTS_GROUP_ID);
+        final Position originalFlowContentsPosition = replacePosition(flowSnapshot.getFlowContents(), new Position(0, 0));
+
+        final GitHubCreateContentRequest createContentRequest = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(commitMessage)
+                .existingContentSha(previousSha)
+                .build();
+
+        final GHContentUpdateResponse createContentResponse = repositoryClient.createContent(createContentRequest);
+        final String createContentCommitSha = createContentResponse.getCommit().getSha();
+
+        final VersionedFlowCoordinates versionedFlowCoordinates = new VersionedFlowCoordinates();
+        versionedFlowCoordinates.setRegistryId(getIdentifier());
+        versionedFlowCoordinates.setBranch(flowLocation.getBranch());
+        versionedFlowCoordinates.setBucketId(flowLocation.getBucketId());
+        versionedFlowCoordinates.setFlowId(flowLocation.getFlowId());
+        versionedFlowCoordinates.setVersion(createContentCommitSha);
+
+        flowSnapshot.getFlowContents().setVersionedFlowCoordinates(versionedFlowCoordinates);
+        flowSnapshot.getFlow().setBranch(branch);
+        flowSnapshot.getSnapshotMetadata().setBranch(branch);
+        flowSnapshot.getSnapshotMetadata().setVersion(createContentCommitSha);
+        flowSnapshot.setLatest(true);
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowLocation.getBucketId());
+
+        // set back to the original id so that the returned snapshot is has the correct values from what was passed in
+        replaceGroupId(flowSnapshot.getFlowContents(), originalFlowContentsGroupId);
+        replacePosition(flowSnapshot.getFlowContents(), originalFlowContentsPosition);
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public Set<RegisteredFlowSnapshotMetadata> getFlowVersions(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Set<RegisteredFlowSnapshotMetadata> snapshotMetadataSet = new LinkedHashSet<>();
+        for (final GHCommit ghCommit : repositoryClient.getCommits(filePath, branch)) {
+            final RegisteredFlowSnapshotMetadata snapshotMetadata = createSnapshotMetadata(ghCommit, flowLocation);
+            if (REGISTER_FLOW_COMMENT.equals(snapshotMetadata.getComments())) {
+                continue;
+            }
+            snapshotMetadataSet.add(snapshotMetadata);
+        }
+        return snapshotMetadataSet;
+    }
+
+    @Override
+    public Optional<String> getLatestVersion(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final List<GHCommit> commits = repositoryClient.getCommits(filePath, branch);
+        final String latestVersion = commits.isEmpty() ? null : commits.getFirst().getSHA1();
+        return Optional.ofNullable(latestVersion);
+    }
+
+    @Override
+    public String generateFlowId(final String flowName) {
+        return flowName.trim()
+                .replaceAll("\\s", "-") // replace whitespace with -
+                .replaceAll("[^a-zA-Z0-9-]", "") // replace all other invalid chars with empty string
+                .replaceAll("(-)\\1+", "$1"); // replace consecutive - with single -
+    }
+
+    private FlowRegistryBucket createFlowRegistryBucket(final String name) {
+        final FlowRegistryPermissions bucketPermissions = new FlowRegistryPermissions();
+        bucketPermissions.setCanRead(true);
+        bucketPermissions.setCanWrite(true);
+        bucketPermissions.setCanDelete(true);
+
+        final FlowRegistryBucket bucket = new FlowRegistryBucket();
+        bucket.setIdentifier(name);
+        bucket.setName(name);
+        bucket.setPermissions(bucketPermissions);
+        return bucket;
+    }
+
+    private RegisteredFlowSnapshotMetadata createSnapshotMetadata(final GHCommit ghCommit, final FlowLocation flowLocation) throws IOException {
+        final GHCommit.ShortInfo shortInfo = ghCommit.getCommitShortInfo();
+
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = new RegisteredFlowSnapshotMetadata();
+        snapshotMetadata.setBranch(flowLocation.getBranch());
+        snapshotMetadata.setBucketIdentifier(flowLocation.getBucketId());
+        snapshotMetadata.setFlowIdentifier(flowLocation.getFlowId());
+        snapshotMetadata.setVersion(ghCommit.getSHA1());
+        snapshotMetadata.setAuthor(ghCommit.getAuthor().getLogin());
+        snapshotMetadata.setComments(shortInfo.getMessage());
+        snapshotMetadata.setTimestamp(shortInfo.getCommitDate().getTime());
+        return snapshotMetadata;
+    }
+
+    private String getSnapshotFilePath(final FlowLocation flowLocation) {
+        return SNAPSHOT_FILE_PATH_FORMAT.formatted(flowLocation.getBucketId(), flowLocation.getFlowId());
+    }
+
+    private RegisteredFlowSnapshot getSnapshot(final String filePath, final String branch) throws IOException, FlowRegistryException {
+        try (final InputStream contentInputStream = repositoryClient.getContentFromBranch(filePath, branch)) {
+            return OBJECT_MAPPER.readValue(contentInputStream, RegisteredFlowSnapshot.class);
+        }
+    }
+
+    private RegisteredFlowSnapshot getSnapshot(final InputStream inputStream) throws IOException {
+        try {
+            return OBJECT_MAPPER.readValue(inputStream, RegisteredFlowSnapshot.class);
+        } finally {
+            IOUtils.closeQuietly(inputStream);
+        }
+    }
+
+    private Position replacePosition(final VersionedProcessGroup group, final Position newPosition) {
+        final Position originalPosition = group.getPosition();
+        group.setPosition(newPosition);
+        return originalPosition;
+    }
+
+    private String replaceGroupId(final VersionedProcessGroup group, final String newGroupId) {
+        final String originalGroupId = group.getIdentifier();
+        group.setIdentifier(newGroupId);
+
+        replaceGroupId(group.getProcessGroups(), newGroupId);
+        replaceGroupId(group.getRemoteProcessGroups(), newGroupId);
+        replaceGroupId(group.getProcessors(), newGroupId);
+        replaceGroupId(group.getFunnels(), newGroupId);
+        replaceGroupId(group.getLabels(), newGroupId);
+        replaceGroupId(group.getInputPorts(), newGroupId);
+        replaceGroupId(group.getOutputPorts(), newGroupId);
+        replaceGroupId(group.getControllerServices(), newGroupId);
+        replaceGroupId(group.getConnections(), newGroupId);
+
+        if (group.getConnections() != null) {
+            for (final VersionedConnection connection : group.getConnections()) {
+                replaceGroupId(connection.getSource(), originalGroupId, newGroupId);
+                replaceGroupId(connection.getDestination(), originalGroupId, newGroupId);
+            }
+        }
+
+        return originalGroupId;
+    }
+
+    private <T extends VersionedComponent> void replaceGroupId(final Collection<T> components, final String newGroupIdentifier) {
+        if (components == null) {
+            return;
+        }
+        components.forEach(c -> c.setGroupIdentifier(newGroupIdentifier));
+    }
+
+    private void replaceGroupId(final ConnectableComponent connectableComponent, final String originalGroupId, final String newGroupId) {
+        if (connectableComponent == null) {
+            return;
+        }
+
+        if (originalGroupId.equals(connectableComponent.getGroupId())) {
+            connectableComponent.setGroupId(newGroupId);
+        }
+    }
+
+    private void updateBucketReferences(final RegisteredFlowSnapshot flowSnapshot, final String bucketId) {
+        final FlowRegistryBucket bucket = createFlowRegistryBucket(bucketId);
+        flowSnapshot.setBucket(bucket);
+
+        final RegisteredFlow flow = flowSnapshot.getFlow();
+        flow.setBucketName(bucketId);
+        flow.setBucketIdentifier(bucketId);
+
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = flowSnapshot.getSnapshotMetadata();
+        snapshotMetadata.setBucketIdentifier(bucketId);
+    }
+
+    // Ensures the snapshot has non-null flow and metadata fields, which would only be null if taking a flow from "Download Flow Definition" and adding directly to GitHub
+    private void populateFlowAndSnapshotMetadata(final RegisteredFlowSnapshot flowSnapshot, final FlowLocation flowLocation) {
+        if (flowSnapshot.getFlow() == null) {
+            final RegisteredFlow registeredFlow = new RegisteredFlow();
+            registeredFlow.setName(flowLocation.getFlowId());
+            registeredFlow.setIdentifier(flowLocation.getFlowId());
+            flowSnapshot.setFlow(registeredFlow);
+        }
+        if (flowSnapshot.getSnapshotMetadata() == null) {
+            final RegisteredFlowSnapshotMetadata snapshotMetadata = new RegisteredFlowSnapshotMetadata();
+            snapshotMetadata.setFlowIdentifier(flowLocation.getFlowId());
+            flowSnapshot.setSnapshotMetadata(snapshotMetadata);
+        }
+    }
+
+    private synchronized GitHubRepositoryClient getRepositoryClient(final FlowRegistryClientConfigurationContext context) throws IOException, FlowRegistryException {
+        if (!initialized.get()) {
+            getLogger().info("Initializing GitHub repository client");
+            repositoryClient = GitHubRepositoryClient.builder()
+                    .apiUrl(context.getProperty(GITHUB_API_URL).getValue())
+                    .authenticationType(GitHubAuthenticationType.valueOf(context.getProperty(AUTHENTICATION_TYPE).getValue()))
+                    .personalAccessToken(context.getProperty(PERSONAL_ACCESS_TOKEN).getValue())
+                    .appInstallationToken(context.getProperty(APP_INSTALLATION_TOKEN).getValue())
+                    .repoOwner(context.getProperty(REPOSITORY_OWNER).getValue())
+                    .repoName(context.getProperty(REPOSITORY_NAME).getValue())
+                    .repoPath(context.getProperty(REPOSITORY_PATH).getValue())
+                    .build();
+            initialized.set(true);
+
+            // If the client is configured to authenticate, then ensure the directory for the default bucket is present and if not create it,
+            // otherwise the client can only be used to import flows from the repo and won't be able to set up the default bucket
+            if (repositoryClient.getAuthenticationType() != GitHubAuthenticationType.NONE) {
+                final String branch = context.getProperty(REPOSITORY_BRANCH).getValue();
+                final Optional<String> defaultBucketKeepFileSha = repositoryClient.getContentSha(DEFAULT_BUCKET_KEEP_FILE_PATH, branch);
+                if (defaultBucketKeepFileSha.isPresent()) {
+                    getLogger().info("Found default bucket with SHA {}", defaultBucketKeepFileSha);
+                } else {
+                    getLogger().info("Creating default bucket");
+                    repositoryClient.createContent(

Review Comment:
   I changed the behavior to only create the default bucket when the client has WRITE permissions AND when there are no other existing buckets.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "exceptionfactory (via GitHub)" <gi...@apache.org>.
exceptionfactory closed pull request #8765: NIFI-13105 Implement a FlowRegistryClient that use GitHub for version…
URL: https://github.com/apache/nifi/pull/8765


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594150514


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");

Review Comment:
   This is a good case to use the specific exception, thanks



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594151939


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;

Review Comment:
   I'm not sure I follow, are you saying to return true here?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594116657


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);
+
+        final RegisteredFlowSnapshot deletedSnapshot = getSnapshot(deletedSnapshotContent.read());
+        updateBucketReferences(deletedSnapshot, flowLocation.getBucketId());
+        return deletedSnapshot.getFlow();
+    }
+
+    @Override
+    public RegisteredFlow getFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+        updateBucketReferences(existingSnapshot, flowLocation.getBucketId());
+
+        final RegisteredFlow registeredFlow = existingSnapshot.getFlow();
+        registeredFlow.setBranch(branch);
+        return registeredFlow;
+    }
+
+    @Override
+    public Set<RegisteredFlow> getFlows(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = bucketLocation.getBranch();
+        final String bucketId = bucketLocation.getBucketId();
+        final Set<RegisteredFlow> registeredFlows = new LinkedHashSet<>();
+
+        for (final String filename : repositoryClient.getFileNames(bucketId, branch)) {
+            if (!filename.endsWith(SNAPSHOT_FILE_EXTENSION)) {
+                continue;
+            }
+
+            final String flowId = filename.replace(SNAPSHOT_FILE_EXTENSION, "");
+            final RegisteredFlow registeredFlow = new RegisteredFlow();
+            registeredFlow.setIdentifier(flowId);
+            registeredFlow.setName(flowId);
+            registeredFlow.setBranch(branch);
+            registeredFlow.setBucketIdentifier(bucketId);
+            registeredFlow.setBucketName(bucketId);
+            registeredFlows.add(registeredFlow);
+        }
+
+        return registeredFlows;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot getFlowContents(final FlowRegistryClientConfigurationContext context, final FlowVersionLocation flowVersionLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String version = flowVersionLocation.getVersion();
+        final String filePath = getSnapshotFilePath(flowVersionLocation);
+
+        final InputStream inputStream = repositoryClient.getContentFromCommit(filePath, version);
+        final RegisteredFlowSnapshot flowSnapshot = getSnapshot(inputStream);
+        populateFlowAndSnapshotMetadata(flowSnapshot, flowVersionLocation);
+
+        // populate values that aren't store in GitHub
+        flowSnapshot.getSnapshotMetadata().setVersion(version);
+        flowSnapshot.getSnapshotMetadata().setBranch(flowVersionLocation.getBranch());
+        flowSnapshot.getFlow().setBranch(flowVersionLocation.getBranch());
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowVersionLocation.getBucketId());
+
+        // determine if the version is the "latest" version by comparing to the response of getLatestVersion
+        final String latestVersion = getLatestVersion(context, flowVersionLocation).orElse(null);
+        flowSnapshot.setLatest(version.equals(latestVersion));
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public RegisteredFlowSnapshot registerFlowSnapshot(final FlowRegistryClientConfigurationContext context, final RegisteredFlowSnapshot flowSnapshot, final RegisterAction action)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final RegisteredFlowSnapshotMetadata snapshotMetadata = flowSnapshot.getSnapshotMetadata();
+
+        final String branch = snapshotMetadata.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(snapshotMetadata.getBranch(), snapshotMetadata.getBucketIdentifier(), snapshotMetadata.getFlowIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final String previousSha = repositoryClient.getContentSha(filePath, branch).orElse(null);
+
+        final String snapshotComments = snapshotMetadata.getComments();
+        final String commitMessage = StringUtils.isBlank(snapshotComments) ? DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE : snapshotComments;
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);
+        populateFlowAndSnapshotMetadata(existingSnapshot, flowLocation);
+
+        final RegisteredFlow existingFlow = existingSnapshot.getFlow();
+        existingFlow.setBranch(null);
+        flowSnapshot.setFlow(existingFlow);
+
+        // Clear values we don't want stored in the json in GitHub
+        flowSnapshot.setBucket(null);
+        flowSnapshot.getSnapshotMetadata().setBucketIdentifier(null);
+        flowSnapshot.getSnapshotMetadata().setBranch(null);
+        flowSnapshot.getSnapshotMetadata().setVersion(null);
+        flowSnapshot.getSnapshotMetadata().setComments(null);
+        flowSnapshot.getSnapshotMetadata().setTimestamp(0);
+
+        // replace the id of the top level group and all of its references with a constant value prior to serializing to avoid
+        // unnecessary diffs when different instances of the same flow are imported and have different top-level PG ids
+        final String originalFlowContentsGroupId = replaceGroupId(flowSnapshot.getFlowContents(), FLOW_CONTENTS_GROUP_ID);
+        final Position originalFlowContentsPosition = replacePosition(flowSnapshot.getFlowContents(), new Position(0, 0));
+
+        final GitHubCreateContentRequest createContentRequest = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(commitMessage)
+                .existingContentSha(previousSha)
+                .build();
+
+        final GHContentUpdateResponse createContentResponse = repositoryClient.createContent(createContentRequest);
+        final String createContentCommitSha = createContentResponse.getCommit().getSha();
+
+        final VersionedFlowCoordinates versionedFlowCoordinates = new VersionedFlowCoordinates();
+        versionedFlowCoordinates.setRegistryId(getIdentifier());
+        versionedFlowCoordinates.setBranch(flowLocation.getBranch());
+        versionedFlowCoordinates.setBucketId(flowLocation.getBucketId());
+        versionedFlowCoordinates.setFlowId(flowLocation.getFlowId());
+        versionedFlowCoordinates.setVersion(createContentCommitSha);
+
+        flowSnapshot.getFlowContents().setVersionedFlowCoordinates(versionedFlowCoordinates);
+        flowSnapshot.getFlow().setBranch(branch);
+        flowSnapshot.getSnapshotMetadata().setBranch(branch);
+        flowSnapshot.getSnapshotMetadata().setVersion(createContentCommitSha);
+        flowSnapshot.setLatest(true);
+
+        // populate outgoing bucket references
+        updateBucketReferences(flowSnapshot, flowLocation.getBucketId());
+
+        // set back to the original id so that the returned snapshot is has the correct values from what was passed in
+        replaceGroupId(flowSnapshot.getFlowContents(), originalFlowContentsGroupId);
+        replacePosition(flowSnapshot.getFlowContents(), originalFlowContentsPosition);
+
+        return flowSnapshot;
+    }
+
+    @Override
+    public Set<RegisteredFlowSnapshotMetadata> getFlowVersions(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation)
+            throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Set<RegisteredFlowSnapshotMetadata> snapshotMetadataSet = new LinkedHashSet<>();
+        for (final GHCommit ghCommit : repositoryClient.getCommits(filePath, branch)) {
+            final RegisteredFlowSnapshotMetadata snapshotMetadata = createSnapshotMetadata(ghCommit, flowLocation);
+            if (REGISTER_FLOW_COMMENT.equals(snapshotMetadata.getComments())) {
+                continue;
+            }
+            snapshotMetadataSet.add(snapshotMetadata);
+        }
+        return snapshotMetadataSet;
+    }
+
+    @Override
+    public Optional<String> getLatestVersion(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final List<GHCommit> commits = repositoryClient.getCommits(filePath, branch);
+        final String latestVersion = commits.isEmpty() ? null : commits.getFirst().getSHA1();
+        return Optional.ofNullable(latestVersion);
+    }
+
+    @Override
+    public String generateFlowId(final String flowName) {
+        return flowName.trim()
+                .replaceAll("\\s", "-") // replace whitespace with -
+                .replaceAll("[^a-zA-Z0-9-]", "") // replace all other invalid chars with empty string
+                .replaceAll("(-)\\1+", "$1"); // replace consecutive - with single -

Review Comment:
   The call to `registerFlow` will fail if a file with the same name already exists.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594235055


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;

Review Comment:
   Looking at this closer and I think I am following now...
   
   Currently the GitHub impl is not setting `storageLocation` on the coordinates, we should set it to something like `git@...` and then implement this method to return true when the location starts with that. 
   
   This way when it gets to the code in the framework that tries to locate the registry client for a nested versioned flow, it can try the correct clients first...
   ```private List<FlowRegistryClientNode> getRegistryClientsForInternalFlow(final String storageLocation) {
           // Sort clients based on whether or not they believe they are applicable for the given storage location
           final List<FlowRegistryClientNode> matchingClients = new ArrayList<>(flowManager.getAllFlowRegistryClients());
           matchingClients.sort(Comparator.comparing(client -> client.isStorageLocationApplicable(storageLocation) ? -1 : 1));
           return matchingClients;
       }
   ```
   
   Is this what you were suggesting?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594079247


##########
nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java:
##########
@@ -5357,6 +5355,8 @@ public RegisteredFlow registerVersionedFlow(final String registryId, final Regis
         }
 
         try {
+            final String generatedId = registry.generateFlowId(flow.getName());

Review Comment:
   Previously the framework generated a UUID in service facade before calling the registerFlow method in registry client, but for GitHub we don't want a UUID, so this just allows the framework to ask the client to generate the id, and the default behavior can return a UUID like before, and the GitHub impl can do something different.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594106906


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());

Review Comment:
   The method `createFlowRegistryBucket` may sound misleading, but it is not creating a bucket in GitHub, it is just creating a bucket object to represent whatever the requested bucket name was. This is because buckets in GitHub aren't really something that needs to exist before hand... if you perform a commit with a path like `Bucket1/MyFlow.json` and `Bucket1` didn't exist, it is created during the commit.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594084373


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubRepositoryClient.java:
##########
@@ -0,0 +1,404 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+
+package org.apache.nifi.github;
+
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+import org.kohsuke.github.GHRef;
+import org.kohsuke.github.GHRepository;
+import org.kohsuke.github.GitHub;
+import org.kohsuke.github.GitHubBuilder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * Client to encapsulate access to a GitHub Repository through the Hub4j GitHub client.
+ */
+public class GitHubRepositoryClient {
+
+    private static final Logger LOGGER = LoggerFactory.getLogger(GitHubRepositoryClient.class);
+
+    private static final String BRANCH_REF_PATTERN = "refs/heads/%s";
+    private static final int COMMIT_PAGE_SIZE = 50;
+
+    private final GHRepository repository;
+    private final String repoPath;
+    private final GitHubAuthenticationType authenticationType;
+
+    private GitHubRepositoryClient(final Builder builder) throws IOException {
+        final GitHubBuilder gitHubBuilder = new GitHubBuilder().withEndpoint(builder.apiUrl);
+
+        authenticationType = builder.authenticationType;
+        switch (authenticationType) {
+            case PERSONAL_ACCESS_TOKEN -> gitHubBuilder.withOAuthToken(builder.personalAccessToken);
+            case APP_INSTALLATION_TOKEN -> gitHubBuilder.withAppInstallationToken(builder.appInstallationToken);
+        }
+
+        final GitHub gitHub = gitHubBuilder.build();
+        try {
+            repository = gitHub.getRepository(builder.repoOwner + "/" + builder.repoName);
+        } catch (final IOException e) {
+            LOGGER.error("Unable to access GitHub repository [{}]", builder.repoName, e);
+            throw e;
+        }
+        repoPath = builder.repoPath;
+    }
+
+    /**
+     * @return the authentication type this client is configured with
+     */
+    public GitHubAuthenticationType getAuthenticationType() {
+        return authenticationType;
+    }
+
+    /**
+     * Creates the content specified by the given builder.
+     *
+     * @param request the request for the content to create
+     * @return the update response
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public GHContentUpdateResponse createContent(final GitHubCreateContentRequest request) throws IOException, FlowRegistryException {
+        final String branch = request.getBranch();
+        final String resolvedPath = getResolvedPath(request.getPath());
+        LOGGER.debug("Creating content at path [{}] on branch [{}] in repo [{}] ", resolvedPath, branch, repository.getName());
+        return execute(() -> {
+            try {
+                return repository.createContent()
+                        .branch(branch)
+                        .path(resolvedPath)
+                        .content(request.getContent())
+                        .message(request.getMessage())
+                        .sha(request.getExistingContentSha())
+                        .commit();
+            } catch (final FileNotFoundException fnf) {
+                throwPathOrBranchNotFound(resolvedPath, branch);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the names of all the branches in the repo.
+     *
+     * @return the set of all branches in the repo
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public Set<String> getBranches() throws IOException, FlowRegistryException {
+        LOGGER.debug("Getting branches for repo [{}]", repository.getName());
+        return execute(() -> repository.getBranches().keySet());
+    }
+
+    /**
+     * Gets an InputStream to read the latest content of the given path from the given branch.
+     * The returned stream already contains the contents of the requested file.
+     *
+     * @param path the path to the content
+     * @param branch the branch
+     * @return an input stream containing the contents of the path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public InputStream getContentFromBranch(final String path, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting content for [{}] from branch [{}] in repo [{}] ", resolvedPath, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHContent ghContent = repository.getFileContent(resolvedPath, branchRef);
+                return ghContent.read();
+            } catch (final FileNotFoundException e) {
+                throwPathOrBranchNotFound(resolvedPath, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the content of the given path from the given commit.
+     * The returned stream already contains the contents of the requested file.
+     *
+     * @param path the path to the content
+     * @param commitSha the commit SHA
+     * @return an input stream containing the contents of the path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public InputStream getContentFromCommit(final String path, final String commitSha) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        LOGGER.debug("Getting content for [{}] from commit [{}] in repo [{}] ", resolvedPath, commitSha, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHContent ghContent = repository.getFileContent(resolvedPath, commitSha);
+                return ghContent.read();
+            } catch (final FileNotFoundException fnf) {
+                throw new FlowRegistryException("Path [" + resolvedPath + "] or Commit [" + commitSha + "] not found");
+            }
+        });
+    }
+
+    /**
+     * Gets the commits for a given path on a given branch.
+     *
+     * @param path the path
+     * @param branch the branch
+     * @return the list of commits for the given path
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public List<GHCommit> getCommits(final String path, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting commits for [{}] from branch [{}] in repo [{}]", resolvedPath, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHRef branchGhRef = repository.getRef(branchRef);
+                return repository.queryCommits()
+                        .path(resolvedPath)
+                        .from(branchGhRef.getObject().getSha())
+                        .pageSize(COMMIT_PAGE_SIZE)
+                        .list()
+                        .toList();
+            } catch (final FileNotFoundException fnf) {
+                throwPathOrBranchNotFound(resolvedPath, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the names of the directories contained within the given directory.
+     *
+     * @param directory the directory to list
+     * @param branch the branch
+     * @return the set of directory names
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public Set<String> getDirectoryNames(final String directory, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedDirectory = getResolvedPath(directory);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting directory names for [{}] from branch [{}] in repo [{}] ", resolvedDirectory, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                return repository.getDirectoryContent(resolvedDirectory, branchRef).stream()
+                        .filter(GHContent::isDirectory)
+                        .map(GHContent::getName)
+                        .collect(Collectors.toSet());
+            } catch (final FileNotFoundException e) {
+                throwPathOrBranchNotFound(resolvedDirectory, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the names of the directories container within the given directory.
+     *
+     * @param directory the directory to list
+     * @param branch the branch
+     * @return the set of file names
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public Set<String> getFileNames(final String directory, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedDirectory = getResolvedPath(directory);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting file names for [{}] from branch [{}] in repo [{}] ", resolvedDirectory, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                return repository.getDirectoryContent(resolvedDirectory, branchRef).stream()
+                        .filter(GHContent::isFile)
+                        .map(GHContent::getName)
+                        .collect(Collectors.toSet());
+            } catch (final FileNotFoundException e) {
+                throwPathOrBranchNotFound(resolvedDirectory, branchRef);
+                return null;
+            }
+        });
+    }
+
+    /**
+     * Gets the current SHA for the given path from the given branch.
+     *
+     * @param path the path to the content
+     * @param branch the branch
+     * @return current sha for the given file, or empty optional
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     */
+    public Optional<String> getContentSha(final String path, final String branch) throws IOException, FlowRegistryException {
+        final String resolvedPath = getResolvedPath(path);
+        final String branchRef = BRANCH_REF_PATTERN.formatted(branch);
+        LOGGER.debug("Getting content SHA for [{}] from branch [{}] in repo [{}] ", resolvedPath, branch, repository.getName());
+
+        return execute(() -> {
+            try {
+                final GHContent ghContent = repository.getFileContent(resolvedPath, branchRef);
+                return Optional.of(ghContent.getSha());
+            } catch (final FileNotFoundException e) {
+                LOGGER.warn("Unable to get content SHA for [{}] from branch [{}] because content does not exist", resolvedPath, branch);
+                return Optional.empty();
+            }
+        });
+    }
+
+    /**
+     * Deletes the contents for the given file on the given branch.
+     *
+     * @param filePath the file path to delete
+     * @param commitMessage the commit message for the delete commit
+     * @param branch the branch to delete from
+     * @return the deleted content
+     *
+     * @throws IOException if an I/O error happens calling GitHub
+     * @throws FlowRegistryException if a non I/O error happens calling GitHub
+     */
+    public GHContent deleteContent(final String filePath, final String commitMessage, final String branch) throws FlowRegistryException, IOException {
+        final String resolvedPath = getResolvedPath(filePath);
+        LOGGER.debug("Deleting file [{}] in repo [{}] on branch [{}]", resolvedPath, repository.getName(), branch);
+        return execute(() -> {
+            try {
+                GHContent ghContent = repository.getFileContent(resolvedPath);
+                ghContent.delete(commitMessage, branch);
+                return ghContent;
+            } catch (final FileNotFoundException fnf) {
+                throwPathOrBranchNotFound(resolvedPath, branch);
+                return null;
+            }
+        });
+    }
+
+    private String getResolvedPath(final String path) {
+        return repoPath == null ? path : repoPath + "/" + path;
+    }
+
+    private void throwPathOrBranchNotFound(final String path, final String branch) throws FlowRegistryException {
+        throw new FlowRegistryException("Path [" + path + "] or Branch [" + branch + "] not found");

Review Comment:
   Thanks for pointing out the sub-classes, I wasn't aware of those. There might be a few cases where we could leverage one of them, but I don't think we can use it here because we don't actually know which part doesn't exist. It could be any one of the bucket directory, the snapshot file name, or the branch. Unless we make a lot of individual requests to check each piece, but I don't want to do that because then we end up making lots of remote calls to GitHub for one single operation, and during background synchronize of many versioned PGs, it will end up being thousands of calls to GitHub.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594087526


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubRepositoryClient.java:
##########
@@ -0,0 +1,404 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+
+package org.apache.nifi.github;
+
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+import org.kohsuke.github.GHRef;
+import org.kohsuke.github.GHRepository;
+import org.kohsuke.github.GitHub;
+import org.kohsuke.github.GitHubBuilder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * Client to encapsulate access to a GitHub Repository through the Hub4j GitHub client.
+ */
+public class GitHubRepositoryClient {
+
+    private static final Logger LOGGER = LoggerFactory.getLogger(GitHubRepositoryClient.class);
+
+    private static final String BRANCH_REF_PATTERN = "refs/heads/%s";
+    private static final int COMMIT_PAGE_SIZE = 50;
+
+    private final GHRepository repository;
+    private final String repoPath;
+    private final GitHubAuthenticationType authenticationType;
+
+    private GitHubRepositoryClient(final Builder builder) throws IOException {
+        final GitHubBuilder gitHubBuilder = new GitHubBuilder().withEndpoint(builder.apiUrl);
+
+        authenticationType = builder.authenticationType;
+        switch (authenticationType) {
+            case PERSONAL_ACCESS_TOKEN -> gitHubBuilder.withOAuthToken(builder.personalAccessToken);
+            case APP_INSTALLATION_TOKEN -> gitHubBuilder.withAppInstallationToken(builder.appInstallationToken);

Review Comment:
   No, it is one or the other.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "bbende (via GitHub)" <gi...@apache.org>.
bbende commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1594149799


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,633 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one or more
+ *  contributor license agreements.  See the NOTICE file distributed with
+ *  this work for additional information regarding copyright ownership.
+ *  The ASF licenses this file to You under the Apache License, Version 2.0
+ *  (the "License"); you may not use this file except in compliance with
+ *  the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor AUTHENTICATION_TYPE = new PropertyDescriptor.Builder()
+            .name("Authentication Type")
+            .description("The type of authentication to use for accessing GitHub")
+            .allowableValues(GitHubAuthenticationType.values())
+            .defaultValue(GitHubAuthenticationType.NONE.name())
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor PERSONAL_ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Personal Access Token")
+            .description("The personal access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.PERSONAL_ACCESS_TOKEN.name())
+            .build();
+
+    static final PropertyDescriptor APP_INSTALLATION_TOKEN = new PropertyDescriptor.Builder()
+            .name("App Installation Token")
+            .description("The app installation token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .sensitive(true)
+            .dependsOn(AUTHENTICATION_TYPE, GitHubAuthenticationType.APP_INSTALLATION_TOKEN.name())
+            .build();
+
+    static final List<PropertyDescriptor> PROPERTY_DESCRIPTORS = List.of(
+            GITHUB_API_URL,
+            REPOSITORY_OWNER,
+            REPOSITORY_NAME,
+            REPOSITORY_BRANCH,
+            REPOSITORY_PATH,
+            AUTHENTICATION_TYPE,
+            PERSONAL_ACCESS_TOKEN,
+            APP_INSTALLATION_TOKEN
+    );
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return PROPERTY_DESCRIPTORS;
+    }
+
+    @Override
+    protected Collection<ValidationResult> customValidate(final ValidationContext validationContext) {
+        final List<ValidationResult> results = new ArrayList<>(super.customValidate(validationContext));
+
+        final String repoPath = validationContext.getProperty(REPOSITORY_PATH).getValue();
+        if (repoPath != null && (repoPath.startsWith("/") || repoPath.endsWith("/"))) {
+            results.add(new ValidationResult.Builder()
+                    .subject(REPOSITORY_PATH.getDisplayName())
+                    .valid(false)
+                    .explanation("Path can not start or end with /")
+                    .build());
+        }
+
+        return results;
+    }
+
+    @Override
+    public void onPropertyModified(final PropertyDescriptor descriptor, final String oldValue, final String newValue) {
+        super.onPropertyModified(descriptor, oldValue, newValue);
+        synchronized (this) {
+            initialized.set(false);
+        }
+    }
+
+    @Override
+    public boolean isStorageLocationApplicable(final FlowRegistryClientConfigurationContext context, final String location) {
+        return false;
+    }
+
+    @Override
+    public boolean isBranchingSupported(final FlowRegistryClientConfigurationContext context) {
+        return true;
+    }
+
+    @Override
+    public Set<FlowRegistryBranch> getBranches(final FlowRegistryClientConfigurationContext context) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        return repositoryClient.getBranches().stream()
+                .map(branchName -> {
+                    final FlowRegistryBranch flowRegistryBranch = new FlowRegistryBranch();
+                    flowRegistryBranch.setName(branchName);
+                    return flowRegistryBranch;
+                }).collect(Collectors.toSet());
+    }
+
+    @Override
+    public FlowRegistryBranch getDefaultBranch(final FlowRegistryClientConfigurationContext context) {
+        final FlowRegistryBranch defaultBranch = new FlowRegistryBranch();
+        defaultBranch.setName(context.getProperty(REPOSITORY_BRANCH).getValue());
+        return defaultBranch;
+    }
+
+    @Override
+    public Set<FlowRegistryBucket> getBuckets(final FlowRegistryClientConfigurationContext context, final String branch) throws IOException, FlowRegistryException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+        final Set<FlowRegistryBucket> buckets = repositoryClient.getDirectoryNames("", branch).stream()
+                .map(this::createFlowRegistryBucket)
+                .collect(Collectors.toSet());
+
+        // if the repository has no top-level directories, then return a default bucket entry, this won't exist in the repository until the first time a flow is saved to it
+        return buckets.isEmpty() ? Set.of(createFlowRegistryBucket(DEFAULT_BUCKET_NAME)) : buckets;
+    }
+
+    @Override
+    public FlowRegistryBucket getBucket(final FlowRegistryClientConfigurationContext context, final BucketLocation bucketLocation) throws FlowRegistryException, IOException {
+        return createFlowRegistryBucket(bucketLocation.getBucketId());
+    }
+
+    @Override
+    public RegisteredFlow registerFlow(final FlowRegistryClientConfigurationContext context, final RegisteredFlow flow) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flow.getBranch();
+        final FlowLocation flowLocation = new FlowLocation(branch, flow.getBucketIdentifier(), flow.getIdentifier());
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final Optional<String> existingFileSha = repositoryClient.getContentSha(filePath, branch);
+        if (existingFileSha.isPresent()) {
+            throw new FlowRegistryException("Another flow is already registered at [" + filePath + "] on branch [" + branch + "]");
+        }
+
+        // Clear values we don't want in the json stored in GitHub
+        final String originalBucketId = flow.getBucketIdentifier();
+        flow.setBucketIdentifier(null);
+        flow.setBucketName(null);
+        flow.setBranch(null);
+
+        final RegisteredFlowSnapshot flowSnapshot = new RegisteredFlowSnapshot();
+        flowSnapshot.setBucket(null);
+        flowSnapshot.setFlow(flow);
+
+        final GitHubCreateContentRequest request = GitHubCreateContentRequest.builder()
+                .branch(branch)
+                .path(filePath)
+                .content(OBJECT_MAPPER.writeValueAsString(flowSnapshot))
+                .message(REGISTER_FLOW_COMMENT)
+                .build();
+
+        repositoryClient.createContent(request);
+
+        // Re-populate fields before returning
+        flow.setBucketName(originalBucketId);
+        flow.setBucketIdentifier(originalBucketId);
+        flow.setBranch(branch);
+
+        return flow;
+    }
+
+    @Override
+    public RegisteredFlow deregisterFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final GitHubRepositoryClient repositoryClient = getRepositoryClient(context);
+
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+        final GHContent deletedSnapshotContent = repositoryClient.deleteContent(filePath, DEREGISTER_FLOW_COMMENT, branch);
+
+        final RegisteredFlowSnapshot deletedSnapshot = getSnapshot(deletedSnapshotContent.read());
+        updateBucketReferences(deletedSnapshot, flowLocation.getBucketId());
+        return deletedSnapshot.getFlow();
+    }
+
+    @Override
+    public RegisteredFlow getFlow(final FlowRegistryClientConfigurationContext context, final FlowLocation flowLocation) throws FlowRegistryException, IOException {
+        final String branch = flowLocation.getBranch();
+        final String filePath = getSnapshotFilePath(flowLocation);
+
+        final RegisteredFlowSnapshot existingSnapshot = getSnapshot(filePath, branch);

Review Comment:
   Will look into it, but mentioned in other comments how we don't really know if it's the bucket, file, or branch that doesn't exist.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-13105 Implement a FlowRegistryClient that use GitHub for version… [nifi]

Posted by "exceptionfactory (via GitHub)" <gi...@apache.org>.
exceptionfactory commented on code in PR #8765:
URL: https://github.com/apache/nifi/pull/8765#discussion_r1592892652


##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubCreateContentRequest.java:
##########
@@ -0,0 +1,101 @@
+/*
+ *
+ *  * Licensed to the Apache Software Foundation (ASF) under one or more
+ *  * contributor license agreements.  See the NOTICE file distributed with
+ *  * this work for additional information regarding copyright ownership.
+ *  * The ASF licenses this file to You under the Apache License, Version 2.0
+ *  * (the "License"); you may not use this file except in compliance with
+ *  * the License.  You may obtain a copy of the License at
+ *  *
+ *  *     http://www.apache.org/licenses/LICENSE-2.0
+ *  *
+ *  * Unless required by applicable law or agreed to in writing, software
+ *  * distributed under the License is distributed on an "AS IS" BASIS,
+ *  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  * See the License for the specific language governing permissions and
+ *  * limitations under the License.

Review Comment:
   This appears to have unnecessary padding.



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,609 @@
+/*
+ *
+ *  * Licensed to the Apache Software Foundation (ASF) under one or more

Review Comment:
   This appears to have unnecessary padding.



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/VersionedComponentModule.java:
##########
@@ -0,0 +1,64 @@
+/*
+ *
+ *  * Licensed to the Apache Software Foundation (ASF) under one or more
+ *  * contributor license agreements.  See the NOTICE file distributed with
+ *  * this work for additional information regarding copyright ownership.
+ *  * The ASF licenses this file to You under the Apache License, Version 2.0
+ *  * (the "License"); you may not use this file except in compliance with
+ *  * the License.  You may obtain a copy of the License at
+ *  *
+ *  *     http://www.apache.org/licenses/LICENSE-2.0
+ *  *
+ *  * Unless required by applicable law or agreed to in writing, software
+ *  * distributed under the License is distributed on an "AS IS" BASIS,
+ *  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  * See the License for the specific language governing permissions and
+ *  * limitations under the License.

Review Comment:
   Unnecessary padding



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,609 @@
+/*
+ *
+ *  * Licensed to the Apache Software Foundation (ASF) under one or more
+ *  * contributor license agreements.  See the NOTICE file distributed with
+ *  * this work for additional information regarding copyright ownership.
+ *  * The ASF licenses this file to You under the Apache License, Version 2.0
+ *  * (the "License"); you may not use this file except in compliance with
+ *  * the License.  You may obtain a copy of the License at
+ *  *
+ *  *     http://www.apache.org/licenses/LICENSE-2.0
+ *  *
+ *  * Unless required by applicable law or agreed to in writing, software
+ *  * distributed under the License is distributed on an "AS IS" BASIS,
+ *  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  * See the License for the specific language governing permissions and
+ *  * limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Access Token")
+            .description("The access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .sensitive(true)
+            .build();
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "Default";

Review Comment:
   What do you think about changing this to `default` lowercase instead?



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubRepositoryClient.java:
##########
@@ -0,0 +1,394 @@
+/*
+ *
+ *  * Licensed to the Apache Software Foundation (ASF) under one or more
+ *  * contributor license agreements.  See the NOTICE file distributed with
+ *  * this work for additional information regarding copyright ownership.
+ *  * The ASF licenses this file to You under the Apache License, Version 2.0
+ *  * (the "License"); you may not use this file except in compliance with
+ *  * the License.  You may obtain a copy of the License at
+ *  *
+ *  *     http://www.apache.org/licenses/LICENSE-2.0
+ *  *
+ *  * Unless required by applicable law or agreed to in writing, software
+ *  * distributed under the License is distributed on an "AS IS" BASIS,
+ *  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  * See the License for the specific language governing permissions and
+ *  * limitations under the License.
+ *
+ */
+
+package org.apache.nifi.github;
+
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+import org.kohsuke.github.GHRef;
+import org.kohsuke.github.GHRepository;
+import org.kohsuke.github.GitHub;
+import org.kohsuke.github.GitHubBuilder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * Client to encapsulate access to a GitHub Repository through the Hub4j GitHub client.
+ */
+public class GitHubRepositoryClient {
+
+    private static final Logger LOGGER = LoggerFactory.getLogger(GitHubRepositoryClient.class);
+
+    private static final String BRANCH_REF_PATTERN = "refs/heads/%s";
+    private static final int COMMIT_PAGE_SIZE = 50;
+
+    private final GHRepository repository;
+    private final String repoPath;
+    private final boolean authenticationPresent;
+
+    private GitHubRepositoryClient(final Builder builder) throws IOException {
+        final GitHubBuilder gitHubBuilder = new GitHubBuilder().withEndpoint(builder.apiUrl);
+
+        final String accessToken = builder.accessToken;
+        if (accessToken != null) {
+            gitHubBuilder.withOAuthToken(accessToken);
+            authenticationPresent = true;
+        } else {
+            authenticationPresent = false;
+        }

Review Comment:
   I recommend reversing this logic:
   ```suggestion
           if (accessToken == null) {
               authenticationPresent = false;
           } else {
               gitHubBuilder.withOAuthToken(accessToken);
               authenticationPresent = true;
           }
   ```



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubFlowRegistryClient.java:
##########
@@ -0,0 +1,609 @@
+/*
+ *
+ *  * Licensed to the Apache Software Foundation (ASF) under one or more
+ *  * contributor license agreements.  See the NOTICE file distributed with
+ *  * this work for additional information regarding copyright ownership.
+ *  * The ASF licenses this file to You under the Apache License, Version 2.0
+ *  * (the "License"); you may not use this file except in compliance with
+ *  * the License.  You may obtain a copy of the License at
+ *  *
+ *  *     http://www.apache.org/licenses/LICENSE-2.0
+ *  *
+ *  * Unless required by applicable law or agreed to in writing, software
+ *  * distributed under the License is distributed on an "AS IS" BASIS,
+ *  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  * See the License for the specific language governing permissions and
+ *  * limitations under the License.
+ *
+ */
+package org.apache.nifi.github;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.type.TypeFactory;
+import com.fasterxml.jackson.module.jakarta.xmlbind.JakartaXmlBindAnnotationIntrospector;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flow.ConnectableComponent;
+import org.apache.nifi.flow.Position;
+import org.apache.nifi.flow.VersionedComponent;
+import org.apache.nifi.flow.VersionedConnection;
+import org.apache.nifi.flow.VersionedFlowCoordinates;
+import org.apache.nifi.flow.VersionedProcessGroup;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.registry.flow.AbstractFlowRegistryClient;
+import org.apache.nifi.registry.flow.BucketLocation;
+import org.apache.nifi.registry.flow.FlowLocation;
+import org.apache.nifi.registry.flow.FlowRegistryBranch;
+import org.apache.nifi.registry.flow.FlowRegistryBucket;
+import org.apache.nifi.registry.flow.FlowRegistryClientConfigurationContext;
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.apache.nifi.registry.flow.FlowRegistryPermissions;
+import org.apache.nifi.registry.flow.FlowVersionLocation;
+import org.apache.nifi.registry.flow.RegisterAction;
+import org.apache.nifi.registry.flow.RegisteredFlow;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshot;
+import org.apache.nifi.registry.flow.RegisteredFlowSnapshotMetadata;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link org.apache.nifi.registry.flow.FlowRegistryClient} that uses GitHub for version controlling flows.
+ */
+public class GitHubFlowRegistryClient extends AbstractFlowRegistryClient {
+
+    static final PropertyDescriptor GITHUB_API_URL = new PropertyDescriptor.Builder()
+            .name("GitHub API URL")
+            .description("The URL of the GitHub API")
+            .addValidator(StandardValidators.URL_VALIDATOR)
+            .defaultValue("https://api.github.com/")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_NAME = new PropertyDescriptor.Builder()
+            .name("Repository Name")
+            .description("The name of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_OWNER = new PropertyDescriptor.Builder()
+            .name("Repository Owner")
+            .description("The owner of the repository")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_BRANCH = new PropertyDescriptor.Builder()
+            .name("Default Branch")
+            .description("The default branch to use for this client")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .defaultValue("main")
+            .required(true)
+            .build();
+
+    static final PropertyDescriptor REPOSITORY_PATH = new PropertyDescriptor.Builder()
+            .name("Repository Path")
+            .description("The path with in the repository that this client will use to store all data. " +
+                    "If left blank, then the root of the repository will be used.")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .build();
+
+    static final PropertyDescriptor ACCESS_TOKEN = new PropertyDescriptor.Builder()
+            .name("Access Token")
+            .description("The access token to use for authentication")
+            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+            .required(false)
+            .sensitive(true)
+            .build();
+
+    private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
+            .serializationInclusion(JsonInclude.Include.NON_NULL)
+            .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
+            .annotationIntrospector(new JakartaXmlBindAnnotationIntrospector(TypeFactory.defaultInstance()))
+            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+            .configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
+            .enable(SerializationFeature.INDENT_OUTPUT)
+            .addModule(new VersionedComponentModule())
+            .build();
+
+    static final String DEFAULT_BUCKET_NAME = "Default";
+    static final String DEFAULT_BUCKET_KEEP_FILE_PATH = DEFAULT_BUCKET_NAME + "/.keep";
+    static final String DEFAULT_BUCKET_KEEP_FILE_CONTENT = "Do Not Delete";
+    static final String DEFAULT_BUCKET_KEEP_FILE_MESSAGE = "Creating Default bucket";
+
+    static final String REGISTER_FLOW_COMMENT = "Register Flow";
+    static final String DEREGISTER_FLOW_COMMENT = "Deregister Flow";
+    static final String DEFAULT_FLOW_SNAPSHOT_COMMIT_MESSAGE = "Saving Flow Snapshot";
+    static final String SNAPSHOT_FILE_EXTENSION = ".json";
+    static final String SNAPSHOT_FILE_PATH_FORMAT = "%s/%s" + SNAPSHOT_FILE_EXTENSION;
+    static final String FLOW_CONTENTS_GROUP_ID = "flow-contents-group";
+
+    private volatile GitHubRepositoryClient repositoryClient;
+    private final AtomicBoolean initialized = new AtomicBoolean(false);
+
+    @Override
+    protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+        return List.of(
+                GITHUB_API_URL,
+                REPOSITORY_OWNER,
+                REPOSITORY_NAME,
+                REPOSITORY_BRANCH,
+                REPOSITORY_PATH,
+                ACCESS_TOKEN
+        );

Review Comment:
   Instead of defining the List in every method call, the list of properties should be defined as a static final variable.



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubRepositoryClient.java:
##########
@@ -0,0 +1,394 @@
+/*
+ *
+ *  * Licensed to the Apache Software Foundation (ASF) under one or more
+ *  * contributor license agreements.  See the NOTICE file distributed with
+ *  * this work for additional information regarding copyright ownership.
+ *  * The ASF licenses this file to You under the Apache License, Version 2.0
+ *  * (the "License"); you may not use this file except in compliance with
+ *  * the License.  You may obtain a copy of the License at
+ *  *
+ *  *     http://www.apache.org/licenses/LICENSE-2.0
+ *  *
+ *  * Unless required by applicable law or agreed to in writing, software
+ *  * distributed under the License is distributed on an "AS IS" BASIS,
+ *  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  * See the License for the specific language governing permissions and
+ *  * limitations under the License.
+ *
+ */
+
+package org.apache.nifi.github;
+
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+import org.kohsuke.github.GHRef;
+import org.kohsuke.github.GHRepository;
+import org.kohsuke.github.GitHub;
+import org.kohsuke.github.GitHubBuilder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * Client to encapsulate access to a GitHub Repository through the Hub4j GitHub client.
+ */
+public class GitHubRepositoryClient {
+
+    private static final Logger LOGGER = LoggerFactory.getLogger(GitHubRepositoryClient.class);
+
+    private static final String BRANCH_REF_PATTERN = "refs/heads/%s";
+    private static final int COMMIT_PAGE_SIZE = 50;
+
+    private final GHRepository repository;
+    private final String repoPath;
+    private final boolean authenticationPresent;
+
+    private GitHubRepositoryClient(final Builder builder) throws IOException {
+        final GitHubBuilder gitHubBuilder = new GitHubBuilder().withEndpoint(builder.apiUrl);
+
+        final String accessToken = builder.accessToken;
+        if (accessToken != null) {
+            gitHubBuilder.withOAuthToken(accessToken);
+            authenticationPresent = true;
+        } else {
+            authenticationPresent = false;
+        }
+
+        final GitHub gitHub = gitHubBuilder.build();
+        try {
+            repository = gitHub.getRepository(builder.repoOwner + "/" + builder.repoName);
+        } catch (final IOException e) {
+            LOGGER.error("Unable to access GitHub repository [{}] due to: {}", builder.repoName, e.getMessage(), e);
+            throw e;
+        }
+        repoPath = builder.repoPath;
+    }
+
+    /**
+     * @return true if the client is configured with credentials to authenticate
+     */
+    public boolean isAuthenticationPresent() {

Review Comment:
   Perhaps naming this `isAuthenticationConfigured`?



##########
nifi-extension-bundles/nifi-github-bundle/nifi-github-extensions/src/main/java/org/apache/nifi/github/GitHubRepositoryClient.java:
##########
@@ -0,0 +1,394 @@
+/*
+ *
+ *  * Licensed to the Apache Software Foundation (ASF) under one or more
+ *  * contributor license agreements.  See the NOTICE file distributed with
+ *  * this work for additional information regarding copyright ownership.
+ *  * The ASF licenses this file to You under the Apache License, Version 2.0
+ *  * (the "License"); you may not use this file except in compliance with
+ *  * the License.  You may obtain a copy of the License at
+ *  *
+ *  *     http://www.apache.org/licenses/LICENSE-2.0
+ *  *
+ *  * Unless required by applicable law or agreed to in writing, software
+ *  * distributed under the License is distributed on an "AS IS" BASIS,
+ *  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  * See the License for the specific language governing permissions and
+ *  * limitations under the License.
+ *
+ */
+
+package org.apache.nifi.github;
+
+import org.apache.nifi.registry.flow.FlowRegistryException;
+import org.kohsuke.github.GHCommit;
+import org.kohsuke.github.GHContent;
+import org.kohsuke.github.GHContentUpdateResponse;
+import org.kohsuke.github.GHRef;
+import org.kohsuke.github.GHRepository;
+import org.kohsuke.github.GitHub;
+import org.kohsuke.github.GitHubBuilder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * Client to encapsulate access to a GitHub Repository through the Hub4j GitHub client.
+ */
+public class GitHubRepositoryClient {
+
+    private static final Logger LOGGER = LoggerFactory.getLogger(GitHubRepositoryClient.class);
+
+    private static final String BRANCH_REF_PATTERN = "refs/heads/%s";
+    private static final int COMMIT_PAGE_SIZE = 50;
+
+    private final GHRepository repository;
+    private final String repoPath;
+    private final boolean authenticationPresent;
+
+    private GitHubRepositoryClient(final Builder builder) throws IOException {
+        final GitHubBuilder gitHubBuilder = new GitHubBuilder().withEndpoint(builder.apiUrl);
+
+        final String accessToken = builder.accessToken;
+        if (accessToken != null) {
+            gitHubBuilder.withOAuthToken(accessToken);
+            authenticationPresent = true;
+        } else {
+            authenticationPresent = false;
+        }
+
+        final GitHub gitHub = gitHubBuilder.build();
+        try {
+            repository = gitHub.getRepository(builder.repoOwner + "/" + builder.repoName);
+        } catch (final IOException e) {
+            LOGGER.error("Unable to access GitHub repository [{}] due to: {}", builder.repoName, e.getMessage(), e);

Review Comment:
   The `due to:` portion is not necessary as it is included in the stack trace.
   ```suggestion
               LOGGER.error("Unable to access GitHub repository [{}]", builder.repoName, e);
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org