You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by si...@apache.org on 2020/11/12 00:56:28 UTC
[pulsar-connectors] branch master updated: Build connectors (#1)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar-connectors.git
The following commit(s) were added to refs/heads/master by this push:
new 33843ab Build connectors (#1)
33843ab is described below
commit 33843ab3e395e10e473a323881a12adf35158202
Author: Sijie Guo <si...@apache.org>
AuthorDate: Wed Nov 11 17:56:21 2020 -0700
Build connectors (#1)
---
.github/ISSUE_TEMPLATE.md | 14 +++
.github/ISSUE_TEMPLATE/bug_report.md | 30 +++++++
.github/ISSUE_TEMPLATE/enhancement_request.md | 20 +++++
.github/ISSUE_TEMPLATE/feature_request.md | 20 +++++
.github/PULL_REQUEST_TEMPLATE.md | 74 ++++++++++++++++
.github/workflows/unit-test.yaml | 103 ++++++++++++++++++++++
.gitignore | 90 +++++++++++++++++++
pom.xml | 119 ++++++++++++++++++++++++++
pulsar-io/pom.xml | 21 +----
9 files changed, 471 insertions(+), 20 deletions(-)
diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md
new file mode 100644
index 0000000..00c4b58
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE.md
@@ -0,0 +1,14 @@
+#### Expected behavior
+
+Tell us what should happen
+
+#### Actual behavior
+
+Tell us what happens instead
+
+#### Steps to reproduce
+
+How can we reproduce the issue
+
+#### System configuration
+**Pulsar version**: x.y
diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md
new file mode 100644
index 0000000..b9be206
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/bug_report.md
@@ -0,0 +1,30 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: type/bug
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Desktop (please complete the following information):**
+ - OS: [e.g. iOS]
+
+**Additional context**
+Add any other context about the problem here.
diff --git a/.github/ISSUE_TEMPLATE/enhancement_request.md b/.github/ISSUE_TEMPLATE/enhancement_request.md
new file mode 100644
index 0000000..98adf72
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/enhancement_request.md
@@ -0,0 +1,20 @@
+---
+name: Enhancement request
+about: Suggest an enhancement for this project
+title: ''
+labels: type/enhancement
+assignees: ''
+
+---
+
+**Is your enhancement request related to a problem? Please describe.**
+A clear and concise description of what the enhancement is.
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.
diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md
new file mode 100644
index 0000000..5b8022c
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/feature_request.md
@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: type/feature
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
new file mode 100644
index 0000000..0277fbf
--- /dev/null
+++ b/.github/PULL_REQUEST_TEMPLATE.md
@@ -0,0 +1,74 @@
+<!--
+### Contribution Checklist
+
+ - Name the pull request in the form "[Issue XYZ][component] Title of the pull request", where *XYZ* should be replaced by the actual issue number.
+ Skip *Issue XYZ* if there is no associated github issue for this pull request.
+ Skip *component* if you are unsure about which is the best component. E.g. `[docs] Fix typo in produce method`.
+
+ - Fill out the template below to describe the changes contributed by the pull request. That will give reviewers the context they need to do the review.
+
+ - Each pull request should address only one issue, not mix up code from multiple issues.
+
+ - Each commit in the pull request has a meaningful commit message
+
+ - Once all items of the checklist are addressed, remove the above text and this checklist, leaving only the filled out template below.
+
+**(The sections below can be removed for hotfixes of typos)**
+-->
+
+*(If this PR fixes a github issue, please add `Fixes #<xyz>`.)*
+
+Fixes #<xyz>
+
+*(or if this PR is one task of a github issue, please add `Master Issue: #<xyz>` to link to the master issue.)*
+
+Master Issue: #<xyz>
+
+### Motivation
+
+
+*Explain here the context, and why you're making that change. What is the problem you're trying to solve.*
+
+### Modifications
+
+*Describe the modifications you've done.*
+
+### Verifying this change
+
+- [ ] Make sure that the change passes the CI checks.
+
+*(Please pick either of the following options)*
+
+This change is a trivial rework / code cleanup without any test coverage.
+
+*(or)*
+
+This change is already covered by existing tests, such as *(please describe tests)*.
+
+*(or)*
+
+This change added tests and can be verified as follows:
+
+*(example:)*
+ - *Added integration tests for end-to-end deployment with large payloads (10MB)*
+ - *Extended integration test for recovery after broker failure*
+
+### Does this pull request potentially affect one of the following parts:
+
+*If `yes` was chosen, please highlight the changes*
+
+ - Dependencies (does it add or upgrade a dependency): (yes / no)
+ - The public API: (yes / no)
+ - The schema: (yes / no / don't know)
+ - The default values of configurations: (yes / no)
+ - The wire protocol: (yes / no)
+ - The rest endpoints: (yes / no)
+ - The admin cli options: (yes / no)
+ - Anything that affects deployment: (yes / no / don't know)
+
+### Documentation
+
+ - Does this pull request introduce a new feature? (yes / no)
+ - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
+ - If a feature is not applicable for documentation, explain why?
+ - If a feature is not documented yet in this PR, please create a followup issue for adding the documentation
diff --git a/.github/workflows/unit-test.yaml b/.github/workflows/unit-test.yaml
new file mode 100644
index 0000000..cf3b5e6
--- /dev/null
+++ b/.github/workflows/unit-test.yaml
@@ -0,0 +1,103 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: CI - Unit
+on:
+ pull_request:
+ branches:
+ - master
+ push:
+ branches:
+ - branch-*
+
+env:
+ MAVEN_OPTS: -Dmaven.wagon.httpconnectionManager.ttlSeconds=25 -Dmaven.wagon.http.retryHandler.count=3
+
+jobs:
+ unit-test:
+ name:
+ runs-on: ubuntu-latest
+ container:
+ image: apachepulsar/pulsar-build:latest
+ timeout-minutes: 120
+
+ steps:
+ - name: Cancel Previous Runs
+ uses: styfle/cancel-workflow-action@0.6.0
+ with:
+ access_token: ${{ github.token }}
+
+ - name: checkout
+ uses: actions/checkout@v2
+ with:
+ fetch-depth: 25
+ ref: ${{ github.event.pull_request.head.sha }}
+ path: connectors
+
+ - name: Clone Pulsar
+ uses: actions/checkout@v2
+ with:
+ fetch-depth: 1
+ repository: apache/pulsar
+ ref: master
+ path: pulsar
+
+ - name: Cache local Maven repository
+ uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ ${{ runner.os }}-maven-
+
+ - name: Set up JDK 1.8
+ uses: actions/setup-java@v1
+ with:
+ java-version: 1.8
+
+ - name: Set up Maven
+ uses: apache/pulsar-test-infra/setup-maven@master
+ with:
+ maven-version: 3.6.1
+
+ - name: Build pulsar
+ run: |
+ cd pulsar
+ mvn -B -DskipTests -Dcheckstyle.skip install
+
+ - name: Build pulsar connectors
+ run: |
+ cd connectors
+ mvn clean install
+
+ - name: package surefire artifacts
+ if: failure()
+ run: |
+ cd connectors
+ rm -rf artifacts
+ mkdir artifacts
+ find . -type d -name "*surefire*" -exec cp --parents -R {} artifacts/ \;
+ zip -r artifacts.zip artifacts
+
+ - uses: actions/upload-artifact@master
+ name: upload surefire-artifacts
+ if: failure()
+ with:
+ name: surefire-artifacts
+ path: connectors/artifacts.zip
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..297f31d
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,90 @@
+status.html
+.metadata
+test-output
+test-results
+dependency-reduced-pom.xml
+logs
+/data
+pulsar-broker/data/
+pulsar-broker/tmp.*
+pulsar-broker/src/test/resources/log4j2.yaml
+pulsar-functions/worker/test-tenant/
+pulsar-broker/src/test/resources/pulsar-functions-api-examples.jar
+pulsar-functions/runtime/src/test/resources/
+pulsar-functions/worker/src/test/resources/
+*.log
+*.nar
+
+*.versionsBackup
+*/bin
+
+# Eclipse
+.classpath
+.project
+.settings/
+.recommenders/
+.factorypath
+
+# Intellij
+.idea/
+*.iml
+*.iws
+
+# Mac
+**/.DS_Store
+
+# VisualStudioCode artifacts
+.vscode/
+
+# Maven
+log/
+target/
+
+# Python
+*.pyc
+
+# Perf tools
+*.hgrm
+
+# tmp files
+*.pid
+
+# Generated website
+generated-site/
+
+# Ansible and Terraform artifacts
+deployment/terraform-ansible/deploy-pulsar.retry
+deployment/terraform-ansible/aws/.terraform/
+deployment/terraform-ansible/aws/.terraform.tfstate.lock.info
+deployment/terraform-ansible/aws/terraform.tfstate
+deployment/terraform-ansible/aws/terraform.tfstate.backup
+
+# Vagrant
+**/.vagrant
+
+
+pulsar-client-cpp/python/pkg/osx/*/*.bak
+pulsar-client-cpp/python/pkg/osx/*/*.whl
+pulsar-client-cpp/python/wheelhouse
+
+# CI generated files
+.repository
+docker.debug-info
+
+# Yarn
+
+**/yarn.lock
+
+# docusaurus
+
+**/website/i18n/*
+**/website/translated_docs*
+**/website/brodocs/index.html
+**/website/brodocs/navData.js
+**/website/brodocs/documents/*.md
+
+# Avro
+examples/flink/src/main/java/org/apache/flink/avro/generated
+pulsar-flink/src/test/java/org/apache/flink/avro/generated
+pulsar-client/src/test/java/org/apache/pulsar/client/avro/generated
+/build/
diff --git a/pom.xml b/pom.xml
new file mode 100644
index 0000000..9bc36a6
--- /dev/null
+++ b/pom.xml
@@ -0,0 +1,119 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <packaging>pom</packaging>
+ <parent>
+ <groupId>org.apache.pulsar</groupId>
+ <artifactId>pulsar</artifactId>
+ <version>2.7.0-SNAPSHOT</version>
+ </parent>
+
+ <groupId>org.apache.pulsar</groupId>
+ <artifactId>pulsar-connectors</artifactId>
+
+ <version>2.7.0-SNAPSHOT</version>
+
+ <name>Pulsar Adapters</name>
+ <description>A collections of Pulsar IO connectors.</description>
+ <url>https://github.com/apache/pulsar-connectors</url>
+
+ <organization>
+ <name>Apache Software Foundation</name>
+ <url>http://www.apache.org/</url>
+ </organization>
+ <inceptionYear>2017</inceptionYear>
+
+ <developers>
+ <developer>
+ <organization>Apache Pulsar developers</organization>
+ <organizationUrl>http://pulsar.apache.org/</organizationUrl>
+ </developer>
+ </developers>
+
+ <licenses>
+ <license>
+ <name>Apache License, Version 2.0</name>
+ <url>https://www.apache.org/licenses/LICENSE-2.0.txt</url>
+ <distribution>repo</distribution>
+ </license>
+ </licenses>
+
+ <scm>
+ <url>https://github.com/apache/pulsar-connectors</url>
+ <connection>scm:git:https://github.com/apache/pulsar-connectors.git</connection>
+ <developerConnection>scm:git:ssh://git@github.com:apache/pulsar-connectors.git</developerConnection>
+ </scm>
+
+ <ciManagement>
+ <system>Travis</system>
+ <url>https://travis-ci.org/apache/pulsar</url>
+ </ciManagement>
+
+ <issueManagement>
+ <system>Github</system>
+ <url>https://github.com/apache/pulsar-connectors/issues</url>
+ </issueManagement>
+
+ <properties>
+ <flink.version>1.6.0</flink.version>
+ <storm.version>2.0.0</storm.version>
+ </properties>
+
+ <modules>
+ <module>pulsar-io</module>
+ </modules>
+
+ <dependencyManagement>
+ </dependencyManagement>
+
+ <dependencies>
+ </dependencies>
+
+ <build>
+ </build>
+
+ <profiles>
+ <profile>
+ <id>windows</id>
+ <activation>
+ <os>
+ <family>Windows</family>
+ </os>
+ </activation>
+ <properties>
+ <rename.netty.native.libs>rename-netty-native-libs.cmd</rename.netty.native.libs>
+ </properties>
+ </profile>
+ </profiles>
+
+ <repositories>
+ <repository>
+ <id>central</id>
+ <layout>default</layout>
+ <url>https://repo1.maven.org/maven2</url>
+ </repository>
+ </repositories>
+
+</project>
+
diff --git a/pulsar-io/pom.xml b/pulsar-io/pom.xml
index fd566de..f1bccf6 100644
--- a/pulsar-io/pom.xml
+++ b/pulsar-io/pom.xml
@@ -24,7 +24,7 @@
<packaging>pom</packaging>
<parent>
<groupId>org.apache.pulsar</groupId>
- <artifactId>pulsar</artifactId>
+ <artifactId>pulsar-connectors</artifactId>
<version>2.7.0-SNAPSHOT</version>
</parent>
@@ -38,10 +38,6 @@
<activeByDefault>true</activeByDefault>
</activation>
<modules>
- <module>core</module>
- <module>batch-discovery-triggerers</module>
- <module>batch-data-generator</module>
- <module>common</module>
<module>docs</module>
<module>aws</module>
<module>twitter</module>
@@ -52,7 +48,6 @@
<module>kinesis</module>
<module>hdfs3</module>
<module>jdbc</module>
- <module>data-generator</module>
<module>elastic-search</module>
<module>kafka-connect-adaptor</module>
<module>debezium</module>
@@ -69,19 +64,5 @@
<module>dynamodb</module>
</modules>
</profile>
-
- <profile>
- <id>core-modules</id>
- <modules>
- <module>core</module>
- <module>batch-discovery-triggerers</module>
- <module>common</module>
- <module>twitter</module>
- <module>cassandra</module>
- <module>data-generator</module>
- </modules>
- </profile>
-
</profiles>
-
</project>