You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/03/25 06:43:49 UTC
[GitHub] [pulsar] lynnmatrix opened a new pull request #6601: Issure 793 Add
support for influxdb2 in pulsar-influxdb-sink(#793)
lynnmatrix opened a new pull request #6601: Issure 793 Add support for influxdb2 in pulsar-influxdb-sink(#793)
URL: https://github.com/apache/pulsar/pull/6601
Fixes #793
### Motivation
The latest InfluxDB release(v2) has changed their write path, 'org'、'bucket'、'token' are required properties to communicate with it; New InfluxDB client library is provided with different package name, from org.influxdb to com.influxdb
This PR would add support to the latest InfluxDB 2.0 write API.
### Modifications
Since the influxdb2 client library changes a lot, even the package name changed, so the new client library is added in this pull request, while the old one is kept, which means the implement of client v1 is kept unchanged, so no migration is needed for influxdbv1 users。
The sink for influxdbv1 fails to decode messages encoded in json, it supposes only avro. Support of json schema is added in this pull request.
The sink for influxdbv1 requires influxdb fields as schema fields, so users should define schemas for each kind of device, that is not convenient. The new added sink for influxdbv2 redefines the message schema, and wraps influxdb 'fields' in one map field, just like 'tags' which is already defined as map in the previous sink for influxdbv1.
### Verifying this change
- Unit tests are added for this sink for influxdbv2.
### Does this pull request potentially affect one of the following parts:
- **Dependencies (does it add or upgrade a dependency): yes**
- The public API: no
- The schema: no
- The default values of configurations: no
- The wire protocol: no
- The rest endpoints: no
- The admin cli options: no
- Anything that affects deployment: no
### Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? docs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] lynnmatrix commented on issue #6601: ISSUE-6514 Add
support for influxdb2 in pulsar-influxdb-sink(#6514)
Posted by GitBox <gi...@apache.org>.
lynnmatrix commented on issue #6601: ISSUE-6514 Add support for influxdb2 in pulsar-influxdb-sink(#6514)
URL: https://github.com/apache/pulsar/pull/6601#issuecomment-604274014
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] lynnmatrix opened a new pull request #6601: ISSUE-6514 Add
support for influxdb2 in pulsar-influxdb-sink(#6514)
Posted by GitBox <gi...@apache.org>.
lynnmatrix opened a new pull request #6601: ISSUE-6514 Add support for influxdb2 in pulsar-influxdb-sink(#6514)
URL: https://github.com/apache/pulsar/pull/6601
Fixes #6514
### Motivation
The latest InfluxDB release(v2) has changed their write path, 'org'、'bucket'、'token' are required properties to communicate with it; New InfluxDB client library is provided with different package name, from org.influxdb to com.influxdb
This PR would add support to the latest InfluxDB 2.0 write API.
### Modifications
Since the influxdb2 client library changes a lot, even the package name changed, so the new client library is added in this pull request, while the old one is kept, which means the implement of client v1 is kept unchanged, so no migration is needed for influxdbv1 users。
The sink for influxdbv1 fails to decode messages encoded in json, it supposes only avro. Support of json schema is added in this pull request.
The sink for influxdbv1 requires influxdb fields as schema fields, so users should define schemas for each kind of device, that is not convenient. The new added sink for influxdbv2 redefines the message schema, and wraps influxdb 'fields' in one map field, just like 'tags' which is already defined as map in the previous sink for influxdbv1.
### Verifying this change
- Unit tests are added for this sink for influxdbv2.
### Does this pull request potentially affect one of the following parts:
- **Dependencies (does it add or upgrade a dependency): yes**
- The public API: no
- The schema: no
- The default values of configurations: no
- The wire protocol: no
- The rest endpoints: no
- The admin cli options: no
- Anything that affects deployment: no
### Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? docs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] sijie commented on issue #6601: ISSUE-6514 Add support for
influxdb2 in pulsar-influxdb-sink(#6514)
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #6601: ISSUE-6514 Add support for influxdb2 in pulsar-influxdb-sink(#6514)
URL: https://github.com/apache/pulsar/pull/6601#issuecomment-604225987
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] lynnmatrix commented on issue #6601: ISSUE-6514 Add
support for influxdb2 in pulsar-influxdb-sink(#6514)
Posted by GitBox <gi...@apache.org>.
lynnmatrix commented on issue #6601: ISSUE-6514 Add support for influxdb2 in pulsar-influxdb-sink(#6514)
URL: https://github.com/apache/pulsar/pull/6601#issuecomment-604279457
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] lynnmatrix opened a new pull request #6601: ISSUE-6514 Add
support for influxdb2 in pulsar-influxdb-sink(#6514)
Posted by GitBox <gi...@apache.org>.
lynnmatrix opened a new pull request #6601: ISSUE-6514 Add support for influxdb2 in pulsar-influxdb-sink(#6514)
URL: https://github.com/apache/pulsar/pull/6601
Fixes #6514
### Motivation
The latest InfluxDB release(v2) has changed their write path, 'org'、'bucket'、'token' are required properties to communicate with it; New InfluxDB client library is provided with different package name, from org.influxdb to com.influxdb
This PR would add support to the latest InfluxDB 2.0 write API.
### Modifications
Since the influxdb2 client library changes a lot, even the package name changed, so the new client library is added in this pull request, while the old one is kept, which means the implement of client v1 is kept unchanged, so no migration is needed for influxdbv1 users。
The sink for influxdbv1 fails to decode messages encoded in json, it supposes only avro. Support of json schema is added in this pull request.
The sink for influxdbv1 requires influxdb fields as schema fields, so users should define schemas for each kind of device, that is not convenient. The new added sink for influxdbv2 redefines the message schema, and wraps influxdb 'fields' in one map field, just like 'tags' which is already defined as map in the previous sink for influxdbv1.
### Verifying this change
- Unit tests are added for this sink for influxdbv2.
### Does this pull request potentially affect one of the following parts:
- **Dependencies (does it add or upgrade a dependency): yes**
- The public API: no
- The schema: no
- The default values of configurations: no
- The wire protocol: no
- The rest endpoints: no
- The admin cli options: no
- Anything that affects deployment: no
### Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? docs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] lynnmatrix commented on issue #6601: ISSUE-6514 Add
support for influxdb2 in pulsar-influxdb-sink(#6514)
Posted by GitBox <gi...@apache.org>.
lynnmatrix commented on issue #6601: ISSUE-6514 Add support for influxdb2 in pulsar-influxdb-sink(#6514)
URL: https://github.com/apache/pulsar/pull/6601#issuecomment-604231736
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] lynnmatrix opened a new pull request #6601: ISSUE-6514 Add
support for influxdb2 in pulsar-influxdb-sink(#6514)
Posted by GitBox <gi...@apache.org>.
lynnmatrix opened a new pull request #6601: ISSUE-6514 Add support for influxdb2 in pulsar-influxdb-sink(#6514)
URL: https://github.com/apache/pulsar/pull/6601
Fixes #6514
### Motivation
The latest InfluxDB release(v2) has changed their write path, 'org'、'bucket'、'token' are required properties to communicate with it; New InfluxDB client library is provided with different package name, from org.influxdb to com.influxdb
This PR would add support to the latest InfluxDB 2.0 write API.
### Modifications
Since the influxdb2 client library changes a lot, even the package name changed, so the new client library is added in this pull request, while the old one is kept, which means the implement of client v1 is kept unchanged, so no migration is needed for influxdbv1 users。
The sink for influxdbv1 fails to decode messages encoded in json, it supposes only avro. Support of json schema is added in this pull request.
The sink for influxdbv1 requires influxdb fields as schema fields, so users should define schemas for each kind of device, that is not convenient. The new added sink for influxdbv2 redefines the message schema, and wraps influxdb 'fields' in one map field, just like 'tags' which is already defined as map in the previous sink for influxdbv1.
### Verifying this change
- Unit tests are added for this sink for influxdbv2.
### Does this pull request potentially affect one of the following parts:
- **Dependencies (does it add or upgrade a dependency): yes**
- The public API: no
- The schema: no
- The default values of configurations: no
- The wire protocol: no
- The rest endpoints: no
- The admin cli options: no
- Anything that affects deployment: no
### Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? docs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] lynnmatrix closed pull request #6601: ISSUE-6514 Add
support for influxdb2 in pulsar-influxdb-sink(#6514)
Posted by GitBox <gi...@apache.org>.
lynnmatrix closed pull request #6601: ISSUE-6514 Add support for influxdb2 in pulsar-influxdb-sink(#6514)
URL: https://github.com/apache/pulsar/pull/6601
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] lynnmatrix removed a comment on issue #6601: ISSUE-6514
Add support for influxdb2 in pulsar-influxdb-sink(#6514)
Posted by GitBox <gi...@apache.org>.
lynnmatrix removed a comment on issue #6601: ISSUE-6514 Add support for influxdb2 in pulsar-influxdb-sink(#6514)
URL: https://github.com/apache/pulsar/pull/6601#issuecomment-604231736
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] lynnmatrix closed pull request #6601: ISSUE-6514 Add
support for influxdb2 in pulsar-influxdb-sink(#6514)
Posted by GitBox <gi...@apache.org>.
lynnmatrix closed pull request #6601: ISSUE-6514 Add support for influxdb2 in pulsar-influxdb-sink(#6514)
URL: https://github.com/apache/pulsar/pull/6601
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] lynnmatrix closed pull request #6601: ISSUE-6514 Add
support for influxdb2 in pulsar-influxdb-sink(#6514)
Posted by GitBox <gi...@apache.org>.
lynnmatrix closed pull request #6601: ISSUE-6514 Add support for influxdb2 in pulsar-influxdb-sink(#6514)
URL: https://github.com/apache/pulsar/pull/6601
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] lynnmatrix commented on issue #6601: ISSUE-6514 Add
support for influxdb2 in pulsar-influxdb-sink(#6514)
Posted by GitBox <gi...@apache.org>.
lynnmatrix commented on issue #6601: ISSUE-6514 Add support for influxdb2 in pulsar-influxdb-sink(#6514)
URL: https://github.com/apache/pulsar/pull/6601#issuecomment-604476198
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] sijie merged pull request #6601: ISSUE-6514 Add support
for influxdb2 in pulsar-influxdb-sink(#6514)
Posted by GitBox <gi...@apache.org>.
sijie merged pull request #6601: ISSUE-6514 Add support for influxdb2 in pulsar-influxdb-sink(#6514)
URL: https://github.com/apache/pulsar/pull/6601
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services