You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@inlong.apache.org by GitBox <gi...@apache.org> on 2022/05/19 09:11:39 UTC
[GitHub] [incubator-inlong] thexiay opened a new issue, #4264: [Bug][Sort] Sort lightweight start 'pulsar-to-kafka' task failed
thexiay opened a new issue, #4264:
URL: https://github.com/apache/incubator-inlong/issues/4264
### What happened
![Uploading image.png…]()
### What you expected to happen
task susscessfully sumbmit
### How to reproduce
1. create a task json file
```
{
"groupId":"1",
"streams":[
{
"streamId":"1",
"nodes":[
{
"type":"pulsarExtract",
"id":"1",
"name":"pulsar_input",
"fields":[
{
"type": "base",
"name":"id",
"formatInfo":{
"type":"long"
}
},
{
"type": "base",
"name":"name",
"formatInfo":{
"type":"string"
}
}
],
"topic": "persistent://public/default/test_stream",
"adminUrl": "http://localhost:8080",
"serviceUrl": "pulsar://localhost:6650",
"format": {
"type": "inLongMsgFormat",
"innerFormat": {
"type":"jsonFormat",
"failOnMissingField":false,
"ignoreParseErrors":true,
"timestampFormatStandard":"SQL",
"mapNullKeyMode":"DROP",
"mapNullKeyLiteral":"null",
"encodeDecimalAsPlainNumber":true
},
"ignoreParseErrors": true
},
"scanStartupMode": "earliest"
},
{
"type":"kafkaLoad",
"id":"2",
"name":"kafka_output",
"fields":[
{
"type":"base",
"name":"id",
"formatInfo":{
"type":"long"
}
},
{
"type":"base",
"name":"name",
"formatInfo":{
"type":"string"
}
}
],
"fieldRelationShips":[
{
"type":"fieldRelationShip",
"inputField":{
"type":"base",
"name":"id",
"formatInfo":{
"type":"long"
}
},
"outputField":{
"type":"base",
"name":"id",
"formatInfo":{
"type":"long"
}
}
},
{
"type":"fieldRelationShip",
"inputField":{
"type":"base",
"name":"name",
"formatInfo":{
"type":"string"
}
},
"outputField":{
"type":"base",
"name":"name",
"formatInfo":{
"type":"string"
}
}
}
],
"topic":"YOUR_TOPIC",
"bootstrapServers":"YOUR_KAFKA_BOOTSTRAP_SERVERS",
"format":{
"type":"jsonFormat",
"failOnMissingField":false,
"ignoreParseErrors":true,
"timestampFormatStandard":"SQL",
"mapNullKeyMode":"DROP",
"mapNullKeyLiteral":"null",
"encodeDecimalAsPlainNumber":true
},
"primaryKey":"id"
}
],
"relations":[
{
"type":"baseRelation",
"inputs":[
"1"
],
"outputs":[
"2"
]
}
]
}
]
}
```
2. move newest sort-single-tenant jar to flink lib path
3. shell to start task
`flink run -c org.apache.inlong.sort.singletenant.flink.Entrance $FLINK_HOME/lib/sort-single-tenant-1.2.0-incubating-SNAPSHOT.jar \
--lightweight true --group.info.file pulsar-to-kafka.json`
### Environment
_No response_
### InLong version
master
### InLong Component
InLong Sort
### Are you willing to submit PR?
- [X] Yes, I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@inlong.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-inlong] healchow closed issue #4264: [Bug][Sort] Sort lightweight start 'pulsar-to-kafka' task failed
Posted by GitBox <gi...@apache.org>.
healchow closed issue #4264: [Bug][Sort] Sort lightweight start 'pulsar-to-kafka' task failed
URL: https://github.com/apache/incubator-inlong/issues/4264
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org