You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2023/01/07 03:25:27 UTC
[GitHub] [incubator-seatunnel] stdnt-xiao opened a new issue, #3893: [Bug] [Maxcompute] Failed to parse some maxcompute type
stdnt-xiao opened a new issue, #3893:
URL: https://github.com/apache/incubator-seatunnel/issues/3893
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
Use maxcompute connector,I got prints like [B@79338251... when parse data of String type.
as follows：
<img width="1338" alt="WechatIMG355" src="https://user-images.githubusercontent.com/20313983/211129096-f51481d7-c90e-4523-94c4-57c86d9afb7b.png">
### SeaTunnel Version
dev & 2.3.1
### SeaTunnel Config
```conf
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
######
###### This config file is a demonstration of streaming processing in seatunnel config
######
######
###### Sample of maxcompute data type
######
# DROP TABLE IF EXISTS fake_source;
#
# CREATE TABLE IF NOT EXISTS fake_source(c1 TINYINT,c2 SMALLINT,c3 INT,c4 BIGINT,c5 FLOAT ,c6 DOUBLE
# ,c7 VARCHAR(10),c8 CHAR(10),c9 STRING,c10 DATE,c11 DATETIME ,c12 TIMESTAMP ,c13 BOOLEAN,c14 BINARY
# ,c15 MAP<STRING,STRING>,c16 ARRAY<INT>,c17 STRUCT<s1:STRING,s2:INT,s3:ARRAY<FLOAT>>);
#
# INSERT INTO fake_source(c1, c2, c3, c4, c5, c6, c7,c8,c9,c10,c11,c12,c13,c14,c15,c16,c17) VALUES (
# CAST(-128 AS TINYINT ),CAST(-32768 AS SMALLINT ) ,0,10000000000000,0.01,0.0000000000000001
# ,CAST("varchar" as VARCHAR(10)),CAST("char" as CHAR(10)),"hello0",CAST("2022-12-31" as DATE )
# ,CAST("2022-12-31 23:59:59" as DATETIME ),CAST("2022-12-31 23:59:59.999" as TIMESTAMP ),FALSE,CAST("bytes" AS BINARY )
# ,MAP("int",1,"str","hello"),ARRAY("11","22"),named_struct("s1","s1","s2",100,"s3",array(1.1, 2.2)));
#
# SELECT * FROM fake_source;
#
# DROP TABLE IF EXISTS fake_sink;
#
# CREATE TABLE IF NOT EXISTS fake_sink LIKE fake_source;
#
# SELECT * FROM fake_sink;
#
env {
# You can set spark configuration here
# see available properties defined by spark: https://spark.apache.org/docs/latest/configuration.html#available-properties
#job.mode = BATCH
job.name = "SeaTunnel"
spark.executor.instances = 1
spark.executor.cores = 1
spark.executor.memory = "1g"
spark.master = local
}
source {
# This is a example source plugin **only for test and demonstrate the feature source plugin**
Maxcompute {
accessId="<your access id>"
accesskey="<your access Key>"
endpoint="<http://service.odps.aliyun.com/api>"
project="<your project>"
table_name="<your table name>"
#partition_spec="<your partition spec>"
#split_row = 10000
}
# If you would like to get more information about how to configure seatunnel and see full list of source plugins,
# please go to https://seatunnel.apache.org/docs/category/source-v2
}
transform {
sql {
source_table_name = "fake"
sql = "select * from fake"
}
# If you would like to get more information about how to configure seatunnel and see full list of transform plugins,
# please go to https://seatunnel.apache.org/docs/category/transform
}
sink {
Maxcompute {
accessId="<your access id>"
accesskey="<your access Key>"
endpoint="<http://service.odps.aliyun.com/api>"
project="<your project>"
table_name="<your table name>"
#partition_spec="<your partition spec>"
#overwrite = false
}
# If you would like to get more information about how to configure seatunnel and see full list of sink plugins,
# please go to https://seatunnel.apache.org/docs/category/sink-v2
}
```
### Running Command
```shell
bin/start-seatunnel-spark-connector-v2.sh -e client -m local[1] --config ./config/v2.batch.config.template
```
### Error Exception
```log
Use maxcompute source,I got print like [B@79338251... when parse data of String type.
```
### Flink or Spark Version
spark default
### Java or Scala Version
java 8
### Screenshots
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #3893: [Bug] [Maxcompute] Failed to parse some maxcompute type
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #3893:
URL: https://github.com/apache/incubator-seatunnel/issues/3893#issuecomment-1430573488
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] closed issue #3893: [Bug] [Maxcompute] Failed to parse some maxcompute type
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #3893: [Bug] [Maxcompute] Failed to parse some maxcompute type
URL: https://github.com/apache/incubator-seatunnel/issues/3893
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org