You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "trunckman (via GitHub)" <gi...@apache.org> on 2024/03/28 08:49:31 UTC
[I] [Bug] (spark load)when use spark load to load data into table, the return of select will be empty usually [doris]
trunckman opened a new issue, #32971:
URL: https://github.com/apache/doris/issues/32971
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
### Version
1.2.7.1-rc01
### What's Wrong?
When i use spark load to load data into my table, and select from the table, it return 0 lines usually;
And I find the __DORIS_DELETE_SIGN__ column is not equal 0;
mysql> select city_name from test_for_spark_load limit 2;
Empty set (0.02 sec)
mysql> select city_name from test_for_spark_load limit 2;
+-----------+
| city_name |
+-----------+
| 上海市 |
| 东莞市 |
+-----------+
2 rows in set (0.02 sec)
mysql> SET show_hidden_columns = true;
Query OK, 0 rows affected (0.02 sec)
mysql> select city_name,__DORIS_DELETE_SIGN__ from test_for_spark_load limit 2;
+-----------+-----------------------+
| city_name | __DORIS_DELETE_SIGN__ |
+-----------+-----------------------+
| 上海市 | 0 |
| 东莞市 | 0 |
+-----------+-----------------------+
2 rows in set (0.02 sec)
mysql> select city_name,__DORIS_DELETE_SIGN__ from test_for_spark_load limit 2;
+-----------+-----------------------+
| city_name | __DORIS_DELETE_SIGN__ |
+-----------+-----------------------+
| 上海市 | 114 |
| 东莞市 | 114 |
+-----------+-----------------------+
2 rows in set (0.02 sec)
### What You Expected?
mysql> select city_name,__DORIS_DELETE_SIGN__ from test_for_spark_load limit 2;
+-----------+-----------------------+
| city_name | __DORIS_DELETE_SIGN__ |
+-----------+-----------------------+
| 上海市 | 0 |
| 东莞市 | 0 |
+-----------+-----------------------+
### How to Reproduce?
1.CREATE TABLE `test_for_spark_load`
( `pdate` date NULL COMMENT '快照日期',
`city_name` varchar(255) NULL COMMENT '城市')
ENGINE=OLAP UNIQUE KEY(`pdate`, `city_name`) COMMENT 'OLAP'
DISTRIBUTED BY HASH(`pdate`)
BUCKETS 1
PROPERTIES ( "replication_allocation" = "tag.location.default: 3", "in_memory" = "false", "storage_format" = "V2", "disable_auto_compaction" = "false" );
2.
LOAD LABEL test_for_spark_load01
(
DATA INFILE('hdfs://xx/*')
INTO TABLE test_for_spark_load
FORMAT AS 'parquet'
(city_name)
COLUMNS FROM PATH AS (pdate)
SET
(pdate=pdate)
)
WITH RESOURCE 'spark'
(
"spark.executor.memory" = "4g",
"spark.shuffle.compress" = "true"
)
3.mysql> select city_name from test_for_spark_load limit 2;
Empty set (0.02 sec)
mysql> SET show_hidden_columns = true;
Query OK, 0 rows affected (0.02 sec)
mysql> select city_name,__DORIS_DELETE_SIGN__ from test_for_spark_load limit 2;
+-----------+-----------------------+
| city_name | __DORIS_DELETE_SIGN__ |
+-----------+-----------------------+
| 上海市 | 114 |
| 东莞市 | 114 |
+-----------+-----------------------+
### Anything Else?
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] [Bug] (spark load)when use spark load to load data into table, the return of select will be empty usually [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman closed issue #32971: [Bug] (spark load)when use spark load to load data into table, the return of select will be empty usually
URL: https://github.com/apache/doris/issues/32971
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] [Bug] (spark load)when use spark load to load data into table, the return of select will be empty usually [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #32971:
URL: https://github.com/apache/doris/issues/32971#issuecomment-3288991459
We're closing this issue because it hasn't been updated in a while.
If you'd like to revive this issue, please reopen it and feel free a maintainer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org