You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "yaooqinn (via GitHub)" <gi...@apache.org> on 2023/12/26 06:08:48 UTC
[PR] [SPARK-46478][SQL][3.5] Revert SPARK-43049 to use oracle varchar(255) for string [spark]
yaooqinn opened a new pull request, #44493:
URL: https://github.com/apache/spark/pull/44493
### What changes were proposed in this pull request?
Revert SPARK-43049 to use Oracle Varchar (255) for string for performance consideration
### Why are the changes needed?
for performance consideration
### Does this PR introduce _any_ user-facing change?
yes, storing strings in Oracle table, which is defined by spark DDL with string columns. Users will get an error if string values exceed 255
```java
org.apache.spark.SparkRuntimeException: [EXCEED_LIMIT_LENGTH] Exceeds char/varchar type length limitation: 255. SQLSTATE: 54006
[info] at org.apache.spark.sql.errors.QueryExecutionErrors$.exceedMaxLimit(QueryExecutionErrors.scala:2512)
```
### How was this patch tested?
revised unit tests
### Was this patch authored or co-authored using generative AI tooling?
no
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46478][SQL][3.5] Revert SPARK-43049 to use oracle varchar(255) for string [spark]
Posted by "yaooqinn (via GitHub)" <gi...@apache.org>.
yaooqinn commented on PR #44493:
URL: https://github.com/apache/spark/pull/44493#issuecomment-1870175967
Hi @dongjoon-hyun
Except for the linter error, which seems to be a common issue on branch-3.5, all lights are green.
```
ERROR: Error installing bundler:
The last version of bundler (>= 0) to support your Ruby & RubyGems was 2.4.22. Try installing it with `gem install bundler -v 2.4.22`
bundler requires Ruby version >= 3.0.0. The current ruby version is 2.7.0.0.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46478][SQL][3.5] Revert SPARK-43049 to use oracle varchar(255) for string [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #44493:
URL: https://github.com/apache/spark/pull/44493#issuecomment-1874413209
To @yaooqinn , FYI, `branch-3.5` linter error is recovered by backporting SPARK-46425 today.
> Except for the linter error, which seems to be a common issue on branch-3.5, all lights are green.
- https://github.com/apache/spark/actions/runs/7388180677/job/20098502832
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46478][SQL][3.5] Revert SPARK-43049 to use oracle varchar(255) for string [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #44493:
URL: https://github.com/apache/spark/pull/44493#issuecomment-1870640601
Merged to branch-3.5.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46478][SQL][3.5] Revert SPARK-43049 to use oracle varchar(255) for string [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun closed pull request #44493: [SPARK-46478][SQL][3.5] Revert SPARK-43049 to use oracle varchar(255) for string
URL: https://github.com/apache/spark/pull/44493
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46478][SQL][3.5] Revert SPARK-43049 to use oracle varchar(255) for string [spark]
Posted by "yaooqinn (via GitHub)" <gi...@apache.org>.
yaooqinn commented on PR #44493:
URL: https://github.com/apache/spark/pull/44493#issuecomment-1874757965
Thank you for the info @dongjoon-hyun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org