You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/06/15 04:05:53 UTC
[GitHub] [iceberg] rajarshisarkar opened a new pull request, #5044: Spark 3.x: Use original option keys in SparkCatalog
rajarshisarkar opened a new pull request, #5044:
URL: https://github.com/apache/iceberg/pull/5044
Backports https://github.com/apache/iceberg/pull/4903
---
cc: @rdblue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rajarshisarkar commented on a diff in pull request #5044: Spark 3.x: Use original option keys in SparkCatalog
Posted by GitBox <gi...@apache.org>.
rajarshisarkar commented on code in PR #5044:
URL: https://github.com/apache/iceberg/pull/5044#discussion_r908219799
##########
spark/v3.1/spark/src/main/java/org/apache/iceberg/spark/SparkCatalog.java:
##########
@@ -118,8 +118,7 @@ public class SparkCatalog extends BaseCatalog {
*/
protected Catalog buildIcebergCatalog(String name, CaseInsensitiveStringMap options) {
Configuration conf = SparkUtil.hadoopConfCatalogOverrides(SparkSession.active(), name);
- Map<String, String> optionsMap = new TreeMap<>(String.CASE_INSENSITIVE_ORDER);
- optionsMap.putAll(options);
+ Map<String, String> optionsMap = new TreeMap<>(options.asCaseSensitiveMap());
Review Comment:
I see, I have made the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue merged pull request #5044: Spark 3.x: Use original option keys in SparkCatalog
Posted by GitBox <gi...@apache.org>.
rdblue merged PR #5044:
URL: https://github.com/apache/iceberg/pull/5044
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #5044: Spark 3.x: Use original option keys in SparkCatalog
Posted by GitBox <gi...@apache.org>.
rdblue commented on PR #5044:
URL: https://github.com/apache/iceberg/pull/5044#issuecomment-1169314449
@rajarshisarkar, can you fix 3.3 as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5044: Spark 3.x: Use original option keys in SparkCatalog
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5044:
URL: https://github.com/apache/iceberg/pull/5044#discussion_r907965437
##########
spark/v3.1/spark/src/main/java/org/apache/iceberg/spark/SparkCatalog.java:
##########
@@ -118,8 +118,7 @@ public class SparkCatalog extends BaseCatalog {
*/
protected Catalog buildIcebergCatalog(String name, CaseInsensitiveStringMap options) {
Configuration conf = SparkUtil.hadoopConfCatalogOverrides(SparkSession.active(), name);
- Map<String, String> optionsMap = new TreeMap<>(String.CASE_INSENSITIVE_ORDER);
- optionsMap.putAll(options);
+ Map<String, String> optionsMap = new TreeMap<>(options.asCaseSensitiveMap());
Review Comment:
@rajarshisarkar, don't we still need to create the map with `CASE_INSENSITIVE_ORDER`? We want to use the original keys, but it should not force case sensitivity. So this should be:
```java
Map<String, String> optionsMap = new TreeMap<>(String.CASE_INSENSITIVE_ORDER);
optionsMap.putAll(options.asCaseSensitiveMap());
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #5044: Spark 3.x: Use original option keys in SparkCatalog
Posted by GitBox <gi...@apache.org>.
rdblue commented on PR #5044:
URL: https://github.com/apache/iceberg/pull/5044#issuecomment-1170207545
Thanks, @rajarshisarkar!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rajarshisarkar commented on pull request #5044: Spark 3.x: Use original option keys in SparkCatalog
Posted by GitBox <gi...@apache.org>.
rajarshisarkar commented on PR #5044:
URL: https://github.com/apache/iceberg/pull/5044#issuecomment-1169528095
@rdblue, I have made changes for Spark 3.3
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org