You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "manuzhang (via GitHub)" <gi...@apache.org> on 2024/01/30 08:29:05 UTC
[PR] Docs: Enhance Java quickstart example [iceberg]
manuzhang opened a new pull request, #9585:
URL: https://github.com/apache/iceberg/pull/9585
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Docs: Enhance Java quickstart example [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #9585:
URL: https://github.com/apache/iceberg/pull/9585#discussion_r1477126245
##########
docs/java-api-quickstart.md:
##########
@@ -38,37 +38,42 @@ The Hive catalog connects to a Hive metastore to keep track of Iceberg tables.
You can initialize a Hive catalog with a name and some properties.
(see: [Catalog properties](../configuration/#catalog-properties))
-**Note:** Currently, `setConf` is always required for hive catalogs, but this will change in the future.
+### Using a Hive catalog
Review Comment:
Looks like the changes are incorrect here. This now duplicates the previous section heading and paragraph.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Docs: Enhance Java quickstart example [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #9585:
URL: https://github.com/apache/iceberg/pull/9585#discussion_r1477126293
##########
docs/java-api-quickstart.md:
##########
@@ -38,37 +38,42 @@ The Hive catalog connects to a Hive metastore to keep track of Iceberg tables.
You can initialize a Hive catalog with a name and some properties.
(see: [Catalog properties](../configuration/#catalog-properties))
-**Note:** Currently, `setConf` is always required for hive catalogs, but this will change in the future.
+### Using a Hive catalog
+
+The Hive catalog connects to a Hive metastore to keep track of Iceberg tables.
+You can initialize a Hive catalog with a name and some properties.
+(see: [Catalog properties](../configuration/#catalog-properties))
```java
+import java.util.HashMap
+import java.util.Map
+
import org.apache.iceberg.hive.HiveCatalog;
HiveCatalog catalog = new HiveCatalog();
-catalog.setConf(spark.sparkContext().hadoopConfiguration()); // Configure using Spark's Hadoop configuration
Map <String, String> properties = new HashMap<String, String>();
-properties.put("warehouse", "...");
-properties.put("uri", "...");
+properties.put("warehouse", "..."); // Override hive.metastore.warehouse.dir
+properties.put("uri", "..."); // Override hive.metastore.uris
Review Comment:
I don't agree with adding `Override` since the Hadoop configuration is now optional. These are the recommended and primary way to configure the warehouse and URI for Hive.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Docs: Enhance Java quickstart example [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #9585:
URL: https://github.com/apache/iceberg/pull/9585#discussion_r1477126737
##########
docs/java-api-quickstart.md:
##########
@@ -38,37 +38,42 @@ The Hive catalog connects to a Hive metastore to keep track of Iceberg tables.
You can initialize a Hive catalog with a name and some properties.
(see: [Catalog properties](../configuration/#catalog-properties))
-**Note:** Currently, `setConf` is always required for hive catalogs, but this will change in the future.
+### Using a Hive catalog
+
+The Hive catalog connects to a Hive metastore to keep track of Iceberg tables.
+You can initialize a Hive catalog with a name and some properties.
+(see: [Catalog properties](../configuration/#catalog-properties))
```java
+import java.util.HashMap
+import java.util.Map
+
import org.apache.iceberg.hive.HiveCatalog;
HiveCatalog catalog = new HiveCatalog();
-catalog.setConf(spark.sparkContext().hadoopConfiguration()); // Configure using Spark's Hadoop configuration
Map <String, String> properties = new HashMap<String, String>();
-properties.put("warehouse", "...");
-properties.put("uri", "...");
+properties.put("warehouse", "..."); // Override hive.metastore.warehouse.dir
+properties.put("uri", "..."); // Override hive.metastore.uris
-catalog.initialize("hive", properties);
+catalog.initialize("hive", properties); // Load default Hadoop and Hive configurations, and apply properties
Review Comment:
We should not recommend loading "default" configurations, so I think the `setConf` line should be brought back and this comment removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Docs: Enhance Java quickstart example [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #9585:
URL: https://github.com/apache/iceberg/pull/9585#discussion_r1477126762
##########
docs/java-api-quickstart.md:
##########
@@ -38,37 +38,42 @@ The Hive catalog connects to a Hive metastore to keep track of Iceberg tables.
You can initialize a Hive catalog with a name and some properties.
(see: [Catalog properties](../configuration/#catalog-properties))
-**Note:** Currently, `setConf` is always required for hive catalogs, but this will change in the future.
+### Using a Hive catalog
+
+The Hive catalog connects to a Hive metastore to keep track of Iceberg tables.
+You can initialize a Hive catalog with a name and some properties.
+(see: [Catalog properties](../configuration/#catalog-properties))
```java
+import java.util.HashMap
+import java.util.Map
+
import org.apache.iceberg.hive.HiveCatalog;
HiveCatalog catalog = new HiveCatalog();
-catalog.setConf(spark.sparkContext().hadoopConfiguration()); // Configure using Spark's Hadoop configuration
Map <String, String> properties = new HashMap<String, String>();
-properties.put("warehouse", "...");
-properties.put("uri", "...");
+properties.put("warehouse", "..."); // Override hive.metastore.warehouse.dir
+properties.put("uri", "..."); // Override hive.metastore.uris
-catalog.initialize("hive", properties);
+catalog.initialize("hive", properties); // Load default Hadoop and Hive configurations, and apply properties
```
-The `Catalog` interface defines methods for working with tables, like `createTable`, `loadTable`, `renameTable`, and `dropTable`. `HiveCatalog` implements the `Catalog` interface.
-
+`HiveCatalog` implements the `Catalog` interface, which defines methods for working with tables, like `createTable`, `loadTable`, `renameTable`, and `dropTable`.
To create a table, pass an `Identifier` and a `Schema` along with other initial metadata:
```java
import org.apache.iceberg.Table;
import org.apache.iceberg.catalog.TableIdentifier;
-TableIdentifier name = TableIdentifier.of("logging", "logs");
-Table table = catalog.createTable(name, schema, spec);
+TableIdentifier identifier = TableIdentifier.of("logging", "logs");
+Table table = catalog.createTable(identifier, schema, spec);
Review Comment:
These changes seem unnecessary to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Docs: Enhance Java quickstart example [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on PR #9585:
URL: https://github.com/apache/iceberg/pull/9585#issuecomment-1925849381
Thanks, @manuzhang!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Docs: Enhance Java quickstart example [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #9585:
URL: https://github.com/apache/iceberg/pull/9585#discussion_r1477126396
##########
docs/java-api-quickstart.md:
##########
@@ -38,37 +38,42 @@ The Hive catalog connects to a Hive metastore to keep track of Iceberg tables.
You can initialize a Hive catalog with a name and some properties.
(see: [Catalog properties](../configuration/#catalog-properties))
-**Note:** Currently, `setConf` is always required for hive catalogs, but this will change in the future.
+### Using a Hive catalog
+
+The Hive catalog connects to a Hive metastore to keep track of Iceberg tables.
+You can initialize a Hive catalog with a name and some properties.
+(see: [Catalog properties](../configuration/#catalog-properties))
```java
+import java.util.HashMap
+import java.util.Map
+
import org.apache.iceberg.hive.HiveCatalog;
HiveCatalog catalog = new HiveCatalog();
-catalog.setConf(spark.sparkContext().hadoopConfiguration()); // Configure using Spark's Hadoop configuration
Review Comment:
I think this should be included, but noted as optional. If the caller has a Hadoop conf, they should supply it. And the caller will almost certainly have one if they are using the Hive Metastore.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Docs: Enhance Java quickstart example [iceberg]
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on PR #9585:
URL: https://github.com/apache/iceberg/pull/9585#issuecomment-1925574651
@rdblue Thanks for review. I've updated the PR accordingly.
I have a question on what is a recommended usage and what's not. How is it conveyed? For example, I don't find much info about `HiveCatalog` on the website. Hence, my intuition is that users should have the least setups to get started.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Docs: Enhance Java quickstart example [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue merged PR #9585:
URL: https://github.com/apache/iceberg/pull/9585
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org