You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/10/08 16:33:50 UTC
[GitHub] [iceberg] samredai opened a new pull request #3253: Docs: Changing java api quickstart to use recommended no-arg constructor
samredai opened a new pull request #3253:
URL: https://github.com/apache/iceberg/pull/3253
This is tied to [#3235](https://github.com/apache/iceberg/issues/3235) and updates the [Java Quickstart](https://iceberg.apache.org/java-api-quickstart/) to use the recommended constructor (instead of the deprecated one). The logic in the quickstart is also dependent on a NPE [fix](https://github.com/apache/iceberg/commit/bcedb21c29e1b821443f0ee703505d589b187ace) recently merged in.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #3253: Docs: Changing java api quickstart to use recommended no-arg constructor
Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #3253:
URL: https://github.com/apache/iceberg/pull/3253#issuecomment-940139665
Thanks, @samredai!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on a change in pull request #3253: Docs: Changing java api quickstart to use recommended no-arg constructor
Posted by GitBox <gi...@apache.org>.
samredai commented on a change in pull request #3253:
URL: https://github.com/apache/iceberg/pull/3253#discussion_r725168191
##########
File path: site/docs/java-api-quickstart.md
##########
@@ -23,12 +23,26 @@ Tables are created using either a [`Catalog`](./javadoc/master/index.html?org/ap
### Using a Hive catalog
-The Hive catalog connects to a Hive MetaStore to keep track of Iceberg tables. This example uses Spark's Hadoop configuration to get a Hive catalog:
+The Hive catalog connects to a Hive MetaStore to keep track of Iceberg tables.
+You can initialize a Hive Catalog with a name and some properties.
+(see: [Catalog properties](https://iceberg.apache.org/configuration/#catalog-properties))
```java
import org.apache.iceberg.hive.HiveCatalog;
-Catalog catalog = new HiveCatalog(spark.sparkContext().hadoopConfiguration());
+Catalog catalog = new HiveCatalog();
+
+Map <String, String> properties = new HashMap<String, String>();
+properties.put("warehouse", "...");
+properties.put("uri", "...");
+
+catalog.initialize("hive", properties);
Review comment:
Yes it currently fails but I think it's isolated to just if `toString()` is called which is hard to get around when using a REPL. The failure actually happens earlier than `initialize()` and seems to happen right at construction. (I'm going to try and confirm that last part)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a change in pull request #3253: Docs: Changing java api quickstart to use recommended no-arg constructor
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #3253:
URL: https://github.com/apache/iceberg/pull/3253#discussion_r725676732
##########
File path: site/docs/java-api-quickstart.md
##########
@@ -23,12 +23,21 @@ Tables are created using either a [`Catalog`](./javadoc/master/index.html?org/ap
### Using a Hive catalog
-The Hive catalog connects to a Hive MetaStore to keep track of Iceberg tables. This example uses Spark's Hadoop configuration to get a Hive catalog:
+The Hive catalog connects to a Hive MetaStore to keep track of Iceberg tables.
+You can initialize a Hive Catalog with a name and some properties.
+(see: [Catalog properties](https://iceberg.apache.org/configuration/#catalog-properties))
Review comment:
Should we call out that initialize will currently fail unless `setConf` is called first?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on a change in pull request #3253: Docs: Changing java api quickstart to use recommended no-arg constructor
Posted by GitBox <gi...@apache.org>.
samredai commented on a change in pull request #3253:
URL: https://github.com/apache/iceberg/pull/3253#discussion_r725427758
##########
File path: site/docs/java-api-quickstart.md
##########
@@ -23,12 +23,26 @@ Tables are created using either a [`Catalog`](./javadoc/master/index.html?org/ap
### Using a Hive catalog
-The Hive catalog connects to a Hive MetaStore to keep track of Iceberg tables. This example uses Spark's Hadoop configuration to get a Hive catalog:
+The Hive catalog connects to a Hive MetaStore to keep track of Iceberg tables.
+You can initialize a Hive Catalog with a name and some properties.
+(see: [Catalog properties](https://iceberg.apache.org/configuration/#catalog-properties))
```java
import org.apache.iceberg.hive.HiveCatalog;
-Catalog catalog = new HiveCatalog(spark.sparkContext().hadoopConfiguration());
+Catalog catalog = new HiveCatalog();
+
+Map <String, String> properties = new HashMap<String, String>();
+properties.put("warehouse", "...");
+properties.put("uri", "...");
+
+catalog.initialize("hive", properties);
Review comment:
Confirmed that initialize is guaranteed to fail without a conf, updating this soon...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue merged pull request #3253: Docs: Changing java api quickstart to use recommended no-arg constructor
Posted by GitBox <gi...@apache.org>.
rdblue merged pull request #3253:
URL: https://github.com/apache/iceberg/pull/3253
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on a change in pull request #3253: Docs: Changing java api quickstart to use recommended no-arg constructor
Posted by GitBox <gi...@apache.org>.
samredai commented on a change in pull request #3253:
URL: https://github.com/apache/iceberg/pull/3253#discussion_r725168996
##########
File path: site/docs/java-api-quickstart.md
##########
@@ -23,12 +23,26 @@ Tables are created using either a [`Catalog`](./javadoc/master/index.html?org/ap
### Using a Hive catalog
-The Hive catalog connects to a Hive MetaStore to keep track of Iceberg tables. This example uses Spark's Hadoop configuration to get a Hive catalog:
+The Hive catalog connects to a Hive MetaStore to keep track of Iceberg tables.
+You can initialize a Hive Catalog with a name and some properties.
+(see: [Catalog properties](https://iceberg.apache.org/configuration/#catalog-properties))
```java
import org.apache.iceberg.hive.HiveCatalog;
-Catalog catalog = new HiveCatalog(spark.sparkContext().hadoopConfiguration());
+Catalog catalog = new HiveCatalog();
+
+Map <String, String> properties = new HashMap<String, String>();
+properties.put("warehouse", "...");
+properties.put("uri", "...");
+
+catalog.initialize("hive", properties);
+```
+
+Alternatively, you can configure the Hive Catalog using Spark's Hadoop configuration.
Review comment:
Ah I see! I missed that part but it makes total sense, I'll update this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on a change in pull request #3253: Docs: Changing java api quickstart to use recommended no-arg constructor
Posted by GitBox <gi...@apache.org>.
samredai commented on a change in pull request #3253:
URL: https://github.com/apache/iceberg/pull/3253#discussion_r725713428
##########
File path: site/docs/java-api-quickstart.md
##########
@@ -23,12 +23,21 @@ Tables are created using either a [`Catalog`](./javadoc/master/index.html?org/ap
### Using a Hive catalog
-The Hive catalog connects to a Hive MetaStore to keep track of Iceberg tables. This example uses Spark's Hadoop configuration to get a Hive catalog:
+The Hive catalog connects to a Hive MetaStore to keep track of Iceberg tables.
+You can initialize a Hive Catalog with a name and some properties.
+(see: [Catalog properties](https://iceberg.apache.org/configuration/#catalog-properties))
Review comment:
Done! Added a note right before the code snippet:
```markdown
**note:** Currently, `setConf` is always required for hive catalogs, but this will change in the future.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a change in pull request #3253: Docs: Changing java api quickstart to use recommended no-arg constructor
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #3253:
URL: https://github.com/apache/iceberg/pull/3253#discussion_r725158832
##########
File path: site/docs/java-api-quickstart.md
##########
@@ -23,12 +23,26 @@ Tables are created using either a [`Catalog`](./javadoc/master/index.html?org/ap
### Using a Hive catalog
-The Hive catalog connects to a Hive MetaStore to keep track of Iceberg tables. This example uses Spark's Hadoop configuration to get a Hive catalog:
+The Hive catalog connects to a Hive MetaStore to keep track of Iceberg tables.
+You can initialize a Hive Catalog with a name and some properties.
+(see: [Catalog properties](https://iceberg.apache.org/configuration/#catalog-properties))
```java
import org.apache.iceberg.hive.HiveCatalog;
-Catalog catalog = new HiveCatalog(spark.sparkContext().hadoopConfiguration());
+Catalog catalog = new HiveCatalog();
+
+Map <String, String> properties = new HashMap<String, String>();
+properties.put("warehouse", "...");
+properties.put("uri", "...");
+
+catalog.initialize("hive", properties);
Review comment:
We know that this currently fails in 0.12.0 because it doesn't pass Configuration right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a change in pull request #3253: Docs: Changing java api quickstart to use recommended no-arg constructor
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #3253:
URL: https://github.com/apache/iceberg/pull/3253#discussion_r725159813
##########
File path: site/docs/java-api-quickstart.md
##########
@@ -23,12 +23,26 @@ Tables are created using either a [`Catalog`](./javadoc/master/index.html?org/ap
### Using a Hive catalog
-The Hive catalog connects to a Hive MetaStore to keep track of Iceberg tables. This example uses Spark's Hadoop configuration to get a Hive catalog:
+The Hive catalog connects to a Hive MetaStore to keep track of Iceberg tables.
+You can initialize a Hive Catalog with a name and some properties.
+(see: [Catalog properties](https://iceberg.apache.org/configuration/#catalog-properties))
```java
import org.apache.iceberg.hive.HiveCatalog;
-Catalog catalog = new HiveCatalog(spark.sparkContext().hadoopConfiguration());
+Catalog catalog = new HiveCatalog();
+
+Map <String, String> properties = new HashMap<String, String>();
+properties.put("warehouse", "...");
+properties.put("uri", "...");
+
+catalog.initialize("hive", properties);
+```
+
+Alternatively, you can configure the Hive Catalog using Spark's Hadoop configuration.
Review comment:
I wouldn't say that adding the conf is an alternative because that implies that you don't need to pass the catalog properties. Catalog properties are separate config so you should always pass them to configure the catalog. The Hive connection URI and warehouse location are defaulted for Hive, but that's not a normal thing. Other catalogs pretty much ignore the Configuration except to load a Hadoop FileSystem internally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org