You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/05/23 06:19:03 UTC
[GitHub] [iceberg] ElapsedSoul opened a new issue, #4837: where to get started for iceberg?
ElapsedSoul opened a new issue, #4837:
URL: https://github.com/apache/iceberg/issues/4837
where to get started for iceberg?
for this page:[jara api quickstart for create a table](https://iceberg.apache.org/docs/latest/java-api-quickstart/#create-a-table)
```java
import org.apache.iceberg.hive.HiveCatalog;
Catalog catalog = new HiveCatalog();
catalog.setConf(spark.sparkContext().hadoopConfiguration()); // Configure using Spark's Hadoop configuration
Map <String, String> properties = new HashMap<String, String>();
properties.put("warehouse", "...");
properties.put("uri", "...");
catalog.initialize("hive", properties);
```
this demo do not import the `Catalog`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ElapsedSoul commented on issue #4837: where to get started for iceberg?
Posted by GitBox <gi...@apache.org>.
ElapsedSoul commented on issue #4837:
URL: https://github.com/apache/iceberg/issues/4837#issuecomment-1135296045
> @ElapsedSoul you might want to also get started with https://tabular.io/blog/java-api-part-1/ and https://tabular.io/blog/java-api-part-2/
Thanks for your website, to let me know more clear about the metadata in iceberg.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ElapsedSoul commented on issue #4837: where to get started for iceberg?
Posted by GitBox <gi...@apache.org>.
ElapsedSoul commented on issue #4837:
URL: https://github.com/apache/iceberg/issues/4837#issuecomment-1135294638
Thanks for @ajantha-bhat ,
I put the demo code here:
```java
import java.util.HashMap;
import java.util.Map;
import org.apache.iceberg.Table;
import org.apache.iceberg.catalog.TableIdentifier;
import org.apache.iceberg.hive.HiveCatalog;
import org.apache.iceberg.spark.actions.SparkActions;
import org.apache.spark.sql.SparkSession;
public class IcebergMergeFile {
public static void main(String[] args) throws Exception {
SparkSession spark = SparkSession
.builder()
.appName("IcebergMergeFile")
.getOrCreate();
HiveCatalog catalog = new HiveCatalog();
catalog.setConf(spark.sparkContext().hadoopConfiguration());
Map <String, String> properties = new HashMap<String, String>();
properties.put("uri", "thrift://127.0.0.1:9083");
catalog.initialize("hive", properties);
TableIdentifier name = TableIdentifier.of("ice_database", "ice_t1");
// or to load an existing table, use the following line
Table table = catalog.loadTable(name);
SparkActions
.get()
.rewriteDataFiles(table)
.option("target-file-size-bytes", Long.toString(500 * 1024 * 1024)) // 500 MB
.execute();
spark.stop();
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ElapsedSoul commented on issue #4837: where to get started for iceberg?
Posted by GitBox <gi...@apache.org>.
ElapsedSoul commented on issue #4837:
URL: https://github.com/apache/iceberg/issues/4837#issuecomment-1134262128
>
> yes, this example can be improved.
>
> You can replace `Catalog` with `HiveCatalog` and go ahead. Feel free to open a PR on the example improvement.
Thank you for your help.
Then how can I get the Table table instance for a table I want?
The final goal for me, it is to rewritedatafiles of the table.
Now I found out there no eg. for me to work out .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] nastra commented on issue #4837: where to get started for iceberg?
Posted by GitBox <gi...@apache.org>.
nastra commented on issue #4837:
URL: https://github.com/apache/iceberg/issues/4837#issuecomment-1134440204
@ElapsedSoul you might want to also get started with https://tabular.io/blog/java-api-part-1/ and https://tabular.io/blog/java-api-part-2/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ajantha-bhat commented on issue #4837: where to get started for iceberg?
Posted by GitBox <gi...@apache.org>.
ajantha-bhat commented on issue #4837:
URL: https://github.com/apache/iceberg/issues/4837#issuecomment-1134238993
yes, this example can be improved.
You can replace `Catalog` with `HiveCatalog` and go ahead. Feel free to open a PR on the example improvement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ajantha-bhat commented on issue #4837: where to get started for iceberg?
Posted by GitBox <gi...@apache.org>.
ajantha-bhat commented on issue #4837:
URL: https://github.com/apache/iceberg/issues/4837#issuecomment-1134465433
Once you have a catalog, you can load the table using identifier
import org.apache.iceberg.Table;
import org.apache.iceberg.catalog.TableIdentifier;
TableIdentifier name = TableIdentifier.of("db1", "t1");
// or to load an existing table, use the following line
Table table = catalog.loadTable(name);
once you have the table object, call rewrite data files actions like this.
https://iceberg.apache.org/docs/latest/maintenance/#compact-data-files
You can also check out the code and look for package names and info.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ElapsedSoul closed issue #4837: where to get started for iceberg?
Posted by GitBox <gi...@apache.org>.
ElapsedSoul closed issue #4837: where to get started for iceberg?
URL: https://github.com/apache/iceberg/issues/4837
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org