You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/03/09 00:45:24 UTC
[GitHub] [iceberg] amogh-jahagirdar opened a new pull request #4294: Core: Use existing SnapshotRefParser when serializing ref information in TableMetadata Parser
amogh-jahagirdar opened a new pull request #4294:
URL: https://github.com/apache/iceberg/pull/4294
This change is a follow on to @rdblue change in https://github.com/apache/iceberg/pull/4260 . Since we have an existing SnapshotRefParser I think we can leverage that and reduce some of the implementation in TableMetadataParser.
Let me know your thoughts! @rdblue @jackye1995
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #4294: Core: Use existing SnapshotRefParser when serializing ref information in TableMetadata Parser
Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #4294:
URL: https://github.com/apache/iceberg/pull/4294#issuecomment-1069261267
Thanks, @amogh-jahagirdar!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a change in pull request #4294: Core: Use existing SnapshotRefParser when serializing ref information in TableMetadata Parser
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #4294:
URL: https://github.com/apache/iceberg/pull/4294#discussion_r824991546
##########
File path: core/src/main/java/org/apache/iceberg/TableMetadataParser.java
##########
@@ -252,22 +248,7 @@ private static void toJson(TableMetadata metadata, JsonGenerator generator) thro
private static void toJson(Map<String, SnapshotRef> refs, JsonGenerator generator) throws IOException {
generator.writeObjectFieldStart(REFS);
for (Map.Entry<String, SnapshotRef> refEntry : refs.entrySet()) {
- generator.writeObjectFieldStart(refEntry.getKey());
-
- SnapshotRef ref = refEntry.getValue();
- generator.writeStringField(TYPE, ref.type().toString().toLowerCase(Locale.ROOT));
- generator.writeNumberField(SNAPSHOT_ID, ref.snapshotId());
- if (ref.maxRefAgeMs() != null) {
- generator.writeNumberField(MAX_REF_AGE_MS, ref.maxRefAgeMs());
- }
- if (ref.maxSnapshotAgeMs() != null) {
- generator.writeNumberField(MAX_SNAPSHOT_AGE_MS, ref.maxSnapshotAgeMs());
- }
- if (ref.minSnapshotsToKeep() != null) {
- generator.writeNumberField(MIN_SNAPSHOTS_TO_KEEP, ref.minSnapshotsToKeep());
- }
-
- generator.writeEndObject();
+ SnapshotRefParser.toJson(refEntry.getKey(), refEntry.getValue(), generator);
Review comment:
I think that the `writeObjectFieldStart` and `writeEndObject` should be here in this function. Moving it to `toJson` in the other class creates an implicit requirement on the state of the `JsonGenerator`. It _must_ be writing an object. We avoid depending on the state of objects passed in where possible, **especially** for methods that are public.
Instead, the state of the generator should always be that it is ready to write any JSON value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue merged pull request #4294: Core: Use existing SnapshotRefParser when serializing ref information in TableMetadata Parser
Posted by GitBox <gi...@apache.org>.
rdblue merged pull request #4294:
URL: https://github.com/apache/iceberg/pull/4294
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] amogh-jahagirdar commented on a change in pull request #4294: Core: Use existing SnapshotRefParser when serializing ref information in TableMetadata Parser
Posted by GitBox <gi...@apache.org>.
amogh-jahagirdar commented on a change in pull request #4294:
URL: https://github.com/apache/iceberg/pull/4294#discussion_r827100289
##########
File path: core/src/main/java/org/apache/iceberg/TableMetadataParser.java
##########
@@ -252,22 +248,7 @@ private static void toJson(TableMetadata metadata, JsonGenerator generator) thro
private static void toJson(Map<String, SnapshotRef> refs, JsonGenerator generator) throws IOException {
generator.writeObjectFieldStart(REFS);
for (Map.Entry<String, SnapshotRef> refEntry : refs.entrySet()) {
- generator.writeObjectFieldStart(refEntry.getKey());
-
- SnapshotRef ref = refEntry.getValue();
- generator.writeStringField(TYPE, ref.type().toString().toLowerCase(Locale.ROOT));
- generator.writeNumberField(SNAPSHOT_ID, ref.snapshotId());
- if (ref.maxRefAgeMs() != null) {
- generator.writeNumberField(MAX_REF_AGE_MS, ref.maxRefAgeMs());
- }
- if (ref.maxSnapshotAgeMs() != null) {
- generator.writeNumberField(MAX_SNAPSHOT_AGE_MS, ref.maxSnapshotAgeMs());
- }
- if (ref.minSnapshotsToKeep() != null) {
- generator.writeNumberField(MIN_SNAPSHOTS_TO_KEEP, ref.minSnapshotsToKeep());
- }
-
- generator.writeEndObject();
+ SnapshotRefParser.toJson(refEntry.getKey(), refEntry.getValue(), generator);
Review comment:
I agree. I am trying to reconcile this with the existing public toJson on SnapshotRefParser which serializes the ref (without the name as a field) object, the simplest way seemed to be add a new method which relies on the state of JsonGenerator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] amogh-jahagirdar commented on a change in pull request #4294: Core: Use existing SnapshotRefParser when serializing ref information in TableMetadata Parser
Posted by GitBox <gi...@apache.org>.
amogh-jahagirdar commented on a change in pull request #4294:
URL: https://github.com/apache/iceberg/pull/4294#discussion_r827118651
##########
File path: core/src/main/java/org/apache/iceberg/TableMetadataParser.java
##########
@@ -252,22 +248,7 @@ private static void toJson(TableMetadata metadata, JsonGenerator generator) thro
private static void toJson(Map<String, SnapshotRef> refs, JsonGenerator generator) throws IOException {
generator.writeObjectFieldStart(REFS);
for (Map.Entry<String, SnapshotRef> refEntry : refs.entrySet()) {
- generator.writeObjectFieldStart(refEntry.getKey());
-
- SnapshotRef ref = refEntry.getValue();
- generator.writeStringField(TYPE, ref.type().toString().toLowerCase(Locale.ROOT));
- generator.writeNumberField(SNAPSHOT_ID, ref.snapshotId());
- if (ref.maxRefAgeMs() != null) {
- generator.writeNumberField(MAX_REF_AGE_MS, ref.maxRefAgeMs());
- }
- if (ref.maxSnapshotAgeMs() != null) {
- generator.writeNumberField(MAX_SNAPSHOT_AGE_MS, ref.maxSnapshotAgeMs());
- }
- if (ref.minSnapshotsToKeep() != null) {
- generator.writeNumberField(MIN_SNAPSHOTS_TO_KEEP, ref.minSnapshotsToKeep());
- }
-
- generator.writeEndObject();
+ SnapshotRefParser.toJson(refEntry.getKey(), refEntry.getValue(), generator);
Review comment:
Updated. We can write the name field and leverage the existing toJson in SnapshotRefParser. I think this approach works because the ref can be serialized independently without any reliance on state. In TableMetadataParser we can serialize the name + ref entity together
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org