You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by GitBox <gi...@apache.org> on 2021/07/14 16:54:50 UTC

[GitHub] [ignite-3] AMashenkov opened a new pull request #216: IGNITE-13669: Support temporal types.

AMashenkov opened a new pull request #216:
URL: https://github.com/apache/ignite-3/pull/216


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite-3] korlov42 commented on a change in pull request #216: IGNITE-13669: Support temporal types.

Posted by GitBox <gi...@apache.org>.
korlov42 commented on a change in pull request #216:
URL: https://github.com/apache/ignite-3/pull/216#discussion_r689451066



##########
File path: modules/schema/src/main/java/org/apache/ignite/internal/schema/row/TemporalTypesHelper.java
##########
@@ -0,0 +1,296 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.schema.row;
+
+import java.time.LocalDate;
+import java.time.LocalTime;
+import org.apache.ignite.internal.schema.TemporalNativeType;
+
+/**
+ * Helper class for temporal type conversions.
+ * <p>
+ * Provides methods to encode/decode temporal types in a compact way  for futher writing to row.
+ * Conversion preserves natural type order.
+ * <p>
+ * DATE is a fixed-length type which compacted representation keeps ordering, value is signed and fit into a 3-bytes.
+ * Thus, DATE value can be compared by bytes where first byte is signed and others - unsigned.
+ * Thus temporal functions, like YEAR(), can easily extracts fields with a mask,
+ * <p>
+ * Date compact structure:
+ * ┌──────────────┬─────────┬────────┐
+ * │ Year(signed) │ Month   │ Day    │
+ * ├──────────────┼─────────┼────────┤
+ * │ 15 bits      │ 4 bits  │ 5 bits │
+ * └──────────────┴─────────┴────────┘
+ * <p>
+ * TIME is a fixed-length type supporting accuracy from 1 second up to 1 nanosecond.
+ * Compacted time representation keeps ordering, values fits to 4-6 bytes value.
+ * The first 18 bits is used for hour, munutes and seconds, and the last bits for fractional seconds:

Review comment:
       ```suggestion
    * The first 18 bits is used for hours, minutes and seconds, and the last bits for fractional seconds:
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite-3] korlov42 commented on a change in pull request #216: IGNITE-13669: Support temporal types.

Posted by GitBox <gi...@apache.org>.
korlov42 commented on a change in pull request #216:
URL: https://github.com/apache/ignite-3/pull/216#discussion_r695533679



##########
File path: modules/api/src/main/java/org/apache/ignite/schema/ColumnType.java
##########
@@ -163,6 +178,38 @@ public static DecimalColumnType decimalOf() {
         );
     }
 
+    /**
+     * Returns timezone-free type representing a time of day in hours, minutes, seconds,
+     * and fractional seconds depending on type precision.
+     *
+     * @param precision Fractional seconds meaningful digits. Allowed values are 0-9 for second to nanosecond precision.
+     * @return Native type.
+     */
+    public static TemporalColumnType time(int precision) {

Review comment:
       ColumnType is a part of public API, thus I would prefer to use enum for precisions. Otherwise we 1) need to improve javadoc, since currently it's unclear what actually precision=4 means, and 2) throw an IllegalArgumentException in case someone pass a precision that out of valid range.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite-3] ygerzhedovich commented on a change in pull request #216: IGNITE-13669: Support temporal types.

Posted by GitBox <gi...@apache.org>.
ygerzhedovich commented on a change in pull request #216:
URL: https://github.com/apache/ignite-3/pull/216#discussion_r691147196



##########
File path: modules/schema/src/main/java/org/apache/ignite/internal/schema/configuration/SchemaConfigurationConverter.java
##########
@@ -90,15 +94,19 @@
         putType(ColumnType.FLOAT);
         putType(ColumnType.DOUBLE);
         putType(ColumnType.UUID);
+        putType(ColumnType.DATE);
     }
 
-    /** */
+    /**
+     *
+     */

Review comment:
       ```suggestion
       /**  */
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite-3] ygerzhedovich commented on a change in pull request #216: IGNITE-13669: Support temporal types.

Posted by GitBox <gi...@apache.org>.
ygerzhedovich commented on a change in pull request #216:
URL: https://github.com/apache/ignite-3/pull/216#discussion_r691145158



##########
File path: modules/api/src/main/java/org/apache/ignite/schema/ColumnType.java
##########
@@ -163,6 +166,37 @@ public static DecimalColumnType decimalOf() {
         );
     }
 
+    /**
+     * Timezone-free value representing a time of day in hours, minutes, seconds,
+     * and fractional seconds depending on type precision.
+     *
+     * @param precision Fractional seconds part length. Allowed values are 3/6/9 for millis/micros/nanos.

Review comment:
       I think it's old javadoc and just should be amended. We should support any values between 0 and 9




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite-3] ygerzhedovich commented on a change in pull request #216: IGNITE-13669: Support temporal types.

Posted by GitBox <gi...@apache.org>.
ygerzhedovich commented on a change in pull request #216:
URL: https://github.com/apache/ignite-3/pull/216#discussion_r674858564



##########
File path: modules/api/src/main/java/org/apache/ignite/schema/ColumnType.java
##########
@@ -116,6 +119,37 @@ public static NumericColumnType number(int precision, int scale) {
         return new NumericColumnType(ColumnTypeSpec.DECIMAL, precision, scale);
     }
 
+    /**
+     * Timezone-free value representing a time of day in hours, minutes, seconds,
+     * and fractional seconds depending on type precision.
+     *
+     * @param precision Fractional seconds part length. Allowed values are 3/6/9 for millis/micros/nanos.

Review comment:
       Why we have such restrictions? For example, Postgresql could use any precision between 0 and 6, in case the user set more than 6 then use 6




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite-3] korlov42 commented on a change in pull request #216: IGNITE-13669: Support temporal types.

Posted by GitBox <gi...@apache.org>.
korlov42 commented on a change in pull request #216:
URL: https://github.com/apache/ignite-3/pull/216#discussion_r689451066



##########
File path: modules/schema/src/main/java/org/apache/ignite/internal/schema/row/TemporalTypesHelper.java
##########
@@ -0,0 +1,296 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.schema.row;
+
+import java.time.LocalDate;
+import java.time.LocalTime;
+import org.apache.ignite.internal.schema.TemporalNativeType;
+
+/**
+ * Helper class for temporal type conversions.
+ * <p>
+ * Provides methods to encode/decode temporal types in a compact way  for futher writing to row.
+ * Conversion preserves natural type order.
+ * <p>
+ * DATE is a fixed-length type which compacted representation keeps ordering, value is signed and fit into a 3-bytes.
+ * Thus, DATE value can be compared by bytes where first byte is signed and others - unsigned.
+ * Thus temporal functions, like YEAR(), can easily extracts fields with a mask,
+ * <p>
+ * Date compact structure:
+ * ┌──────────────┬─────────┬────────┐
+ * │ Year(signed) │ Month   │ Day    │
+ * ├──────────────┼─────────┼────────┤
+ * │ 15 bits      │ 4 bits  │ 5 bits │
+ * └──────────────┴─────────┴────────┘
+ * <p>
+ * TIME is a fixed-length type supporting accuracy from 1 second up to 1 nanosecond.
+ * Compacted time representation keeps ordering, values fits to 4-6 bytes value.
+ * The first 18 bits is used for hour, munutes and seconds, and the last bits for fractional seconds:

Review comment:
       ```suggestion
    * The first 18 bits is used for hour, minutes and seconds, and the last bits for fractional seconds:
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite-3] korlov42 commented on a change in pull request #216: IGNITE-13669: Support temporal types.

Posted by GitBox <gi...@apache.org>.
korlov42 commented on a change in pull request #216:
URL: https://github.com/apache/ignite-3/pull/216#discussion_r689472023



##########
File path: modules/schema/src/main/java/org/apache/ignite/internal/schema/row/TemporalTypesHelper.java
##########
@@ -0,0 +1,296 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.schema.row;
+
+import java.time.LocalDate;
+import java.time.LocalTime;
+import org.apache.ignite.internal.schema.TemporalNativeType;
+
+/**
+ * Helper class for temporal type conversions.
+ * <p>
+ * Provides methods to encode/decode temporal types in a compact way  for futher writing to row.
+ * Conversion preserves natural type order.
+ * <p>
+ * DATE is a fixed-length type which compacted representation keeps ordering, value is signed and fit into a 3-bytes.
+ * Thus, DATE value can be compared by bytes where first byte is signed and others - unsigned.
+ * Thus temporal functions, like YEAR(), can easily extracts fields with a mask,
+ * <p>
+ * Date compact structure:
+ * ┌──────────────┬─────────┬────────┐
+ * │ Year(signed) │ Month   │ Day    │
+ * ├──────────────┼─────────┼────────┤
+ * │ 15 bits      │ 4 bits  │ 5 bits │
+ * └──────────────┴─────────┴────────┘
+ * <p>
+ * TIME is a fixed-length type supporting accuracy from 1 second up to 1 nanosecond.
+ * Compacted time representation keeps ordering, values fits to 4-6 bytes value.
+ * The first 18 bits is used for hour, munutes and seconds, and the last bits for fractional seconds:
+ * 14 for millisecond precision and 30 for nanosecond.
+ * Values of a type of any intermediate precisions is normalized to the type,
+ * then stored as shortest possible structure without a precision lost.
+ * <p>
+ * Time compact structure:
+ * ┌─────────┬─────────┬──────────┬────────────┐
+ * │ Hours   │ Minutes │ Seconds  │ Subseconds │
+ * ├─────────┼─────────┼──────────┼────────────┤
+ * │ 6 bit   │ 6 bits  │ 6 bit    │ 14 bits    │ - 32-bits in total.
+ * │ 6 bit   │ 6 bits  │ 6 bit    │ 30 bits    │ - 48-bits in total.
+ * └─────────┴─────────┴──────────┴────────────┘
+ * <p>
+ * DATETIME is just a concatenation of DATE and TIME values.
+ * <p>
+ * TIMESTAMP has similart structure to {@link java.time.Instant} and supports precision from 1 second up to 1 nanosecond.
+ * Subsecond part stores in a separate bit sequence which is omited for {@code 0} accuracy.
+ * <p>
+ * Total value size is 8/12 bytes depending on the type precision.
+ * <p>
+ * Timestamp compact structure:
+ * ┌──────────────────────────┬─────────────┐
+ * │ Seconds since the epoch  │ Subseconds  │
+ * ├──────────────────────────┼─────────────┤
+ * │    64 bits               │ 0/32 bits   │
+ * └──────────────────────────┴─────────────┘
+ *
+ * @see org.apache.ignite.internal.schema.row.Row
+ * @see org.apache.ignite.internal.schema.row.RowAssembler
+ */
+public class TemporalTypesHelper {
+    /** Month field length. */
+    public static final int MONTH_FIELD_LENGTH = 4;
+
+    /** Day field length. */
+    public static final int DAY_FIELD_LENGTH = 5;
+
+    /** Hours field length. */
+    public static final int HOUR_FIELD_LENGTH = 6;
+
+    /** Minutes field length. */
+    public static final int MINUTES_FIELD_LENGTH = 6;
+
+    /** Seconds field length. */
+    public static final int SECONDS_FIELD_LENGTH = 6;
+
+    /** Max year boundary. */
+    public static final int MAX_YEAR = (1 << 14) - 1;
+
+    /** Min year boundary. */
+    public static final int MIN_YEAR = -(1 << 14);
+
+    /** Fractional part length for millis precision. */
+    public static final int MILLIS_PART_LEN = 14;
+
+    /** Fractional part mask for millis precision. */
+    public static final long MILLIS_PART_MASK = (1L << MILLIS_PART_LEN) - 1;
+
+    /** Fractional part length for nanos precision. */
+    public static final int NANOS_PART_LEN = 30;
+
+    /** Fractional part mask for nanos precision. */
+    public static final long NANOS_PART_MASK = (1L << NANOS_PART_LEN) - 1;
+
+    /**
+     * @param len Mask length in bits.
+     * @return Mask.
+     */
+    private static int mask(int len) {
+        return (1 << len) - 1;
+    }
+
+    /**
+     * Compact LocalDate.
+     *
+     * @param date Date.
+     * @return Encoded date.
+     */
+    public static int encodeDate(LocalDate date) {
+        int val = date.getYear() << MONTH_FIELD_LENGTH;
+        val = (val | date.getMonthValue()) << DAY_FIELD_LENGTH;
+        val |= date.getDayOfMonth();
+
+        return val & (0x00FF_FFFF);
+    }
+
+    /**
+     * Expands to LocalDate.
+     *
+     * @param date Encoded date.
+     * @return LocalDate instance.
+     */
+    public static LocalDate decodeDate(int date) {
+        date = (date << 8) >> 8; // Restore sign.
+
+        int day = (date) & mask(DAY_FIELD_LENGTH);
+        int mon = (date >>= DAY_FIELD_LENGTH) & mask(MONTH_FIELD_LENGTH); // Sign matters.

Review comment:
       does sign really matter for month?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite-3] korlov42 commented on a change in pull request #216: IGNITE-13669: Support temporal types.

Posted by GitBox <gi...@apache.org>.
korlov42 commented on a change in pull request #216:
URL: https://github.com/apache/ignite-3/pull/216#discussion_r689449420



##########
File path: modules/schema/src/main/java/org/apache/ignite/internal/schema/row/TemporalTypesHelper.java
##########
@@ -0,0 +1,296 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.schema.row;
+
+import java.time.LocalDate;
+import java.time.LocalTime;
+import org.apache.ignite.internal.schema.TemporalNativeType;
+
+/**
+ * Helper class for temporal type conversions.
+ * <p>
+ * Provides methods to encode/decode temporal types in a compact way  for futher writing to row.

Review comment:
       ```suggestion
    * Provides methods to encode/decode temporal types in a compact way for futher writing to row.
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite-3] ygerzhedovich commented on a change in pull request #216: IGNITE-13669: Support temporal types.

Posted by GitBox <gi...@apache.org>.
ygerzhedovich commented on a change in pull request #216:
URL: https://github.com/apache/ignite-3/pull/216#discussion_r691145158



##########
File path: modules/api/src/main/java/org/apache/ignite/schema/ColumnType.java
##########
@@ -163,6 +166,37 @@ public static DecimalColumnType decimalOf() {
         );
     }
 
+    /**
+     * Timezone-free value representing a time of day in hours, minutes, seconds,
+     * and fractional seconds depending on type precision.
+     *
+     * @param precision Fractional seconds part length. Allowed values are 3/6/9 for millis/micros/nanos.

Review comment:
       I think it's old javadoc and just should be amended. We should support any values between 0 and 9

##########
File path: modules/schema/src/main/java/org/apache/ignite/internal/schema/configuration/SchemaConfigurationConverter.java
##########
@@ -90,15 +94,19 @@
         putType(ColumnType.FLOAT);
         putType(ColumnType.DOUBLE);
         putType(ColumnType.UUID);
+        putType(ColumnType.DATE);
     }
 
-    /** */
+    /**
+     *
+     */

Review comment:
       ```suggestion
       /**  */
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite-3] AMashenkov merged pull request #216: IGNITE-13669: Support temporal types.

Posted by GitBox <gi...@apache.org>.
AMashenkov merged pull request #216:
URL: https://github.com/apache/ignite-3/pull/216


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite-3] korlov42 commented on a change in pull request #216: IGNITE-13669: Support temporal types.

Posted by GitBox <gi...@apache.org>.
korlov42 commented on a change in pull request #216:
URL: https://github.com/apache/ignite-3/pull/216#discussion_r689388472



##########
File path: modules/schema/src/main/java/org/apache/ignite/internal/schema/configuration/SchemaDescriptorConverter.java
##########
@@ -95,21 +99,37 @@ private static NativeType convert(ColumnType colType) {
             case BITMASK:
                 return NativeTypes.bitmaskOf(((ColumnType.VarLenColumnType)colType).length());
 
-            case STRING:
+            case STRING: {
                 int strLen = ((ColumnType.VarLenColumnType)colType).length();
 
                 if (strLen == 0)
                     strLen = Integer.MAX_VALUE;
 
                 return NativeTypes.stringOf(strLen);
-
-            case BLOB:
+            }
+            case BLOB: {
                 int blobLen = ((ColumnType.VarLenColumnType)colType).length();
 
                 if (blobLen == 0)
                     blobLen = Integer.MAX_VALUE;
 
                 return NativeTypes.blobOf(blobLen);
+            }
+            case TIME: {

Review comment:
       should we add here DATE as well? 

##########
File path: modules/schema/src/main/java/org/apache/ignite/internal/schema/row/Row.java
##########
@@ -31,13 +35,20 @@
 import org.apache.ignite.internal.schema.InvalidTypeException;
 import org.apache.ignite.internal.schema.NativeTypeSpec;
 import org.apache.ignite.internal.schema.SchemaDescriptor;
+import org.apache.ignite.internal.schema.TemporalNativeType;
 
 /**
  * Schema-aware row.
  * <p>
  * The class contains non-generic methods to read boxed and unboxed primitives based on the schema column types.
  * Any type conversions and coercions should be implemented outside the row by the key-value or query runtime.
+ * <p>
  * When a non-boxed primitive is read from a null column value, it is converted to the primitive type default value.
+ * When a unknown column value requested, the a column default will be returned.

Review comment:
       ```suggestion
    * When an unknown column value requested, the column default will be returned.
   ```

##########
File path: modules/api/src/main/java/org/apache/ignite/schema/ColumnType.java
##########
@@ -163,6 +166,37 @@ public static DecimalColumnType decimalOf() {
         );
     }
 
+    /**
+     * Timezone-free value representing a time of day in hours, minutes, seconds,
+     * and fractional seconds depending on type precision.
+     *
+     * @param precision Fractional seconds part length. Allowed values are 3/6/9 for millis/micros/nanos.

Review comment:
       perhaps, it would be better to introduce enum for such length




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite-3] korlov42 commented on a change in pull request #216: IGNITE-13669: Support temporal types.

Posted by GitBox <gi...@apache.org>.
korlov42 commented on a change in pull request #216:
URL: https://github.com/apache/ignite-3/pull/216#discussion_r689397678



##########
File path: modules/schema/src/main/java/org/apache/ignite/internal/schema/NativeTypes.java
##########
@@ -108,6 +111,65 @@ public static NativeType decimalOf(int precision, int scale) {
         return new DecimalNativeType(precision, scale);
     }
 
+    /**
+     * Creates a TIME type with subsecond precision.
+     *
+     * @param precision Time subsecond part length in digits. Allowed values 0-9.
+     * @return Native type.
+     */
+    public static NativeType time(int precision) {
+        return TemporalNativeType.time(precision);
+    }
+
+    /**
+     * Creates DATETIME type as pair (date, time).
+     *
+     * @param precision Time subsecond part length in digits. Allowed values 0-9.
+     * @return Native type.
+     */
+    public static NativeType datetime(int precision) {
+        return TemporalNativeType.datetime(precision);
+    }
+
+    /**
+     * Creates TIMESTAMP type.
+     *
+     * @param precision Time subsecond part length. Allowed values are 3/6/9 for millis/micros/nanos.
+     * @return Native type.
+     */
+    public static NativeType timestamp(int precision) {
+        return TemporalNativeType.timestamp(precision);
+    }
+
+    /**
+     * Creates a TIME type with default precision.
+     *
+     * @return Native type.
+     * @see #time(int)
+     */
+    public static NativeType time() {
+        return TemporalNativeType.time(ColumnType.TemporalColumnType.DEFAULT_PRECISION);
+    }
+
+    /**
+     * Creates DATETIME type with default precision.
+     *
+     * @return Native type.
+     * @see #datetime(int)
+     */
+    public static NativeType datetime() {
+        return TemporalNativeType.datetime(ColumnType.TemporalColumnType.DEFAULT_PRECISION);
+    }
+
+    /**
+     * Creates TIMESTAMP type with default precision.
+     *
+     * @return Native type.

Review comment:
       ```suggestion
        * @return Native type.
        * @see #timestamp(int)
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite-3] korlov42 commented on a change in pull request #216: IGNITE-13669: Support temporal types.

Posted by GitBox <gi...@apache.org>.
korlov42 commented on a change in pull request #216:
URL: https://github.com/apache/ignite-3/pull/216#discussion_r689466951



##########
File path: modules/schema/src/main/java/org/apache/ignite/internal/schema/row/TemporalTypesHelper.java
##########
@@ -0,0 +1,296 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.schema.row;
+
+import java.time.LocalDate;
+import java.time.LocalTime;
+import org.apache.ignite.internal.schema.TemporalNativeType;
+
+/**
+ * Helper class for temporal type conversions.
+ * <p>
+ * Provides methods to encode/decode temporal types in a compact way  for futher writing to row.
+ * Conversion preserves natural type order.
+ * <p>
+ * DATE is a fixed-length type which compacted representation keeps ordering, value is signed and fit into a 3-bytes.
+ * Thus, DATE value can be compared by bytes where first byte is signed and others - unsigned.
+ * Thus temporal functions, like YEAR(), can easily extracts fields with a mask,
+ * <p>
+ * Date compact structure:
+ * ┌──────────────┬─────────┬────────┐
+ * │ Year(signed) │ Month   │ Day    │
+ * ├──────────────┼─────────┼────────┤
+ * │ 15 bits      │ 4 bits  │ 5 bits │
+ * └──────────────┴─────────┴────────┘
+ * <p>
+ * TIME is a fixed-length type supporting accuracy from 1 second up to 1 nanosecond.
+ * Compacted time representation keeps ordering, values fits to 4-6 bytes value.
+ * The first 18 bits is used for hour, munutes and seconds, and the last bits for fractional seconds:
+ * 14 for millisecond precision and 30 for nanosecond.
+ * Values of a type of any intermediate precisions is normalized to the type,
+ * then stored as shortest possible structure without a precision lost.
+ * <p>
+ * Time compact structure:
+ * ┌─────────┬─────────┬──────────┬────────────┐
+ * │ Hours   │ Minutes │ Seconds  │ Subseconds │
+ * ├─────────┼─────────┼──────────┼────────────┤
+ * │ 6 bit   │ 6 bits  │ 6 bit    │ 14 bits    │ - 32-bits in total.
+ * │ 6 bit   │ 6 bits  │ 6 bit    │ 30 bits    │ - 48-bits in total.
+ * └─────────┴─────────┴──────────┴────────────┘
+ * <p>
+ * DATETIME is just a concatenation of DATE and TIME values.
+ * <p>
+ * TIMESTAMP has similart structure to {@link java.time.Instant} and supports precision from 1 second up to 1 nanosecond.
+ * Subsecond part stores in a separate bit sequence which is omited for {@code 0} accuracy.
+ * <p>
+ * Total value size is 8/12 bytes depending on the type precision.
+ * <p>
+ * Timestamp compact structure:
+ * ┌──────────────────────────┬─────────────┐
+ * │ Seconds since the epoch  │ Subseconds  │
+ * ├──────────────────────────┼─────────────┤
+ * │    64 bits               │ 0/32 bits   │
+ * └──────────────────────────┴─────────────┘
+ *
+ * @see org.apache.ignite.internal.schema.row.Row
+ * @see org.apache.ignite.internal.schema.row.RowAssembler
+ */
+public class TemporalTypesHelper {
+    /** Month field length. */
+    public static final int MONTH_FIELD_LENGTH = 4;
+
+    /** Day field length. */
+    public static final int DAY_FIELD_LENGTH = 5;
+
+    /** Hours field length. */
+    public static final int HOUR_FIELD_LENGTH = 6;

Review comment:
       Why do we use 6 bits to store hours?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite-3] korlov42 commented on a change in pull request #216: IGNITE-13669: Support temporal types.

Posted by GitBox <gi...@apache.org>.
korlov42 commented on a change in pull request #216:
URL: https://github.com/apache/ignite-3/pull/216#discussion_r689446898



##########
File path: modules/schema/src/main/java/org/apache/ignite/internal/schema/row/RowAssembler.java
##########
@@ -39,15 +43,22 @@
 import org.apache.ignite.internal.schema.NativeTypes;
 import org.apache.ignite.internal.schema.NumberNativeType;
 import org.apache.ignite.internal.schema.SchemaDescriptor;
+import org.apache.ignite.internal.schema.TemporalNativeType;
 
 import static org.apache.ignite.internal.schema.BinaryRow.RowFlags.KEY_FLAGS_OFFSET;
 import static org.apache.ignite.internal.schema.BinaryRow.RowFlags.VAL_FLAGS_OFFSET;
 
 /**
  * Utility class to build rows using column appending pattern. The external user of this class must consult
  * with the schema and provide the columns in strict internal column sort order during the row construction.
+ * <p>
  * Additionally, the user of this class should pre-calculate the resulting row size when possible to avoid
- * unnecessary data copies and allow some size-optimizations can be applied.
+ * unnecessary data copies  and allow some size-optimizations can be applied.

Review comment:
       ```suggestion
    * unnecessary data copies and allow some size optimizations to be applied.
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite-3] korlov42 commented on a change in pull request #216: IGNITE-13669: Support temporal types.

Posted by GitBox <gi...@apache.org>.
korlov42 commented on a change in pull request #216:
URL: https://github.com/apache/ignite-3/pull/216#discussion_r689473012



##########
File path: modules/schema/src/main/java/org/apache/ignite/internal/schema/row/TemporalTypesHelper.java
##########
@@ -0,0 +1,296 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.schema.row;
+
+import java.time.LocalDate;
+import java.time.LocalTime;
+import org.apache.ignite.internal.schema.TemporalNativeType;
+
+/**
+ * Helper class for temporal type conversions.
+ * <p>
+ * Provides methods to encode/decode temporal types in a compact way  for futher writing to row.
+ * Conversion preserves natural type order.
+ * <p>
+ * DATE is a fixed-length type which compacted representation keeps ordering, value is signed and fit into a 3-bytes.
+ * Thus, DATE value can be compared by bytes where first byte is signed and others - unsigned.
+ * Thus temporal functions, like YEAR(), can easily extracts fields with a mask,
+ * <p>
+ * Date compact structure:
+ * ┌──────────────┬─────────┬────────┐
+ * │ Year(signed) │ Month   │ Day    │
+ * ├──────────────┼─────────┼────────┤
+ * │ 15 bits      │ 4 bits  │ 5 bits │
+ * └──────────────┴─────────┴────────┘
+ * <p>
+ * TIME is a fixed-length type supporting accuracy from 1 second up to 1 nanosecond.
+ * Compacted time representation keeps ordering, values fits to 4-6 bytes value.
+ * The first 18 bits is used for hour, munutes and seconds, and the last bits for fractional seconds:
+ * 14 for millisecond precision and 30 for nanosecond.
+ * Values of a type of any intermediate precisions is normalized to the type,
+ * then stored as shortest possible structure without a precision lost.
+ * <p>
+ * Time compact structure:
+ * ┌─────────┬─────────┬──────────┬────────────┐
+ * │ Hours   │ Minutes │ Seconds  │ Subseconds │
+ * ├─────────┼─────────┼──────────┼────────────┤
+ * │ 6 bit   │ 6 bits  │ 6 bit    │ 14 bits    │ - 32-bits in total.
+ * │ 6 bit   │ 6 bits  │ 6 bit    │ 30 bits    │ - 48-bits in total.
+ * └─────────┴─────────┴──────────┴────────────┘
+ * <p>
+ * DATETIME is just a concatenation of DATE and TIME values.
+ * <p>
+ * TIMESTAMP has similart structure to {@link java.time.Instant} and supports precision from 1 second up to 1 nanosecond.
+ * Subsecond part stores in a separate bit sequence which is omited for {@code 0} accuracy.
+ * <p>
+ * Total value size is 8/12 bytes depending on the type precision.
+ * <p>
+ * Timestamp compact structure:
+ * ┌──────────────────────────┬─────────────┐
+ * │ Seconds since the epoch  │ Subseconds  │
+ * ├──────────────────────────┼─────────────┤
+ * │    64 bits               │ 0/32 bits   │
+ * └──────────────────────────┴─────────────┘
+ *
+ * @see org.apache.ignite.internal.schema.row.Row
+ * @see org.apache.ignite.internal.schema.row.RowAssembler
+ */
+public class TemporalTypesHelper {
+    /** Month field length. */
+    public static final int MONTH_FIELD_LENGTH = 4;
+
+    /** Day field length. */
+    public static final int DAY_FIELD_LENGTH = 5;
+
+    /** Hours field length. */
+    public static final int HOUR_FIELD_LENGTH = 6;
+
+    /** Minutes field length. */
+    public static final int MINUTES_FIELD_LENGTH = 6;
+
+    /** Seconds field length. */
+    public static final int SECONDS_FIELD_LENGTH = 6;
+
+    /** Max year boundary. */
+    public static final int MAX_YEAR = (1 << 14) - 1;
+
+    /** Min year boundary. */
+    public static final int MIN_YEAR = -(1 << 14);
+
+    /** Fractional part length for millis precision. */
+    public static final int MILLIS_PART_LEN = 14;
+
+    /** Fractional part mask for millis precision. */
+    public static final long MILLIS_PART_MASK = (1L << MILLIS_PART_LEN) - 1;
+
+    /** Fractional part length for nanos precision. */
+    public static final int NANOS_PART_LEN = 30;
+
+    /** Fractional part mask for nanos precision. */
+    public static final long NANOS_PART_MASK = (1L << NANOS_PART_LEN) - 1;
+
+    /**
+     * @param len Mask length in bits.
+     * @return Mask.
+     */
+    private static int mask(int len) {
+        return (1 << len) - 1;
+    }
+
+    /**
+     * Compact LocalDate.
+     *
+     * @param date Date.
+     * @return Encoded date.
+     */
+    public static int encodeDate(LocalDate date) {
+        int val = date.getYear() << MONTH_FIELD_LENGTH;
+        val = (val | date.getMonthValue()) << DAY_FIELD_LENGTH;
+        val |= date.getDayOfMonth();
+
+        return val & (0x00FF_FFFF);
+    }
+
+    /**
+     * Expands to LocalDate.
+     *
+     * @param date Encoded date.
+     * @return LocalDate instance.
+     */
+    public static LocalDate decodeDate(int date) {
+        date = (date << 8) >> 8; // Restore sign.
+
+        int day = (date) & mask(DAY_FIELD_LENGTH);
+        int mon = (date >>= DAY_FIELD_LENGTH) & mask(MONTH_FIELD_LENGTH); // Sign matters.
+        int year = (date >> MONTH_FIELD_LENGTH); // Sign matters.
+
+        return LocalDate.of(year, mon, day);
+    }
+
+    /**
+     * Encode LocalTime to long as concatenation of 2 ints:
+     * encoded time with precision of seconds and fractional seconds.
+     *
+     * @param type Native temporal type.
+     * @param localTime Time.
+     * @return Encoded local time.
+     * @see #NANOS_PART_LEN
+     * @see #MILLIS_PART_LEN
+     */
+    public static long encodeTime(TemporalNativeType type, LocalTime localTime) {
+        int time = localTime.getHour() << (MINUTES_FIELD_LENGTH + SECONDS_FIELD_LENGTH);
+        time |= localTime.getMinute() << SECONDS_FIELD_LENGTH;
+        time |= localTime.getSecond();
+
+        int fractinal = truncateTo(type.precision(), localTime.getNano());

Review comment:
       ```suggestion
           int fractional = truncateTo(type.precision(), localTime.getNano());
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite-3] korlov42 commented on a change in pull request #216: IGNITE-13669: Support temporal types.

Posted by GitBox <gi...@apache.org>.
korlov42 commented on a change in pull request #216:
URL: https://github.com/apache/ignite-3/pull/216#discussion_r689453098



##########
File path: modules/schema/src/main/java/org/apache/ignite/internal/schema/row/TemporalTypesHelper.java
##########
@@ -0,0 +1,296 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.schema.row;
+
+import java.time.LocalDate;
+import java.time.LocalTime;
+import org.apache.ignite.internal.schema.TemporalNativeType;
+
+/**
+ * Helper class for temporal type conversions.
+ * <p>
+ * Provides methods to encode/decode temporal types in a compact way  for futher writing to row.
+ * Conversion preserves natural type order.
+ * <p>
+ * DATE is a fixed-length type which compacted representation keeps ordering, value is signed and fit into a 3-bytes.
+ * Thus, DATE value can be compared by bytes where first byte is signed and others - unsigned.
+ * Thus temporal functions, like YEAR(), can easily extracts fields with a mask,
+ * <p>
+ * Date compact structure:
+ * ┌──────────────┬─────────┬────────┐
+ * │ Year(signed) │ Month   │ Day    │
+ * ├──────────────┼─────────┼────────┤
+ * │ 15 bits      │ 4 bits  │ 5 bits │
+ * └──────────────┴─────────┴────────┘
+ * <p>
+ * TIME is a fixed-length type supporting accuracy from 1 second up to 1 nanosecond.
+ * Compacted time representation keeps ordering, values fits to 4-6 bytes value.
+ * The first 18 bits is used for hour, munutes and seconds, and the last bits for fractional seconds:
+ * 14 for millisecond precision and 30 for nanosecond.
+ * Values of a type of any intermediate precisions is normalized to the type,
+ * then stored as shortest possible structure without a precision lost.
+ * <p>
+ * Time compact structure:
+ * ┌─────────┬─────────┬──────────┬────────────┐
+ * │ Hours   │ Minutes │ Seconds  │ Subseconds │
+ * ├─────────┼─────────┼──────────┼────────────┤
+ * │ 6 bit   │ 6 bits  │ 6 bit    │ 14 bits    │ - 32-bits in total.
+ * │ 6 bit   │ 6 bits  │ 6 bit    │ 30 bits    │ - 48-bits in total.
+ * └─────────┴─────────┴──────────┴────────────┘
+ * <p>
+ * DATETIME is just a concatenation of DATE and TIME values.
+ * <p>
+ * TIMESTAMP has similart structure to {@link java.time.Instant} and supports precision from 1 second up to 1 nanosecond.
+ * Subsecond part stores in a separate bit sequence which is omited for {@code 0} accuracy.

Review comment:
       ```suggestion
    * TIMESTAMP has a structure similar to {@link java.time.Instant} and supports precision from 1 second up to 1 nanosecond.
    * Subsecond part is stored in a separate bit sequence which is omitted for {@code 0} accuracy.
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org