You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/04/12 07:45:22 UTC
[GitHub] [pinot] nicovak opened a new issue, #8512: Handling optional parts in date format
nicovak opened a new issue, #8512:
URL: https://github.com/apache/pinot/issues/8512
Hello,
I think It could be a nice feature to handle optional parts in date format with `[]`
For example in my case, I can have DateTime with second fractions or not (it's adaptive).
Actually, I need to perform a `Substr` :
```json
FromDateTime(Substr(JSONPATH(payload, '$.created_at'), 0, 19), 'yyyy-MM-dd''T''HH:mm:ss''Z''')
```
But It could be handled like that :
```json
FromDateTime(JSONPATH(payload, '$.created_at'), 'yyyy-MM-dd''T''HH:mm:ss[.SSS]''Z''')
```
It's already provided by Java 8 and [DateTimeFormatter](https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html), in documentation we have :
```
[ optional section start
] optional section end
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #8512: [Feature Request] Handling optional parts in date format
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #8512:
URL: https://github.com/apache/pinot/issues/8512#issuecomment-1097101989
We don't support it right now because we are using joda library and haven't moved to java library for performance reason. Seems the author of joda is also recommending switching to java library, and this is another motivation for us to switch. @richardstartin how much performance difference do you observe between joda and java date time format?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] KKcorps commented on issue #8512: [Feature Request] Handling optional parts in date format
Posted by GitBox <gi...@apache.org>.
KKcorps commented on issue #8512:
URL: https://github.com/apache/pinot/issues/8512#issuecomment-1101265188
```
Benchmark Mode Cnt Score Error Units
BenchmarkJoda.benchmarkJavaDateStringToEpoch avgt 5 9241.257 ± 1233.009 us/op
BenchmarkJoda.benchmarkJodaStringToEpoch avgt 5 7330.508 ± 399.998 us/op
```
Updated the gist with code for this as well. Joda is faster in both cases
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] KKcorps commented on issue #8512: [Feature Request] Handling optional parts in date format
Posted by GitBox <gi...@apache.org>.
KKcorps commented on issue #8512:
URL: https://github.com/apache/pinot/issues/8512#issuecomment-1101281476
The above benchmarks were for Java 11. For Java 17, they seem much closer
```
Benchmark Mode Cnt Score Error Units
BenchmarkJoda.benchmarkJavaTime avgt 5 4986.148 ± 551.520 us/op
BenchmarkJoda.benchmarkJavaTimePreCompute avgt 5 4075.269 ± 691.534 us/op
BenchmarkJoda.benchmarkJoda avgt 5 3719.916 ± 319.401 us/op
BenchmarkJoda.benchmarkJavaDateStringToEpoch avgt 5 6362.567 ± 1251.276 us/op
BenchmarkJoda.benchmarkJodaStringToEpoch avgt 5 9887.073 ± 1456.975 us/op
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] KKcorps commented on issue #8512: [Feature Request] Handling optional parts in date format
Posted by GitBox <gi...@apache.org>.
KKcorps commented on issue #8512:
URL: https://github.com/apache/pinot/issues/8512#issuecomment-1097575529
From my benchmarks, java time seems 20-30% slower than Joda time.
```
Benchmark Mode Cnt Score Error Units
BenchmarkJoda.benchmarkJavaTime avgt 5 6077.279 ± 282.175 us/op
BenchmarkJoda.benchmarkJavaTimePerCompute avgt 5 6055.735 ± 254.637 us/op
BenchmarkJoda.benchmarkJoda avgt 5 4671.964 ± 93.222 us/op
```
Code - https://gist.github.com/KKcorps/19b6b093bfa15b7d62b27db0b8012e51
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] richardstartin commented on issue #8512: [Feature Request] Handling optional parts in date format
Posted by GitBox <gi...@apache.org>.
richardstartin commented on issue #8512:
URL: https://github.com/apache/pinot/issues/8512#issuecomment-1103567354
The decision requires a little more due diligence than that ;) I have seen much larger differences in favour of Joda recently, and the bit directly above those results which says "REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on why the numbers are the way they are." has been omitted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] richardstartin commented on issue #8512: [Feature Request] Handling optional parts in date format
Posted by GitBox <gi...@apache.org>.
richardstartin commented on issue #8512:
URL: https://github.com/apache/pinot/issues/8512#issuecomment-1103605984
Case in point, I found that Joda was 6x faster than Java time for year conversion here: https://github.com/apache/pinot/pull/8397#issuecomment-1077589112
There has been some progress in java.time recently:
* https://github.com/openjdk/jdk/pull/7957
* https://github.com/openjdk/jdk/pull/7989
* https://github.com/openjdk/jdk/pull/7990
* https://github.com/openjdk/jdk/pull/8014
* https://github.com/openjdk/jdk/pull/8039
* https://github.com/openjdk/jdk/pull/8056
Some of these improvements will be backported to JDK17, so we should reassess once Pinot can run on JDK17.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #8512: [Feature Request] Handling optional parts in date format
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #8512:
URL: https://github.com/apache/pinot/issues/8512#issuecomment-1098582246
@KKcorps Thanks for the benchmark! Can we also add a benchmark for parsing the formatted timestamp to long?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #8512: [Feature Request] Handling optional parts in date format
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #8512:
URL: https://github.com/apache/pinot/issues/8512#issuecomment-1103232634
Cool! Since the difference is not big, I think we should start migrating to java library
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org