You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2021/11/02 13:36:26 UTC
[GitHub] [druid] EgorEna opened a new issue #11868: Druid jdbc lookups: why poll period isn't working?
EgorEna opened a new issue #11868:
URL: https://github.com/apache/druid/issues/11868
Hey, I created a lookup and was hoping that it will update within given time range
But actually it's not (data remains the same )
Here is what specified in the config.
```
{
"type": "cachedNamespace",
"extractionNamespace": {
"type": "jdbc",
"connectorConfig": {
"connectURI": "someuri",
"user": "someuser",
"password": "somepass"
},
"table": "sometable",
"keyColumn": "somekey",
"valueColumn": "somevalue",
"filters": "not thing_name is null"
},
"firstCacheTimeout": 0,
"pollPeriod": "PT5M",
"injective": true
}
```
Also I tried to create a lookup from example located [here](https://druid.apache.org/docs/latest/development/extensions-core/druid-lookups.html#example-of-polling-on-heap-lookup)
It looks like
```
{
"type": "pollingLookup",
"pollPeriod": "PT10M",
"dataFetcher": {
"type": "jdbcDataFetcher",
"connectorConfig": "someuri",
"table": "sometable",
"keyColumn": "somekey",
"valueColumn": "somevalue"
},
"cacheFactory": {
"type": "onHeapPolling"
}
}
```
First config is creating a lookup which is not updating
Second one (straight from the docs) doesn't appear in the lookup schema at all (telling that this lookup does not exist)
Can someone help and answer what is missing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] igorvpcleao commented on issue #11868: Druid jdbc lookups: why poll period isn't working?
Posted by GitBox <gi...@apache.org>.
igorvpcleao commented on issue #11868:
URL: https://github.com/apache/druid/issues/11868#issuecomment-960424966
Hey @EgorEna,
First things first, I'm just someone with moderate knowledge trying to help, not a Druid expert here.
By focusing on your first spec, do you see something like this when you look at your historical node logs?
```
2021-11-03T20:37:49,917 INFO [NamespaceExtractionCacheManager-1] org.apache.druid.server.lookup.namespace.JdbcCacheGenerator - Finished loading 2990136 values for namespace [JdbcExtractionNamespace{connectorConfig=DbConnectorConfig{createTables=false, connectURI='jdbc:mysql://REDACTED:3306/something?user='REDACTED', passwordProvider=org.apache.druid.metadata.DefaultPasswordProvider, dbcpProperties=null}, table='REDACTED', keyColumn='id', valueColumn='type', tsColumn='updated_at', filter='null', pollPeriod=P1D}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@22e17b56
```
1. I'd like to make sure that you're able to see a **Finished loading x values** message recently (or at least see an exception while loading it).
2. Also, does `"injective": true` really applies to you? It should only be used when both keys and values are unique.
3. Finally, by not updating you mean changing existing values, or not deleting entries that were previously deleted from your database?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org