You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/09/07 06:30:21 UTC
[jira] [Commented] (KAFKA-3776) Unify store and downstream caching
in streams
[ https://issues.apache.org/jira/browse/KAFKA-3776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15469728#comment-15469728 ]
ASF GitHub Bot commented on KAFKA-3776:
---------------------------------------
GitHub user enothereska reopened a pull request:
https://github.com/apache/kafka/pull/1752
KAFKA-3776: Unify store and downstream caching in streams
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/enothereska/kafka KAFKA-3776-poc
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/1752.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1752
----
commit 9e6d1e0b7b50b8c9bb1848c9e72913701a93cbb0
Author: Eno Thereska <en...@gmail.com>
Date: 2016-08-16T09:45:00Z
Initial commit with stub cache
commit 2a3e46770b2aaddefd843f7cc6cc6727dd1cbc9e
Author: Eno Thereska <en...@gmail.com>
Date: 2016-08-17T08:22:23Z
Adjustments so tests compile
commit 86bb6dcb6beac3d136e01f45cd727e6a6a691b5b
Author: Eno Thereska <en...@gmail.com>
Date: 2016-08-17T10:21:11Z
Remove old cache from RocksDbStore and add global cache to RocksDb
commit 989bce66ddbfc150081cbf74edc8d3d2b6ae70d3
Author: Eno Thereska <en...@gmail.com>
Date: 2016-08-17T11:27:08Z
Create unique cache key per store
commit 959885d4807e7f90030bca946dc5c1bdc4098754
Author: Eno Thereska <en...@gmail.com>
Date: 2016-08-17T12:51:17Z
Enable caching for RocksDBWindowStore
commit 6f582ceb98a5dbd37732e148a0e2f064a90d5d13
Author: Eno Thereska <en...@gmail.com>
Date: 2016-08-17T14:17:07Z
Unit test for windowed stores
commit 7810ab08b02b44e03812e9838062f811c64be0e7
Author: Eno Thereska <en...@gmail.com>
Date: 2016-08-19T14:45:27Z
Plugged in new byte-based cache
commit 333e7992b5f038d1d46d1a17d31a41b700552c77
Author: Damian Guy <da...@gmail.com>
Date: 2016-08-22T15:07:00Z
Add ProcessorRecordContext for tracking timestamp,offset etc of record. Update Procesor and Store APIs to use context. Dont forward values from Processors that are using a Store. RocksDBStores now have a flush listener and forward to processors whenever the cache is flushed. Left one failing test that highlights the need for the range queries to not forward values downstream
commit 124ffdf84a42e0fe14f3be6897baecdb8f8b7ef0
Author: Damian Guy <da...@gmail.com>
Date: 2016-08-22T15:56:36Z
merge
commit 34985fae49084b4ee37f0a1cc0c3fdd024a89056
Author: Eno Thereska <en...@gmail.com>
Date: 2016-08-23T06:58:49Z
Initial pass at range queries. Cache based on TreeMap
commit 30bde7301e7541cf2937f56b0303a43900db1fae
Author: Eno Thereska <en...@gmail.com>
Date: 2016-08-23T07:15:19Z
Remove deleted entry from cache
commit 9dbafc07701c5acd6d2c2edbb1f3770670180673
Author: Damian Guy <da...@gmail.com>
Date: 2016-08-23T09:37:17Z
revert api changes. Track RecordContext via ProcessorContext
commit 63aacc465510bfc8cdb0b675b2d4ebb041a5e99c
Author: Eno Thereska <en...@gmail.com>
Date: 2016-08-23T13:18:19Z
Pass at the 'all' method using cache iterator
commit af6686fb298c2417c5eff9a46cbb8b898df894df
Author: Eno Thereska <en...@gmail.com>
Date: 2016-08-23T13:18:24Z
Merge branch 'KAFKA-3776-poc' of https://github.com/enothereska/kafka into KAFKA-3776-poc
commit 235b8612a121283841f8b3b8795ff6ff5c425c07
Author: Damian Guy <da...@gmail.com>
Date: 2016-08-24T09:24:02Z
disable caching for joins. expose enableCaching method on PersistentKeyValueFactory
commit a6986a66b08d653b4aa019557f39f0713949fe83
Author: Eno Thereska <en...@gmail.com>
Date: 2016-08-24T10:17:30Z
Cleanup memory cache
commit 5b71d68d84007a03d59a3095467785da461d3c20
Author: Eno Thereska <en...@gmail.com>
Date: 2016-08-24T10:25:44Z
Merged
commit 7658e02a4b203d09d607c0a510d0ae3421eb71a9
Author: Damian Guy <da...@gmail.com>
Date: 2016-08-24T11:21:35Z
forward before changelog.
commit 91bab64db1bf3673169879c80a15af67f4670b70
Author: Damian Guy <da...@gmail.com>
Date: 2016-08-24T11:21:40Z
Merge changes from Eno
commit fef6cae098b44cf2e715db00c84d97650114802f
Author: Eno Thereska <en...@gmail.com>
Date: 2016-08-24T13:19:12Z
Flush order should be top-down
commit e1bcd35730a10c06a6e49d3882d787611b106f95
Author: Eno Thereska <en...@gmail.com>
Date: 2016-08-24T15:50:56Z
Merged with trunk
commit 591d4802383d6e8dc0a3e0e97cd619dd99d3db0e
Author: Damian Guy <da...@gmail.com>
Date: 2016-08-24T19:35:28Z
extract caching out of store
commit dbd383c9066de031f48e8539ad071834b294b7b0
Author: Damian Guy <da...@gmail.com>
Date: 2016-08-25T06:30:27Z
Merge pull request #1 from enothereska/dg-3776-poc
extract caching out of store
commit 290a66b087847ceb67640574a7a6fc8c8e4d33db
Author: Damian Guy <da...@gmail.com>
Date: 2016-08-25T07:12:55Z
rename MergedSortedCacheRocksDBIterator -> MergedSortedCacheKeyValueStoreIterator
commit 8a9981b16e59f2dcf656e698545a09733600a5b3
Author: Damian Guy <da...@gmail.com>
Date: 2016-08-25T10:34:07Z
refactor MergedSortedIterator. Some tests for MemoryLRUCacheIterator. Add PeekingKeyValueIterator
commit c5f45775fb8ff275bd27bf6bef90290a53bb2199
Author: Damian Guy <da...@gmail.com>
Date: 2016-08-25T14:37:21Z
more tests. fix bugs
commit 9ebdbd49164fdb281609bd8968c0d68453c2236c
Author: Damian Guy <da...@gmail.com>
Date: 2016-08-25T16:54:53Z
move all namespace related code into cache. make sure overlapping names do not clash
commit 4b4228a27b5f0d5ab3f3153e7d2898423d230712
Author: Damian Guy <da...@gmail.com>
Date: 2016-08-25T17:16:01Z
add generic types to DelegatingPeekingKeyValueIterator and tests
commit df7e77ac7264aa88dc9091f75ac93f217c38407c
Author: Damian Guy <da...@gmail.com>
Date: 2016-08-26T10:20:18Z
tidy up and tests
commit f5cca5cedcbc11eb918c5f0c41474c228053e53d
Author: Eno Thereska <en...@gmail.com>
Date: 2016-08-26T10:25:02Z
Isolate caches for each store
----
> Unify store and downstream caching in streams
> ---------------------------------------------
>
> Key: KAFKA-3776
> URL: https://issues.apache.org/jira/browse/KAFKA-3776
> Project: Kafka
> Issue Type: New Feature
> Components: streams
> Affects Versions: 0.10.1.0
> Reporter: Eno Thereska
> Assignee: Eno Thereska
> Fix For: 0.10.1.0
>
>
> This is an umbrella story for capturing changes to processor caching in Streams as first described in KIP-63. https://cwiki.apache.org/confluence/display/KAFKA/KIP-63%3A+Unify+store+and+downstream+caching+in+streams
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)