You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Christopher Gustafson <ch...@kth.se> on 2022/02/09 12:34:05 UTC
Changing StateBackend for Statefun Harness E2E tests
Hi everyone,
I am looking into the code of StateFun, trying to understand how it works. I was trying to run the Harness E2E in my IDE, and tried to change the StateBackend to rocksdb, at which point I got an error saying it wasn't found. My first question then becomes, why is this? Shouldn't the regular Flink including the rocksdb state backend be included as a dependency? Since I am a bit confused about the layout, if someone could provide a description of how the StateFun project relates to the Flink codebase, and how one can work effectively with these two in an IDE would be very appreciated.
Thanks,
Christopher
Re: Changing StateBackend for Statefun Harness E2E tests
Posted by Igal Shilman <ig...@gmail.com>.
Hi Christopher,
Great to hear you've solved it, and thanks for sharing your findings with
the community!
Indeed RocksDB is a separate component that has to be added as a
dependency.
On Wed, Feb 9, 2022 at 3:55 PM Christopher Gustafson <ch...@kth.se> wrote:
> Solved it, and posting here in case anyone run into the same issue!
>
>
> Since the Harness uses StreamExecutionEnvironment to set the flink
> configurations, you have to set the state backend explicitly, as described
> here:
>
> https://github.com/apache/flink/blob/master/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBStateBackend.java#L59
>
>
> All the best
> ------------------------------
> *Från:* Christopher Gustafson
> *Skickat:* den 9 februari 2022 13:34:05
> *Till:* user@flink.apache.org
> *Ämne:* Changing StateBackend for Statefun Harness E2E tests
>
>
> Hi everyone,
>
>
> I am looking into the code of StateFun, trying to understand how it works.
> I was trying to run the Harness E2E in my IDE, and tried to change the
> StateBackend to rocksdb, at which point I got an error saying it wasn't
> found. My first question then becomes, why is this? Shouldn't the regular
> Flink including the rocksdb state backend be included as a dependency?
> Since I am a bit confused about the layout, if someone could provide a
> description of how the StateFun project relates to the Flink codebase, and
> how one can work effectively with these two in an IDE would be very
> appreciated.
>
>
> Thanks,
>
> Christopher
>
SV: Changing StateBackend for Statefun Harness E2E tests
Posted by Christopher Gustafson <ch...@kth.se>.
Solved it, and posting here in case anyone run into the same issue!
Since the Harness uses StreamExecutionEnvironment to set the flink configurations, you have to set the state backend explicitly, as described here:
<file:///tmp/checkpoints>https://github.com/apache/flink/blob/master/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBStateBackend.java#L59
All the best
________________________________
Från: Christopher Gustafson
Skickat: den 9 februari 2022 13:34:05
Till: user@flink.apache.org
Ämne: Changing StateBackend for Statefun Harness E2E tests
Hi everyone,
I am looking into the code of StateFun, trying to understand how it works. I was trying to run the Harness E2E in my IDE, and tried to change the StateBackend to rocksdb, at which point I got an error saying it wasn't found. My first question then becomes, why is this? Shouldn't the regular Flink including the rocksdb state backend be included as a dependency? Since I am a bit confused about the layout, if someone could provide a description of how the StateFun project relates to the Flink codebase, and how one can work effectively with these two in an IDE would be very appreciated.
Thanks,
Christopher