You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Jacques Nadeau <ja...@gmail.com> on 2014/11/11 08:06:12 UTC
Review Request 27857: DRILL-1684,
DRILL-1517: Profile and cancellation updates
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27857/
-----------------------------------------------------------
Review request for drill, Jinfeng Ni and Steven Phillips.
Repository: drill-git
Description
-------
- Remove any storage of persisted profiles.
- Store a separate query info object for active queries.
- Update cancellation and running profile loading to query foreman server.
- Make file store support HDFS APIs
- Update PStoreProvider to use configuration to decide if you want PERSISTENT, EPHEMERAL, or BLOB storage rather than separate interfaces.
- Update ZkPStore's persistent mode to leverage a cache and respond to changes rather than actively probing values.
- Update ZkPStore's cache to be effectively write-through.
- Automatically delete deprecated or default value options from PStore.
Diffs
-----
contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/config/HBasePStore.java b5a697c
contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/config/HBasePStoreProvider.java 52808d4
contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseTableProvider.java b1ff44c
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/control/ControlRpcConfig.java 1308c37
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/control/ControlTunnel.java 461cd8a
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataRpcConfig.java 75fa17c
exec/java-exec/src/main/java/org/apache/drill/exec/server/options/FallbackOptionManager.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/server/options/FragmentOptionManager.java 46d316b
exec/java-exec/src/main/java/org/apache/drill/exec/server/options/InMemoryOptionManager.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/server/options/QueryOptionManager.java 82fc5ba
exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SessionOptionManager.java 4268d02
exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java e802b44
exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePluginRegistry.java d6c8ee0
exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/WorkspaceSchemaFactory.java acd8fcb
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/EStore.java 7214092
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/EStoreProvider.java 32bf0b1
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/PStore.java 040a99d
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/PStoreConfig.java 7d5243f
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/PStoreProvider.java f6154e2
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/FilePStore.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/LocalEStoreProvider.java b505fd4
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/LocalPStore.java c10f862
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/LocalPStoreProvider.java c413866
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/MapEStore.java 84e5027
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/NoWriteLocalPStore.java d1ef931
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/zk/ZkAbstractStore.java b88ff74
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/zk/ZkEStoreProvider.java 34b59a7
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/zk/ZkPStore.java 601ba8f
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/zk/ZkPStoreProvider.java aa64f53
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/ControlHandlerImpl.java fc0972f
exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 0979f34
exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/QueryStatus.java 06c8e28
exec/java-exec/src/main/java/org/apache/drill/exec/work/user/UserWorker.java db76057
exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/PStoreTestUtil.java 6f7794b
exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestPStoreProviders.java 18d87c7
protocol/src/main/java/org/apache/drill/exec/proto/BitControl.java a0eba74
protocol/src/main/java/org/apache/drill/exec/proto/SchemaUserBitShared.java f6bd17b
protocol/src/main/java/org/apache/drill/exec/proto/UserBitShared.java 37d0eda
protocol/src/main/java/org/apache/drill/exec/proto/beans/QueryInfo.java PRE-CREATION
protocol/src/main/protobuf/BitControl.proto 1fc5bb3
protocol/src/main/protobuf/UserBitShared.proto fc2e89f
Diff: https://reviews.apache.org/r/27857/diff/
Testing
-------
Thanks,
Jacques Nadeau
Re: Review Request 27857: DRILL-1684,
DRILL-1517: Profile and cancellation updates
Posted by Aditya Kishore <ad...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27857/#review60765
-----------------------------------------------------------
Ship it!
Ship It!
- Aditya Kishore
On Nov. 10, 2014, 11:14 p.m., Jacques Nadeau wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27857/
> -----------------------------------------------------------
>
> (Updated Nov. 10, 2014, 11:14 p.m.)
>
>
> Review request for drill, Jinfeng Ni and Steven Phillips.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> - Remove any storage of persisted profiles.
> - Store a separate query info object for active queries.
> - Update cancellation and running profile loading to query foreman server.
> - Make file store support HDFS APIs
> - Update PStoreProvider to use configuration to decide if you want PERSISTENT, EPHEMERAL, or BLOB storage rather than separate interfaces.
> - Update ZkPStore's persistent mode to leverage a cache and respond to changes rather than actively probing values.
> - Update ZkPStore's cache to be effectively write-through.
> - Automatically delete deprecated or default value options from PStore.
>
>
> Diffs
> -----
>
> contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/config/HBasePStore.java b5a697c
> contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/config/HBasePStoreProvider.java 52808d4
> contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseTableProvider.java b1ff44c
> contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/config/MongoPStore.java f256c98
> contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/config/MongoPStoreProvider.java eb4ba53
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/control/ControlRpcConfig.java 1308c37
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/control/ControlTunnel.java 461cd8a
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataRpcConfig.java 75fa17c
> exec/java-exec/src/main/java/org/apache/drill/exec/server/options/FallbackOptionManager.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/server/options/FragmentOptionManager.java 46d316b
> exec/java-exec/src/main/java/org/apache/drill/exec/server/options/InMemoryOptionManager.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/server/options/QueryOptionManager.java 82fc5ba
> exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SessionOptionManager.java 4268d02
> exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java e802b44
> exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePluginRegistry.java d6c8ee0
> exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/WorkspaceSchemaFactory.java acd8fcb
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/EStore.java 7214092
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/EStoreProvider.java 32bf0b1
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/PStore.java 040a99d
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/PStoreConfig.java 7d5243f
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/PStoreProvider.java f6154e2
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/FilePStore.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/LocalEStoreProvider.java b505fd4
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/LocalPStore.java c10f862
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/LocalPStoreProvider.java c413866
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/MapEStore.java 84e5027
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/NoWriteLocalPStore.java d1ef931
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/zk/ZkAbstractStore.java b88ff74
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/zk/ZkEStoreProvider.java 34b59a7
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/zk/ZkPStore.java 601ba8f
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/zk/ZkPStoreProvider.java aa64f53
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/ControlHandlerImpl.java fc0972f
> exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 0979f34
> exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/QueryStatus.java 06c8e28
> exec/java-exec/src/main/java/org/apache/drill/exec/work/user/UserWorker.java db76057
> exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/PStoreTestUtil.java 6f7794b
> exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestPStoreProviders.java 18d87c7
> protocol/src/main/java/org/apache/drill/exec/proto/BitControl.java a0eba74
> protocol/src/main/java/org/apache/drill/exec/proto/SchemaUserBitShared.java f6bd17b
> protocol/src/main/java/org/apache/drill/exec/proto/UserBitShared.java 37d0eda
> protocol/src/main/java/org/apache/drill/exec/proto/beans/QueryInfo.java PRE-CREATION
> protocol/src/main/protobuf/BitControl.proto 1fc5bb3
> protocol/src/main/protobuf/UserBitShared.proto fc2e89f
>
> Diff: https://reviews.apache.org/r/27857/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jacques Nadeau
>
>
Re: Review Request 27857: DRILL-1684,
DRILL-1517: Profile and cancellation updates
Posted by Jinfeng Ni <jn...@maprtech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27857/#review60840
-----------------------------------------------------------
Ship it!
Ship It!
- Jinfeng Ni
On Nov. 10, 2014, 11:14 p.m., Jacques Nadeau wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27857/
> -----------------------------------------------------------
>
> (Updated Nov. 10, 2014, 11:14 p.m.)
>
>
> Review request for drill, Jinfeng Ni and Steven Phillips.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> - Remove any storage of persisted profiles.
> - Store a separate query info object for active queries.
> - Update cancellation and running profile loading to query foreman server.
> - Make file store support HDFS APIs
> - Update PStoreProvider to use configuration to decide if you want PERSISTENT, EPHEMERAL, or BLOB storage rather than separate interfaces.
> - Update ZkPStore's persistent mode to leverage a cache and respond to changes rather than actively probing values.
> - Update ZkPStore's cache to be effectively write-through.
> - Automatically delete deprecated or default value options from PStore.
>
>
> Diffs
> -----
>
> contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/config/HBasePStore.java b5a697c
> contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/config/HBasePStoreProvider.java 52808d4
> contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseTableProvider.java b1ff44c
> contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/config/MongoPStore.java f256c98
> contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/config/MongoPStoreProvider.java eb4ba53
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/control/ControlRpcConfig.java 1308c37
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/control/ControlTunnel.java 461cd8a
> exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataRpcConfig.java 75fa17c
> exec/java-exec/src/main/java/org/apache/drill/exec/server/options/FallbackOptionManager.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/server/options/FragmentOptionManager.java 46d316b
> exec/java-exec/src/main/java/org/apache/drill/exec/server/options/InMemoryOptionManager.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/server/options/QueryOptionManager.java 82fc5ba
> exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SessionOptionManager.java 4268d02
> exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java e802b44
> exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePluginRegistry.java d6c8ee0
> exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/WorkspaceSchemaFactory.java acd8fcb
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/EStore.java 7214092
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/EStoreProvider.java 32bf0b1
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/PStore.java 040a99d
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/PStoreConfig.java 7d5243f
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/PStoreProvider.java f6154e2
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/FilePStore.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/LocalEStoreProvider.java b505fd4
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/LocalPStore.java c10f862
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/LocalPStoreProvider.java c413866
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/MapEStore.java 84e5027
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/NoWriteLocalPStore.java d1ef931
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/zk/ZkAbstractStore.java b88ff74
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/zk/ZkEStoreProvider.java 34b59a7
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/zk/ZkPStore.java 601ba8f
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/zk/ZkPStoreProvider.java aa64f53
> exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/ControlHandlerImpl.java fc0972f
> exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 0979f34
> exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/QueryStatus.java 06c8e28
> exec/java-exec/src/main/java/org/apache/drill/exec/work/user/UserWorker.java db76057
> exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/PStoreTestUtil.java 6f7794b
> exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestPStoreProviders.java 18d87c7
> protocol/src/main/java/org/apache/drill/exec/proto/BitControl.java a0eba74
> protocol/src/main/java/org/apache/drill/exec/proto/SchemaUserBitShared.java f6bd17b
> protocol/src/main/java/org/apache/drill/exec/proto/UserBitShared.java 37d0eda
> protocol/src/main/java/org/apache/drill/exec/proto/beans/QueryInfo.java PRE-CREATION
> protocol/src/main/protobuf/BitControl.proto 1fc5bb3
> protocol/src/main/protobuf/UserBitShared.proto fc2e89f
>
> Diff: https://reviews.apache.org/r/27857/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jacques Nadeau
>
>
Re: Review Request 27857: DRILL-1684,
DRILL-1517: Profile and cancellation updates
Posted by Jacques Nadeau <ja...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27857/
-----------------------------------------------------------
(Updated Nov. 11, 2014, 7:14 a.m.)
Review request for drill, Jinfeng Ni and Steven Phillips.
Changes
-------
Added MongoDB PStore changes.
Repository: drill-git
Description
-------
- Remove any storage of persisted profiles.
- Store a separate query info object for active queries.
- Update cancellation and running profile loading to query foreman server.
- Make file store support HDFS APIs
- Update PStoreProvider to use configuration to decide if you want PERSISTENT, EPHEMERAL, or BLOB storage rather than separate interfaces.
- Update ZkPStore's persistent mode to leverage a cache and respond to changes rather than actively probing values.
- Update ZkPStore's cache to be effectively write-through.
- Automatically delete deprecated or default value options from PStore.
Diffs (updated)
-----
contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/config/HBasePStore.java b5a697c
contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/config/HBasePStoreProvider.java 52808d4
contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseTableProvider.java b1ff44c
contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/config/MongoPStore.java f256c98
contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/config/MongoPStoreProvider.java eb4ba53
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/control/ControlRpcConfig.java 1308c37
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/control/ControlTunnel.java 461cd8a
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataRpcConfig.java 75fa17c
exec/java-exec/src/main/java/org/apache/drill/exec/server/options/FallbackOptionManager.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/server/options/FragmentOptionManager.java 46d316b
exec/java-exec/src/main/java/org/apache/drill/exec/server/options/InMemoryOptionManager.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/server/options/QueryOptionManager.java 82fc5ba
exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SessionOptionManager.java 4268d02
exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java e802b44
exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePluginRegistry.java d6c8ee0
exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/WorkspaceSchemaFactory.java acd8fcb
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/EStore.java 7214092
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/EStoreProvider.java 32bf0b1
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/PStore.java 040a99d
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/PStoreConfig.java 7d5243f
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/PStoreProvider.java f6154e2
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/FilePStore.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/LocalEStoreProvider.java b505fd4
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/LocalPStore.java c10f862
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/LocalPStoreProvider.java c413866
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/MapEStore.java 84e5027
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/NoWriteLocalPStore.java d1ef931
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/zk/ZkAbstractStore.java b88ff74
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/zk/ZkEStoreProvider.java 34b59a7
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/zk/ZkPStore.java 601ba8f
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/zk/ZkPStoreProvider.java aa64f53
exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/ControlHandlerImpl.java fc0972f
exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 0979f34
exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/QueryStatus.java 06c8e28
exec/java-exec/src/main/java/org/apache/drill/exec/work/user/UserWorker.java db76057
exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/PStoreTestUtil.java 6f7794b
exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestPStoreProviders.java 18d87c7
protocol/src/main/java/org/apache/drill/exec/proto/BitControl.java a0eba74
protocol/src/main/java/org/apache/drill/exec/proto/SchemaUserBitShared.java f6bd17b
protocol/src/main/java/org/apache/drill/exec/proto/UserBitShared.java 37d0eda
protocol/src/main/java/org/apache/drill/exec/proto/beans/QueryInfo.java PRE-CREATION
protocol/src/main/protobuf/BitControl.proto 1fc5bb3
protocol/src/main/protobuf/UserBitShared.proto fc2e89f
Diff: https://reviews.apache.org/r/27857/diff/
Testing
-------
Thanks,
Jacques Nadeau