You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Sudheesh Katkam <sk...@maprtech.com> on 2015/03/11 18:04:59 UTC
Review Request 31938: DRILL-2275: Need implementations of sys tables
for drill memory and threads profiles
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31938/
-----------------------------------------------------------
Review request for drill and Venki Korukanti.
Repository: drill-git
Description
-------
DRILL-2275: Added support to allow for querying cluster state information.
+ If table isDistributed(), BatchCreator and SystemTableScan allow for a distributed query.
+ SystemRecordReader reads SystemRecords.
+ There is now a generic data type for static tables.
+ GroupScan can enforce width to be maximum width on ExcessiveExchangeRemover.
+ GroupScan has minimum width for SimpleParallelizer.
Diffs
-----
common/src/main/java/org/apache/drill/common/JSONOptions.java 64e6d52
exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScan.java 276ecb5
exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/GroupScan.java 23860a3
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/ParallelizationInfo.java 75a009e
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java f8d1803
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Stats.java e61b38f
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/StatsCollector.java 1f56556
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/ExcessiveExchangeIdentifier.java a237014
exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java c1e64e6
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/StaticDrillTable.java c1e8dd1
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecordReader.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecords.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTable.java 0bf2156
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java a1bec1e
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePlugin.java 2c70fd4
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePluginConfig.java 93fe68e
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableScan.java cdd0d18
exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java c1803bc
Diff: https://reviews.apache.org/r/31938/diff/
Testing
-------
Tested on 3-node cluster and in embedded mode.
```
> select * from sys.memory;
+------------+--------------+------------+---------------------+
| host_name | total_memory | heap_size | direct_alloc_memory |
+------------+--------------+------------+---------------------+
| perfnode206.perf.lab:31010 | 1073741824 | 395823432 | 5000000 |
| perfnode208.perf.lab:31010 | 1073741824 | 337127496 | 2000000 |
| perfnode207.perf.lab:31010 | 1073741824 | 289272760 | 2000000 |
+------------+--------------+------------+---------------------+
```
Thanks,
Sudheesh Katkam
Re: Review Request 31938: DRILL-2275: Need implementations of sys
tables for drill memory and threads profiles
Posted by Sudheesh Katkam <sk...@maprtech.com>.
> On March 11, 2015, 10:22 p.m., Chris Westin wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecords.java, line 48
> > <https://reviews.apache.org/r/31938/diff/1/?file=891400#file891400line48>
> >
> > This, and all the nested static classes should be separate classes in this package. Each of them can have a private constructor, with a public final INSTANCE variable, similar to what you've already done.
I am just curious here.
The rationale was to have the collection of implementations of SystemRecord in one place. And access to the singleton of each implementation is only through SystemRecords e.g. SystemRecords.getMemoryRecord() and not through individual classes e.g. MemoryRecord.getInstance(). Is there a problem with this approach?
> On March 11, 2015, 10:22 p.m., Chris Westin wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java, line 62
> > <https://reviews.apache.org/r/31938/diff/1/?file=891402#file891402line62>
> >
> > Could this be a method on the SystemTable enum instead?
It's possible but again, I am curious here.
I thought it would be awkward to have:
switch(this) {
case OPTION:
...
}
(once for local tables and once for distributed tables)
> On March 11, 2015, 10:22 p.m., Chris Westin wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java, line 77
> > <https://reviews.apache.org/r/31938/diff/1/?file=891402#file891402line77>
> >
> > Could this be a method on the SystemTable enum instead?
^ Same as above.
On March 11, 2015, 10:22 p.m., Sudheesh Katkam wrote:
> > Does the sys.drillbits table list the host and port separately? If so, these should do so as well. If I'm misremembering that, ignore this.
It does.
Should I remove the port altogether?
Yes) This way users can join with sys.drillbits table.
No) Then, should I list user port, control port and data port?
- Sudheesh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31938/#review76138
-----------------------------------------------------------
On March 11, 2015, 5:04 p.m., Sudheesh Katkam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31938/
> -----------------------------------------------------------
>
> (Updated March 11, 2015, 5:04 p.m.)
>
>
> Review request for drill and Venki Korukanti.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> DRILL-2275: Added support to allow for querying cluster state information.
> + If table isDistributed(), BatchCreator and SystemTableScan allow for a distributed query.
> + SystemRecordReader reads SystemRecords.
> + There is now a generic data type for static tables.
> + GroupScan can enforce width to be maximum width on ExcessiveExchangeRemover.
> + GroupScan has minimum width for SimpleParallelizer.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/drill/common/JSONOptions.java 64e6d52
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScan.java 276ecb5
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/GroupScan.java 23860a3
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/ParallelizationInfo.java 75a009e
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java f8d1803
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Stats.java e61b38f
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/StatsCollector.java 1f56556
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/ExcessiveExchangeIdentifier.java a237014
> exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java c1e64e6
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/StaticDrillTable.java c1e8dd1
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecordReader.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecords.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTable.java 0bf2156
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java a1bec1e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePlugin.java 2c70fd4
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePluginConfig.java 93fe68e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableScan.java cdd0d18
> exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java c1803bc
>
> Diff: https://reviews.apache.org/r/31938/diff/
>
>
> Testing
> -------
>
> Tested on 3-node cluster and in embedded mode.
> ```
> > select * from sys.memory;
> +------------+--------------+------------+---------------------+
> | host_name | total_memory | heap_size | direct_alloc_memory |
> +------------+--------------+------------+---------------------+
> | perfnode206.perf.lab:31010 | 1073741824 | 395823432 | 5000000 |
> | perfnode208.perf.lab:31010 | 1073741824 | 337127496 | 2000000 |
> | perfnode207.perf.lab:31010 | 1073741824 | 289272760 | 2000000 |
> +------------+--------------+------------+---------------------+
> ```
>
>
> Thanks,
>
> Sudheesh Katkam
>
>
Re: Review Request 31938: DRILL-2275: Need implementations of sys
tables for drill memory and threads profiles
Posted by Chris Westin <ch...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31938/#review76138
-----------------------------------------------------------
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/ParallelizationInfo.java
<https://reviews.apache.org/r/31938/#comment123594>
You shouldn't need the this. here.
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java
<https://reviews.apache.org/r/31938/#comment123595>
Excellent. Thank you for improving that error message.
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/StatsCollector.java
<https://reviews.apache.org/r/31938/#comment123597>
add a
final Stats stats = wrapper.getStats();
then reuse it
stats.addMaxWidth(...);
stats.addMinWidth(...);
exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java
<https://reviews.apache.org/r/31938/#comment123598>
Can these locals be final?
exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java
<https://reviews.apache.org/r/31938/#comment123599>
Good code improvement.
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/StaticDrillTable.java
<https://reviews.apache.org/r/31938/#comment123600>
I like private loggers.
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecords.java
<https://reviews.apache.org/r/31938/#comment123607>
This, and all the nested static classes should be separate classes in this package. Each of them can have a private constructor, with a public final INSTANCE variable, similar to what you've already done.
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecords.java
<https://reviews.apache.org/r/31938/#comment123602>
final
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecords.java
<https://reviews.apache.org/r/31938/#comment123604>
final
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecords.java
<https://reviews.apache.org/r/31938/#comment123605>
final
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecords.java
<https://reviews.apache.org/r/31938/#comment123606>
get the endpoint once, and reuse it. then, get the mutators once each, and reuse them. The locals should be final.
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java
<https://reviews.apache.org/r/31938/#comment123608>
can this be final?
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java
<https://reviews.apache.org/r/31938/#comment123611>
Could this be a method on the SystemTable enum instead?
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java
<https://reviews.apache.org/r/31938/#comment123610>
can these locals be final? Also, can we keep them private to this case by using braces for the case block?
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java
<https://reviews.apache.org/r/31938/#comment123612>
Could this be a method on the SystemTable enum instead?
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePluginConfig.java
<https://reviews.apache.org/r/31938/#comment123615>
Good catch that this was missing for the singleton.
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableScan.java
<https://reviews.apache.org/r/31938/#comment123616>
final
Does the sys.drillbits table list the host and port separately? If so, these should do so as well. If I'm misremembering that, ignore this.
- Chris Westin
On March 11, 2015, 10:04 a.m., Sudheesh Katkam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31938/
> -----------------------------------------------------------
>
> (Updated March 11, 2015, 10:04 a.m.)
>
>
> Review request for drill and Venki Korukanti.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> DRILL-2275: Added support to allow for querying cluster state information.
> + If table isDistributed(), BatchCreator and SystemTableScan allow for a distributed query.
> + SystemRecordReader reads SystemRecords.
> + There is now a generic data type for static tables.
> + GroupScan can enforce width to be maximum width on ExcessiveExchangeRemover.
> + GroupScan has minimum width for SimpleParallelizer.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/drill/common/JSONOptions.java 64e6d52
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScan.java 276ecb5
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/GroupScan.java 23860a3
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/ParallelizationInfo.java 75a009e
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java f8d1803
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Stats.java e61b38f
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/StatsCollector.java 1f56556
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/ExcessiveExchangeIdentifier.java a237014
> exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java c1e64e6
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/StaticDrillTable.java c1e8dd1
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecordReader.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecords.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTable.java 0bf2156
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java a1bec1e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePlugin.java 2c70fd4
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePluginConfig.java 93fe68e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableScan.java cdd0d18
> exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java c1803bc
>
> Diff: https://reviews.apache.org/r/31938/diff/
>
>
> Testing
> -------
>
> Tested on 3-node cluster and in embedded mode.
> ```
> > select * from sys.memory;
> +------------+--------------+------------+---------------------+
> | host_name | total_memory | heap_size | direct_alloc_memory |
> +------------+--------------+------------+---------------------+
> | perfnode206.perf.lab:31010 | 1073741824 | 395823432 | 5000000 |
> | perfnode208.perf.lab:31010 | 1073741824 | 337127496 | 2000000 |
> | perfnode207.perf.lab:31010 | 1073741824 | 289272760 | 2000000 |
> +------------+--------------+------------+---------------------+
> ```
>
>
> Thanks,
>
> Sudheesh Katkam
>
>
Re: Review Request 31938: DRILL-2275: Need implementations of sys
tables for drill memory and threads profiles
Posted by Chris Westin <ch...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31938/#review76768
-----------------------------------------------------------
Ship it!
Ship It!
- Chris Westin
On March 17, 2015, 8:51 a.m., Sudheesh Katkam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31938/
> -----------------------------------------------------------
>
> (Updated March 17, 2015, 8:51 a.m.)
>
>
> Review request for drill and Venki Korukanti.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> DRILL-2275: Added support to allow for querying cluster state information.
> + If table isDistributed(), BatchCreator and SystemTableScan allow for a distributed query.
> + SystemRecordReader reads SystemRecords.
> + There is now a generic data type for static tables.
> + GroupScan can enforce width to be maximum width on ExcessiveExchangeRemover.
> + GroupScan has minimum width for SimpleParallelizer.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/drill/common/JSONOptions.java 64e6d52
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScan.java 276ecb5
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/GroupScan.java 23860a3
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/ParallelizationInfo.java 75a009e
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java f8d1803
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Stats.java e61b38f
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/StatsCollector.java 1f56556
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/ExcessiveExchangeIdentifier.java a237014
> exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java c1e64e6
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryRecord.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/StaticDrillTable.java c1e8dd1
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecord.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecordReader.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTable.java 0bf2156
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java a1bec1e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePlugin.java 2c70fd4
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePluginConfig.java 93fe68e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableScan.java cdd0d18
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/ThreadsRecord.java PRE-CREATION
> exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java c1803bc
>
> Diff: https://reviews.apache.org/r/31938/diff/
>
>
> Testing
> -------
>
> Tested on 3-node cluster and in embedded mode.
> ```
> > select * from sys.memory;
> +------------+--------------+------------+---------------------+
> | host_name | user_port | total_memory | heap_size | direct_alloc_memory |
> +------------+--------------+------------+---------------------+
> | perfnode206.perf.lab | 31010 | 1073741824 | 395823432 | 5000000 |
> | perfnode208.perf.lab | 31010 | 1073741824 | 337127496 | 2000000 |
> | perfnode207.perf.lab | 31010 | 1073741824 | 289272760 | 2000000 |
> +------------+--------------+------------+---------------------+
> ```
>
>
> Thanks,
>
> Sudheesh Katkam
>
>
Re: Review Request 31938: DRILL-2275: Need implementations of sys
tables for drill memory and threads profiles
Posted by Chris Westin <ch...@gmail.com>.
> On March 19, 2015, 3:16 p.m., Jacques Nadeau wrote:
> > Can we use the following for column anmes:
> >
> > hostname
> > user_port
> > heap_current
> > heap_max
> > direct_current
> > direct_max
Doesn't sys.drillbits use host_name? If so, we should be consistent with that.
- Chris
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31938/#review77119
-----------------------------------------------------------
On March 19, 2015, 1:48 p.m., Sudheesh Katkam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31938/
> -----------------------------------------------------------
>
> (Updated March 19, 2015, 1:48 p.m.)
>
>
> Review request for drill and Venki Korukanti.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> DRILL-2275: Added support to allow for querying cluster state information.
> + If table isDistributed(), BatchCreator and SystemTableScan allow for a distributed query.
> + SystemRecordReader reads SystemRecords.
> + There is now a generic data type for static tables.
> + GroupScan can enforce width to be maximum width on ExcessiveExchangeRemover.
> + GroupScan has minimum width for SimpleParallelizer.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/drill/common/JSONOptions.java 64e6d52
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScan.java 276ecb5
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/GroupScan.java 23860a3
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/ParallelizationInfo.java 75a009e
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java f8d1803
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Stats.java e61b38f
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/StatsCollector.java 1f56556
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/ExcessiveExchangeIdentifier.java a237014
> exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java c1e64e6
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/DrillbitIterator.java 67d7cf9
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryRecord.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/StaticDrillTable.java c1e8dd1
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecord.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecordReader.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTable.java 0bf2156
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java a1bec1e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePlugin.java 2c70fd4
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePluginConfig.java 93fe68e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableScan.java cdd0d18
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/ThreadsRecord.java PRE-CREATION
> exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java c1803bc
>
> Diff: https://reviews.apache.org/r/31938/diff/
>
>
> Testing
> -------
>
> Tested on 3-node cluster and in embedded mode.
> ```
> > select * from sys.memory;
> +------------+------------+-----------------+-------------------+---------------------+
> | host_name | user_port | total_heap_size | current_heap_size | direct_alloc_memory |
> +------------+------------+-----------------+-------------------+---------------------+
> | perfnode208.perf.lab | 31010 | 4225236992 | 483863736 | 5000147 |
> | perfnode207.perf.lab | 31010 | 4225236992 | 341408256 | 2000000 |
> | perfnode206.perf.lab | 31010 | 4225236992 | 311670104 | 2000000 |
> +------------+------------+-----------------+-------------------+---------------------+
> ```
>
>
> Thanks,
>
> Sudheesh Katkam
>
>
Re: Review Request 31938: DRILL-2275: Need implementations of sys
tables for drill memory and threads profiles
Posted by Sudheesh Katkam <sk...@maprtech.com>.
> On March 19, 2015, 10:16 p.m., Jacques Nadeau wrote:
> > Can we use the following for column anmes:
> >
> > hostname
> > user_port
> > heap_current
> > heap_max
> > direct_current
> > direct_max
>
> Chris Westin wrote:
> Doesn't sys.drillbits use host_name? If so, we should be consistent with that.
It used 'host'
I changed it to 'host_name'
I'll change both to 'hostname'
- Sudheesh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31938/#review77119
-----------------------------------------------------------
On March 19, 2015, 8:48 p.m., Sudheesh Katkam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31938/
> -----------------------------------------------------------
>
> (Updated March 19, 2015, 8:48 p.m.)
>
>
> Review request for drill and Venki Korukanti.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> DRILL-2275: Added support to allow for querying cluster state information.
> + If table isDistributed(), BatchCreator and SystemTableScan allow for a distributed query.
> + SystemRecordReader reads SystemRecords.
> + There is now a generic data type for static tables.
> + GroupScan can enforce width to be maximum width on ExcessiveExchangeRemover.
> + GroupScan has minimum width for SimpleParallelizer.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/drill/common/JSONOptions.java 64e6d52
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScan.java 276ecb5
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/GroupScan.java 23860a3
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/ParallelizationInfo.java 75a009e
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java f8d1803
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Stats.java e61b38f
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/StatsCollector.java 1f56556
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/ExcessiveExchangeIdentifier.java a237014
> exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java c1e64e6
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/DrillbitIterator.java 67d7cf9
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryRecord.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/StaticDrillTable.java c1e8dd1
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecord.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecordReader.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTable.java 0bf2156
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java a1bec1e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePlugin.java 2c70fd4
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePluginConfig.java 93fe68e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableScan.java cdd0d18
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/ThreadsRecord.java PRE-CREATION
> exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java c1803bc
>
> Diff: https://reviews.apache.org/r/31938/diff/
>
>
> Testing
> -------
>
> Tested on 3-node cluster and in embedded mode.
> ```
> > select * from sys.memory;
> +------------+------------+-----------------+-------------------+---------------------+
> | host_name | user_port | total_heap_size | current_heap_size | direct_alloc_memory |
> +------------+------------+-----------------+-------------------+---------------------+
> | perfnode208.perf.lab | 31010 | 4225236992 | 483863736 | 5000147 |
> | perfnode207.perf.lab | 31010 | 4225236992 | 341408256 | 2000000 |
> | perfnode206.perf.lab | 31010 | 4225236992 | 311670104 | 2000000 |
> +------------+------------+-----------------+-------------------+---------------------+
> ```
>
>
> Thanks,
>
> Sudheesh Katkam
>
>
Re: Review Request 31938: DRILL-2275: Need implementations of sys
tables for drill memory and threads profiles
Posted by Jacques Nadeau <ja...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31938/#review77119
-----------------------------------------------------------
Can we use the following for column anmes:
hostname
user_port
heap_current
heap_max
direct_current
direct_max
- Jacques Nadeau
On March 19, 2015, 8:48 p.m., Sudheesh Katkam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31938/
> -----------------------------------------------------------
>
> (Updated March 19, 2015, 8:48 p.m.)
>
>
> Review request for drill and Venki Korukanti.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> DRILL-2275: Added support to allow for querying cluster state information.
> + If table isDistributed(), BatchCreator and SystemTableScan allow for a distributed query.
> + SystemRecordReader reads SystemRecords.
> + There is now a generic data type for static tables.
> + GroupScan can enforce width to be maximum width on ExcessiveExchangeRemover.
> + GroupScan has minimum width for SimpleParallelizer.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/drill/common/JSONOptions.java 64e6d52
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScan.java 276ecb5
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/GroupScan.java 23860a3
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/ParallelizationInfo.java 75a009e
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java f8d1803
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Stats.java e61b38f
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/StatsCollector.java 1f56556
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/ExcessiveExchangeIdentifier.java a237014
> exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java c1e64e6
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/DrillbitIterator.java 67d7cf9
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryRecord.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/StaticDrillTable.java c1e8dd1
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecord.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecordReader.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTable.java 0bf2156
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java a1bec1e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePlugin.java 2c70fd4
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePluginConfig.java 93fe68e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableScan.java cdd0d18
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/ThreadsRecord.java PRE-CREATION
> exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java c1803bc
>
> Diff: https://reviews.apache.org/r/31938/diff/
>
>
> Testing
> -------
>
> Tested on 3-node cluster and in embedded mode.
> ```
> > select * from sys.memory;
> +------------+------------+-----------------+-------------------+---------------------+
> | host_name | user_port | total_heap_size | current_heap_size | direct_alloc_memory |
> +------------+------------+-----------------+-------------------+---------------------+
> | perfnode208.perf.lab | 31010 | 4225236992 | 483863736 | 5000147 |
> | perfnode207.perf.lab | 31010 | 4225236992 | 341408256 | 2000000 |
> | perfnode206.perf.lab | 31010 | 4225236992 | 311670104 | 2000000 |
> +------------+------------+-----------------+-------------------+---------------------+
> ```
>
>
> Thanks,
>
> Sudheesh Katkam
>
>
Re: Review Request 31938: DRILL-2275: Need implementations of sys
tables for drill memory and threads profiles
Posted by Sudheesh Katkam <sk...@maprtech.com>.
> On March 19, 2015, 9:27 p.m., Chris Westin wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryRecord.java, line 108
> > <https://reviews.apache.org/r/31938/diff/4-5/?file=897740#file897740line108>
> >
> > I thought we were going to use "max_heap_size" since that's what it is?
My bad '-_-
- Sudheesh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31938/#review77111
-----------------------------------------------------------
On March 19, 2015, 8:48 p.m., Sudheesh Katkam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31938/
> -----------------------------------------------------------
>
> (Updated March 19, 2015, 8:48 p.m.)
>
>
> Review request for drill and Venki Korukanti.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> DRILL-2275: Added support to allow for querying cluster state information.
> + If table isDistributed(), BatchCreator and SystemTableScan allow for a distributed query.
> + SystemRecordReader reads SystemRecords.
> + There is now a generic data type for static tables.
> + GroupScan can enforce width to be maximum width on ExcessiveExchangeRemover.
> + GroupScan has minimum width for SimpleParallelizer.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/drill/common/JSONOptions.java 64e6d52
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScan.java 276ecb5
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/GroupScan.java 23860a3
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/ParallelizationInfo.java 75a009e
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java f8d1803
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Stats.java e61b38f
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/StatsCollector.java 1f56556
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/ExcessiveExchangeIdentifier.java a237014
> exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java c1e64e6
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/DrillbitIterator.java 67d7cf9
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryRecord.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/StaticDrillTable.java c1e8dd1
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecord.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecordReader.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTable.java 0bf2156
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java a1bec1e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePlugin.java 2c70fd4
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePluginConfig.java 93fe68e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableScan.java cdd0d18
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/ThreadsRecord.java PRE-CREATION
> exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java c1803bc
>
> Diff: https://reviews.apache.org/r/31938/diff/
>
>
> Testing
> -------
>
> Tested on 3-node cluster and in embedded mode.
> ```
> > select * from sys.memory;
> +------------+------------+-----------------+-------------------+---------------------+
> | host_name | user_port | total_heap_size | current_heap_size | direct_alloc_memory |
> +------------+------------+-----------------+-------------------+---------------------+
> | perfnode208.perf.lab | 31010 | 4225236992 | 483863736 | 5000147 |
> | perfnode207.perf.lab | 31010 | 4225236992 | 341408256 | 2000000 |
> | perfnode206.perf.lab | 31010 | 4225236992 | 311670104 | 2000000 |
> +------------+------------+-----------------+-------------------+---------------------+
> ```
>
>
> Thanks,
>
> Sudheesh Katkam
>
>
Re: Review Request 31938: DRILL-2275: Need implementations of sys
tables for drill memory and threads profiles
Posted by Chris Westin <ch...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31938/#review77111
-----------------------------------------------------------
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryRecord.java
<https://reviews.apache.org/r/31938/#comment125004>
I thought we were going to use "max_heap_size" since that's what it is?
- Chris Westin
On March 19, 2015, 1:48 p.m., Sudheesh Katkam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31938/
> -----------------------------------------------------------
>
> (Updated March 19, 2015, 1:48 p.m.)
>
>
> Review request for drill and Venki Korukanti.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> DRILL-2275: Added support to allow for querying cluster state information.
> + If table isDistributed(), BatchCreator and SystemTableScan allow for a distributed query.
> + SystemRecordReader reads SystemRecords.
> + There is now a generic data type for static tables.
> + GroupScan can enforce width to be maximum width on ExcessiveExchangeRemover.
> + GroupScan has minimum width for SimpleParallelizer.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/drill/common/JSONOptions.java 64e6d52
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScan.java 276ecb5
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/GroupScan.java 23860a3
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/ParallelizationInfo.java 75a009e
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java f8d1803
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Stats.java e61b38f
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/StatsCollector.java 1f56556
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/ExcessiveExchangeIdentifier.java a237014
> exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java c1e64e6
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/DrillbitIterator.java 67d7cf9
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryRecord.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/StaticDrillTable.java c1e8dd1
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecord.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecordReader.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTable.java 0bf2156
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java a1bec1e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePlugin.java 2c70fd4
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePluginConfig.java 93fe68e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableScan.java cdd0d18
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/ThreadsRecord.java PRE-CREATION
> exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java c1803bc
>
> Diff: https://reviews.apache.org/r/31938/diff/
>
>
> Testing
> -------
>
> Tested on 3-node cluster and in embedded mode.
> ```
> > select * from sys.memory;
> +------------+------------+-----------------+-------------------+---------------------+
> | host_name | user_port | total_heap_size | current_heap_size | direct_alloc_memory |
> +------------+------------+-----------------+-------------------+---------------------+
> | perfnode208.perf.lab | 31010 | 4225236992 | 483863736 | 5000147 |
> | perfnode207.perf.lab | 31010 | 4225236992 | 341408256 | 2000000 |
> | perfnode206.perf.lab | 31010 | 4225236992 | 311670104 | 2000000 |
> +------------+------------+-----------------+-------------------+---------------------+
> ```
>
>
> Thanks,
>
> Sudheesh Katkam
>
>
Re: Review Request 31938: DRILL-2275: Need implementations of sys
tables for drill memory and threads profiles
Posted by Chris Westin <ch...@gmail.com>.
> On March 19, 2015, 6:28 p.m., Chris Westin wrote:
> > exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java, line 48
> > <https://reviews.apache.org/r/31938/diff/5-6/?file=900471#file900471line48>
> >
> > Shouldn't this be a TODO? Of course, inquiring minds what to know "enhance it how?" If you're going to put in a comment like this (which I think is great), you need to be more specific.
>
> Sudheesh Katkam wrote:
> This is actually meant for me. I should've mentioned my name.
But you might not be the one to get to it. Another option is to file a JIRA, and then use // TODO(DRILL-<jira number>) <optional short description here>
- Chris
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31938/#review77148
-----------------------------------------------------------
On March 19, 2015, 5:02 p.m., Sudheesh Katkam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31938/
> -----------------------------------------------------------
>
> (Updated March 19, 2015, 5:02 p.m.)
>
>
> Review request for drill and Venki Korukanti.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> DRILL-2275: Added support to allow for querying cluster state information.
> + If table isDistributed(), BatchCreator and SystemTableScan allow for a distributed query.
> + SystemRecordReader reads SystemRecords.
> + There is now a generic data type for static tables.
> + GroupScan can enforce width to be maximum width on ExcessiveExchangeRemover.
> + GroupScan has minimum width for SimpleParallelizer.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/drill/common/JSONOptions.java 64e6d52
> exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java af8c1dc
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScan.java 276ecb5
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/GroupScan.java 23860a3
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/ParallelizationInfo.java 75a009e
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java 12043ce
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Stats.java e61b38f
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/StatsCollector.java 1f56556
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/ExcessiveExchangeIdentifier.java a237014
> exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java c1e64e6
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/DrillbitIterator.java 67d7cf9
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryRecord.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/StaticDrillTable.java c1e8dd1
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecord.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecordReader.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTable.java 0bf2156
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java a1bec1e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePlugin.java 2c70fd4
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePluginConfig.java 93fe68e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableScan.java cdd0d18
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/ThreadsRecord.java PRE-CREATION
> exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java c1803bc
>
> Diff: https://reviews.apache.org/r/31938/diff/
>
>
> Testing
> -------
>
> Tested on 3-node cluster and in embedded mode.
> ```
> > select * from sys.memory;
> +------------+------------+--------------+------------+----------------+------------+
> | hostname | user_port | heap_current | heap_max | direct_current | direct_max |
> +------------+------------+--------------+------------+----------------+------------+
> | perfnode206.perf.lab | 31010 | 375599272 | 4225236992 | 6048576 | 8589934592 |
> | perfnode207.perf.lab | 31010 | 275524512 | 4225236992 | 2000000 | 8589934592 |
> | perfnode208.perf.lab | 31010 | 289288416 | 4225236992 | 2000000 | 8589934592 |
> +------------+------------+--------------+------------+----------------+------------+
> ```
>
>
> Thanks,
>
> Sudheesh Katkam
>
>
Re: Review Request 31938: DRILL-2275: Need implementations of sys
tables for drill memory and threads profiles
Posted by Sudheesh Katkam <sk...@maprtech.com>.
> On March 20, 2015, 1:28 a.m., Chris Westin wrote:
> > exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java, line 48
> > <https://reviews.apache.org/r/31938/diff/5-6/?file=900471#file900471line48>
> >
> > Shouldn't this be a TODO? Of course, inquiring minds what to know "enhance it how?" If you're going to put in a comment like this (which I think is great), you need to be more specific.
This is actually meant for me. I should've mentioned my name.
- Sudheesh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31938/#review77148
-----------------------------------------------------------
On March 20, 2015, 12:02 a.m., Sudheesh Katkam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31938/
> -----------------------------------------------------------
>
> (Updated March 20, 2015, 12:02 a.m.)
>
>
> Review request for drill and Venki Korukanti.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> DRILL-2275: Added support to allow for querying cluster state information.
> + If table isDistributed(), BatchCreator and SystemTableScan allow for a distributed query.
> + SystemRecordReader reads SystemRecords.
> + There is now a generic data type for static tables.
> + GroupScan can enforce width to be maximum width on ExcessiveExchangeRemover.
> + GroupScan has minimum width for SimpleParallelizer.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/drill/common/JSONOptions.java 64e6d52
> exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java af8c1dc
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScan.java 276ecb5
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/GroupScan.java 23860a3
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/ParallelizationInfo.java 75a009e
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java 12043ce
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Stats.java e61b38f
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/StatsCollector.java 1f56556
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/ExcessiveExchangeIdentifier.java a237014
> exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java c1e64e6
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/DrillbitIterator.java 67d7cf9
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryRecord.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/StaticDrillTable.java c1e8dd1
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecord.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecordReader.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTable.java 0bf2156
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java a1bec1e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePlugin.java 2c70fd4
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePluginConfig.java 93fe68e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableScan.java cdd0d18
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/ThreadsRecord.java PRE-CREATION
> exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java c1803bc
>
> Diff: https://reviews.apache.org/r/31938/diff/
>
>
> Testing
> -------
>
> Tested on 3-node cluster and in embedded mode.
> ```
> > select * from sys.memory;
> +------------+------------+--------------+------------+----------------+------------+
> | hostname | user_port | heap_current | heap_max | direct_current | direct_max |
> +------------+------------+--------------+------------+----------------+------------+
> | perfnode206.perf.lab | 31010 | 375599272 | 4225236992 | 6048576 | 8589934592 |
> | perfnode207.perf.lab | 31010 | 275524512 | 4225236992 | 2000000 | 8589934592 |
> | perfnode208.perf.lab | 31010 | 289288416 | 4225236992 | 2000000 | 8589934592 |
> +------------+------------+--------------+------------+----------------+------------+
> ```
>
>
> Thanks,
>
> Sudheesh Katkam
>
>
Re: Review Request 31938: DRILL-2275: Need implementations of sys
tables for drill memory and threads profiles
Posted by Chris Westin <ch...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31938/#review77148
-----------------------------------------------------------
One minor comment about a comment, but looks good otherwise.
exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java
<https://reviews.apache.org/r/31938/#comment125029>
Shouldn't this be a TODO? Of course, inquiring minds what to know "enhance it how?" If you're going to put in a comment like this (which I think is great), you need to be more specific.
- Chris Westin
On March 19, 2015, 5:02 p.m., Sudheesh Katkam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31938/
> -----------------------------------------------------------
>
> (Updated March 19, 2015, 5:02 p.m.)
>
>
> Review request for drill and Venki Korukanti.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> DRILL-2275: Added support to allow for querying cluster state information.
> + If table isDistributed(), BatchCreator and SystemTableScan allow for a distributed query.
> + SystemRecordReader reads SystemRecords.
> + There is now a generic data type for static tables.
> + GroupScan can enforce width to be maximum width on ExcessiveExchangeRemover.
> + GroupScan has minimum width for SimpleParallelizer.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/drill/common/JSONOptions.java 64e6d52
> exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java af8c1dc
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScan.java 276ecb5
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/GroupScan.java 23860a3
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/ParallelizationInfo.java 75a009e
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java 12043ce
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Stats.java e61b38f
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/StatsCollector.java 1f56556
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/ExcessiveExchangeIdentifier.java a237014
> exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java c1e64e6
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/DrillbitIterator.java 67d7cf9
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryRecord.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/StaticDrillTable.java c1e8dd1
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecord.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecordReader.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTable.java 0bf2156
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java a1bec1e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePlugin.java 2c70fd4
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePluginConfig.java 93fe68e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableScan.java cdd0d18
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/ThreadsRecord.java PRE-CREATION
> exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java c1803bc
>
> Diff: https://reviews.apache.org/r/31938/diff/
>
>
> Testing
> -------
>
> Tested on 3-node cluster and in embedded mode.
> ```
> > select * from sys.memory;
> +------------+------------+--------------+------------+----------------+------------+
> | hostname | user_port | heap_current | heap_max | direct_current | direct_max |
> +------------+------------+--------------+------------+----------------+------------+
> | perfnode206.perf.lab | 31010 | 375599272 | 4225236992 | 6048576 | 8589934592 |
> | perfnode207.perf.lab | 31010 | 275524512 | 4225236992 | 2000000 | 8589934592 |
> | perfnode208.perf.lab | 31010 | 289288416 | 4225236992 | 2000000 | 8589934592 |
> +------------+------------+--------------+------------+----------------+------------+
> ```
>
>
> Thanks,
>
> Sudheesh Katkam
>
>
Re: Review Request 31938: DRILL-2275: Need implementations of sys
tables for drill memory and threads profiles
Posted by Sudheesh Katkam <sk...@maprtech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31938/
-----------------------------------------------------------
(Updated March 20, 2015, 12:02 a.m.)
Review request for drill and Venki Korukanti.
Changes
-------
Made changes requested by Venki, Chris and Jacques.
Repository: drill-git
Description
-------
DRILL-2275: Added support to allow for querying cluster state information.
+ If table isDistributed(), BatchCreator and SystemTableScan allow for a distributed query.
+ SystemRecordReader reads SystemRecords.
+ There is now a generic data type for static tables.
+ GroupScan can enforce width to be maximum width on ExcessiveExchangeRemover.
+ GroupScan has minimum width for SimpleParallelizer.
Diffs (updated)
-----
common/src/main/java/org/apache/drill/common/JSONOptions.java 64e6d52
exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java af8c1dc
exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScan.java 276ecb5
exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/GroupScan.java 23860a3
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/ParallelizationInfo.java 75a009e
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java 12043ce
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Stats.java e61b38f
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/StatsCollector.java 1f56556
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/ExcessiveExchangeIdentifier.java a237014
exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java c1e64e6
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/DrillbitIterator.java 67d7cf9
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryRecord.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/StaticDrillTable.java c1e8dd1
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecord.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecordReader.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTable.java 0bf2156
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java a1bec1e
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePlugin.java 2c70fd4
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePluginConfig.java 93fe68e
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableScan.java cdd0d18
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/ThreadsRecord.java PRE-CREATION
exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java c1803bc
Diff: https://reviews.apache.org/r/31938/diff/
Testing (updated)
-------
Tested on 3-node cluster and in embedded mode.
```
> select * from sys.memory;
+------------+------------+--------------+------------+----------------+------------+
| hostname | user_port | heap_current | heap_max | direct_current | direct_max |
+------------+------------+--------------+------------+----------------+------------+
| perfnode206.perf.lab | 31010 | 375599272 | 4225236992 | 6048576 | 8589934592 |
| perfnode207.perf.lab | 31010 | 275524512 | 4225236992 | 2000000 | 8589934592 |
| perfnode208.perf.lab | 31010 | 289288416 | 4225236992 | 2000000 | 8589934592 |
+------------+------------+--------------+------------+----------------+------------+
```
Thanks,
Sudheesh Katkam
Re: Review Request 31938: DRILL-2275: Need implementations of sys
tables for drill memory and threads profiles
Posted by Sudheesh Katkam <sk...@maprtech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31938/
-----------------------------------------------------------
(Updated March 19, 2015, 8:48 p.m.)
Review request for drill and Venki Korukanti.
Changes
-------
Renamed columns.
Get heap max memory and not init memory (+ refactoring)
Repository: drill-git
Description
-------
DRILL-2275: Added support to allow for querying cluster state information.
+ If table isDistributed(), BatchCreator and SystemTableScan allow for a distributed query.
+ SystemRecordReader reads SystemRecords.
+ There is now a generic data type for static tables.
+ GroupScan can enforce width to be maximum width on ExcessiveExchangeRemover.
+ GroupScan has minimum width for SimpleParallelizer.
Diffs (updated)
-----
common/src/main/java/org/apache/drill/common/JSONOptions.java 64e6d52
exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScan.java 276ecb5
exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/GroupScan.java 23860a3
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/ParallelizationInfo.java 75a009e
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java f8d1803
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Stats.java e61b38f
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/StatsCollector.java 1f56556
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/ExcessiveExchangeIdentifier.java a237014
exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java c1e64e6
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/DrillbitIterator.java 67d7cf9
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryRecord.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/StaticDrillTable.java c1e8dd1
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecord.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecordReader.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTable.java 0bf2156
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java a1bec1e
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePlugin.java 2c70fd4
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePluginConfig.java 93fe68e
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableScan.java cdd0d18
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/ThreadsRecord.java PRE-CREATION
exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java c1803bc
Diff: https://reviews.apache.org/r/31938/diff/
Testing (updated)
-------
Tested on 3-node cluster and in embedded mode.
```
> select * from sys.memory;
+------------+------------+-----------------+-------------------+---------------------+
| host_name | user_port | total_heap_size | current_heap_size | direct_alloc_memory |
+------------+------------+-----------------+-------------------+---------------------+
| perfnode208.perf.lab | 31010 | 4225236992 | 483863736 | 5000147 |
| perfnode207.perf.lab | 31010 | 4225236992 | 341408256 | 2000000 |
| perfnode206.perf.lab | 31010 | 4225236992 | 311670104 | 2000000 |
+------------+------------+-----------------+-------------------+---------------------+
```
Thanks,
Sudheesh Katkam
Re: Review Request 31938: DRILL-2275: Need implementations of sys
tables for drill memory and threads profiles
Posted by Sudheesh Katkam <sk...@maprtech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31938/
-----------------------------------------------------------
(Updated March 17, 2015, 3:51 p.m.)
Review request for drill and Venki Korukanti.
Changes
-------
private static finals.
Repository: drill-git
Description
-------
DRILL-2275: Added support to allow for querying cluster state information.
+ If table isDistributed(), BatchCreator and SystemTableScan allow for a distributed query.
+ SystemRecordReader reads SystemRecords.
+ There is now a generic data type for static tables.
+ GroupScan can enforce width to be maximum width on ExcessiveExchangeRemover.
+ GroupScan has minimum width for SimpleParallelizer.
Diffs (updated)
-----
common/src/main/java/org/apache/drill/common/JSONOptions.java 64e6d52
exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScan.java 276ecb5
exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/GroupScan.java 23860a3
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/ParallelizationInfo.java 75a009e
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java f8d1803
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Stats.java e61b38f
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/StatsCollector.java 1f56556
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/ExcessiveExchangeIdentifier.java a237014
exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java c1e64e6
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryRecord.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/StaticDrillTable.java c1e8dd1
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecord.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecordReader.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTable.java 0bf2156
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java a1bec1e
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePlugin.java 2c70fd4
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePluginConfig.java 93fe68e
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableScan.java cdd0d18
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/ThreadsRecord.java PRE-CREATION
exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java c1803bc
Diff: https://reviews.apache.org/r/31938/diff/
Testing
-------
Tested on 3-node cluster and in embedded mode.
```
> select * from sys.memory;
+------------+--------------+------------+---------------------+
| host_name | user_port | total_memory | heap_size | direct_alloc_memory |
+------------+--------------+------------+---------------------+
| perfnode206.perf.lab | 31010 | 1073741824 | 395823432 | 5000000 |
| perfnode208.perf.lab | 31010 | 1073741824 | 337127496 | 2000000 |
| perfnode207.perf.lab | 31010 | 1073741824 | 289272760 | 2000000 |
+------------+--------------+------------+---------------------+
```
Thanks,
Sudheesh Katkam
Re: Review Request 31938: DRILL-2275: Need implementations of sys
tables for drill memory and threads profiles
Posted by Chris Westin <ch...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31938/#review76693
-----------------------------------------------------------
Ship it!
Just one minor comment; if that works and you can do that, great. Ship it in any case.
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryRecord.java
<https://reviews.apache.org/r/31938/#comment124322>
In this case, can the MaterializedFields also be private final statics? Just add them to the output and capture for use? Or does that not work?
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/ThreadsRecord.java
<https://reviews.apache.org/r/31938/#comment124323>
Can the MaterializedFields be private final statics, as for the other file?
- Chris Westin
On March 16, 2015, 6:01 p.m., Sudheesh Katkam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31938/
> -----------------------------------------------------------
>
> (Updated March 16, 2015, 6:01 p.m.)
>
>
> Review request for drill and Venki Korukanti.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> DRILL-2275: Added support to allow for querying cluster state information.
> + If table isDistributed(), BatchCreator and SystemTableScan allow for a distributed query.
> + SystemRecordReader reads SystemRecords.
> + There is now a generic data type for static tables.
> + GroupScan can enforce width to be maximum width on ExcessiveExchangeRemover.
> + GroupScan has minimum width for SimpleParallelizer.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/drill/common/JSONOptions.java 64e6d52
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScan.java 276ecb5
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/GroupScan.java 23860a3
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/ParallelizationInfo.java 75a009e
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java f8d1803
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Stats.java e61b38f
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/StatsCollector.java 1f56556
> exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/ExcessiveExchangeIdentifier.java a237014
> exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java c1e64e6
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryRecord.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/StaticDrillTable.java c1e8dd1
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecord.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecordReader.java PRE-CREATION
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTable.java 0bf2156
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java a1bec1e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePlugin.java 2c70fd4
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePluginConfig.java 93fe68e
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableScan.java cdd0d18
> exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/ThreadsRecord.java PRE-CREATION
> exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java c1803bc
>
> Diff: https://reviews.apache.org/r/31938/diff/
>
>
> Testing
> -------
>
> Tested on 3-node cluster and in embedded mode.
> ```
> > select * from sys.memory;
> +------------+--------------+------------+---------------------+
> | host_name | user_port | total_memory | heap_size | direct_alloc_memory |
> +------------+--------------+------------+---------------------+
> | perfnode206.perf.lab | 31010 | 1073741824 | 395823432 | 5000000 |
> | perfnode208.perf.lab | 31010 | 1073741824 | 337127496 | 2000000 |
> | perfnode207.perf.lab | 31010 | 1073741824 | 289272760 | 2000000 |
> +------------+--------------+------------+---------------------+
> ```
>
>
> Thanks,
>
> Sudheesh Katkam
>
>
Re: Review Request 31938: DRILL-2275: Need implementations of sys
tables for drill memory and threads profiles
Posted by Sudheesh Katkam <sk...@maprtech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31938/
-----------------------------------------------------------
(Updated March 17, 2015, 1:01 a.m.)
Review request for drill and Venki Korukanti.
Repository: drill-git
Description
-------
DRILL-2275: Added support to allow for querying cluster state information.
+ If table isDistributed(), BatchCreator and SystemTableScan allow for a distributed query.
+ SystemRecordReader reads SystemRecords.
+ There is now a generic data type for static tables.
+ GroupScan can enforce width to be maximum width on ExcessiveExchangeRemover.
+ GroupScan has minimum width for SimpleParallelizer.
Diffs (updated)
-----
common/src/main/java/org/apache/drill/common/JSONOptions.java 64e6d52
exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScan.java 276ecb5
exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/GroupScan.java 23860a3
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/ParallelizationInfo.java 75a009e
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java f8d1803
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Stats.java e61b38f
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/StatsCollector.java 1f56556
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/ExcessiveExchangeIdentifier.java a237014
exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java c1e64e6
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryRecord.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/StaticDrillTable.java c1e8dd1
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecord.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecordReader.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTable.java 0bf2156
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java a1bec1e
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePlugin.java 2c70fd4
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePluginConfig.java 93fe68e
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableScan.java cdd0d18
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/ThreadsRecord.java PRE-CREATION
exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java c1803bc
Diff: https://reviews.apache.org/r/31938/diff/
Testing
-------
Tested on 3-node cluster and in embedded mode.
```
> select * from sys.memory;
+------------+--------------+------------+---------------------+
| host_name | user_port | total_memory | heap_size | direct_alloc_memory |
+------------+--------------+------------+---------------------+
| perfnode206.perf.lab | 31010 | 1073741824 | 395823432 | 5000000 |
| perfnode208.perf.lab | 31010 | 1073741824 | 337127496 | 2000000 |
| perfnode207.perf.lab | 31010 | 1073741824 | 289272760 | 2000000 |
+------------+--------------+------------+---------------------+
```
Thanks,
Sudheesh Katkam
Re: Review Request 31938: DRILL-2275: Need implementations of sys
tables for drill memory and threads profiles
Posted by Sudheesh Katkam <sk...@maprtech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31938/
-----------------------------------------------------------
(Updated March 12, 2015, 5:44 a.m.)
Review request for drill and Venki Korukanti.
Changes
-------
Made changes suggseted by Chris.
Repository: drill-git
Description
-------
DRILL-2275: Added support to allow for querying cluster state information.
+ If table isDistributed(), BatchCreator and SystemTableScan allow for a distributed query.
+ SystemRecordReader reads SystemRecords.
+ There is now a generic data type for static tables.
+ GroupScan can enforce width to be maximum width on ExcessiveExchangeRemover.
+ GroupScan has minimum width for SimpleParallelizer.
Diffs (updated)
-----
common/src/main/java/org/apache/drill/common/JSONOptions.java 64e6d52
exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScan.java 276ecb5
exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/GroupScan.java 23860a3
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/ParallelizationInfo.java 75a009e
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java f8d1803
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Stats.java e61b38f
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/StatsCollector.java 1f56556
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/ExcessiveExchangeIdentifier.java a237014
exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java c1e64e6
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryRecord.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/StaticDrillTable.java c1e8dd1
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecord.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemRecordReader.java PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTable.java 0bf2156
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableBatchCreator.java a1bec1e
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePlugin.java 2c70fd4
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTablePluginConfig.java 93fe68e
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/SystemTableScan.java cdd0d18
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/ThreadsRecord.java PRE-CREATION
exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java c1803bc
Diff: https://reviews.apache.org/r/31938/diff/
Testing (updated)
-------
Tested on 3-node cluster and in embedded mode.
```
> select * from sys.memory;
+------------+--------------+------------+---------------------+
| host_name | user_port | total_memory | heap_size | direct_alloc_memory |
+------------+--------------+------------+---------------------+
| perfnode206.perf.lab | 31010 | 1073741824 | 395823432 | 5000000 |
| perfnode208.perf.lab | 31010 | 1073741824 | 337127496 | 2000000 |
| perfnode207.perf.lab | 31010 | 1073741824 | 289272760 | 2000000 |
+------------+--------------+------------+---------------------+
```
Thanks,
Sudheesh Katkam