[kudu-CR] WIP: flatbuffers-based codegen

["Todd Lipcon (Code Review)" <ge...@cloudera.org>]

Todd Lipcon has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/2903

Change subject: WIP: flatbuffers-based codegen
......................................................................

WIP: flatbuffers-based codegen

This switches to using flatbuffers as a mechanism to pass structured
constants into the precompiled code. The idea here is as follows;
- any "runtime constants" get described in a flatbuffer
- at codegen time, we construct a flatbuffer with all of that information
- we serialize the flat buffer into a 'const char[]' global variable
  inside the codegen module, and pass a pointer to it into the
  precompiled code
- the precompiled code 'deserializes' it, but since the flatbuffer generated
  code is all inlined in headers, clang is smart enough to see that it's all
  constant accesses and thus propagate the constants, unroll loops, etc.

The net here is that it's much easier to write the code generation, since almost
no LLVM code has to be written.

Somewhat suprisingly, this seems to reduce the number of instructions
substantially for some projections, and only increases it by one instruction
for a few cases. I ran codegen-test -codegen_print_mc and grepped out the
number of instructions for each test case:

Before patch    After patch
----------------------------
8		8
281		254
8		8
8		8
57		57
57		57
222		210
222		210
62		63
62		63
206		187
206		187
281		254
281		254
102		109
53		54
390		344
330		300
107		108

My guess is that this is due to better aliasing analysis when the whole thing
is inside a single function, but who knows.

WIP:
- the test coverage doesn't seem good -- I had a bug where I wasn't copying
  to arenas and nothing failed, and also had a bug where I wasn't handling
  defaults at all and nothing failed.
- the FindFlatbuffers.cmake file has a lot of copy-paste from the protobuf
  one that could probably be edited

Change-Id: I583ef1edad379bb13b34362f121263de28c8334c
---
M CMakeLists.txt
A cmake_modules/FindFlatbuffers.cmake
M src/kudu/codegen/CMakeLists.txt
A src/kudu/codegen/codegen_params.fbs
M src/kudu/codegen/module_builder.cc
M src/kudu/codegen/module_builder.h
M src/kudu/codegen/precompiled.cc
M src/kudu/codegen/row_projector.cc
M thirdparty/build-definitions.sh
M thirdparty/build-thirdparty.sh
M thirdparty/download-thirdparty.sh
M thirdparty/vars.sh
12 files changed, 450 insertions(+), 201 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/03/2903/1
-- 
To view, visit http://gerrit.cloudera.org:8080/2903
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I583ef1edad379bb13b34362f121263de28c8334c
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>