Building Avro C++ Natively On Windows

[Gary Price <pe...@gmail.com>]

I built Avro C++ natively on 64-bit Windows. My aim was to avoid
touching the Windows build and keep all build-related changes in the
CMake world.

1. Get Cygwin. This is to provide an up-to-date version of Flex, not
to use the Cygwin tool chain. Set CYGWIN_ROOT to where it is
installed, say c:\Cygwin.

2. Get Perl. Actually I already had version 2.6 so I used that. It
might be better to use a later version, as I had to make one Perl
script change.

2. Get Boost. I used 1_45_0. Compile regex, filesystem, system,
program_options, thread, date_time. The command I used was

c:\>bjam --toolset=msvc-9.0 threading=multi address-model=64
--build-dir=_output/x64 --stagedir=stage/x64 --prefix=\boost install

This builds all possible 64-bit versions and puts them in C:\boost,
along with the headers. Set the environment variable BOOST_ROOT to
c:\boost.
It's worth building everything like that so that you can change the
linkage later if you want. However, I used static linkage, and it that
were your aim you could add linkage=static. And if you just wanted to
build the specific libs, add --with-xxxx for each. Also I built for
release only.
Note: the CMakeFiles.txt specifies only the first four of those I
listed. However, one of the test apps needs thread and it needs
date_time.
Note: a noob error, which I made, is to add an additional -- on items
in the build command that don't have them, on address-model to be
specific.

3. Get and install CMake.

5. Get and unpack Avro C++ zip file. I am using 1.6.1.

6. Provide some headers. This is to satisfy Avro's need for various
UNIX-style headers, as well as direct it to the FlexLexer.h in Cygwin.
As well, it provides a place to fix some conflicts between Windows
type definitions and those from other places. Make a new folder
msvc_extras under the Avro root. Put the following under it:

- FlexLexer.h:

#ifndef HEADER_MSVC_FLEXLEXER_WRAPPER
#define HEADER_MSVC_FLEXLEXER_WRAPPER
#pragma once
#include <usr/include/FlexLexer.h>
#endif

- stdint.h:

#ifndef HEADER_MSVC_STDINT_WRAPPER
#define HEADER_MSVC_STDINT_WRAPPER
#pragma once

#include <crtdefs.h>
#define __intptr_t_defined
#define __uintptr_t_defined

// This is to prevent the stdint header defining WCHAR_MIN, which
leads to many warnings
// as it is defined by a Windows header.
// We do this if WCHAR_MIN is not yet defined, and then undo it only
if we did it.
#ifndef WCHAR_MIN
#define WCHAR_MIN
#define WCHAR_MIN_DEFINED_IN_STDINT_H_WRAPPER
#endif

#include <usr/include/stdint.h>

#ifdef WCHAR_MIN_DEFINED_IN_STDINT_H_WRAPPER
#undef WCHAR_MIN
#endif // WCHAR_MIN_DEFINED_IN_STDINT_H_WRAPPER

#endif // HEADER_MSVC_STDINT_WRAPPER

- unistd.h:

#ifndef HEADER_MSVC_UNISTD_WRAPPER
#define HEADER_MSVC_UNISTD_WRAPPER
#pragma once
#include <io.h>
#endif // HEADER_MSVC_UNISTD_WRAPPER

A subdirectory "sys", containing

time.h:

#ifndef _MSVC_EXTRAS_TIME_H_
#define _MSVC_EXTRAS_TIME_H_
#pragma once
#include <time.h>
#endif // _MSVC_EXTRAS_TIME_H_

uio.h:

#ifndef HEADER_UIO_WRAPPER
#define HEADER_UIO_WRAPPER
#pragma once
#include <io.h>
#endif // HEADER_UIO_WRAPPER

7. Changes in directory "api":
BufferPrint.hh: line 64, 77: parenthesize std::min to avoid
interpretation as a preprocessor macro.
BufferReader.hh: three more instances of std::min.
BufferStreambuf.hh: another instance of std::min. Also, the use of
_Ios_Openmode is deprecated, so change it to ios::openmode in three
places.
NodeConcepts.hh, line 117: in friend declaration, change "class" to "struct".
NodeImpl.hh: lines 406, 407 - use a cast to remove a warning.

8. Changes in directory "impl":
Compiler.cc: line 24, change the yyparse prototype so it returns int
to fix a linker error.
Resolver.cc: line 509: The line
 uint8_t val[size_];
won't compile as size_ is not const. Making size_ const and setting it
in the constructor initializer list doesn't help. Use a vector (add
the header as well):
	std::vector<uint8_t> val(size_);
        reader.readFixed(&val[0], size_);
Types.cc: line 66: This does not compile. The error is
error C2679: binary '<<' : no operator found which takes a right-hand
operand of type 'const std::string' (or there is no acceptable
conversion). Add".c_str()":
		os << strings::typeToString[type].c_str();

9. Changes in "scripts" directory:
gen-cppcode.py: line 589: At least for my version of Python (2.6), on
Windows, raw_input() returns a token that includes a trailing \r.
Remove this by adding
        line = line.rstrip()
I am not a Python expert so this might not be optimal or even needed
in later Python versions.

10. Changes in "test" directory:
AvroGencppTests.cpp: For the four places that
std::numeric_limits<>::max and min are used, there is a conflict with
a macro that is available in MSVC. Parenthesize them:
(std::numeric_limits<double>::min)().
buffertest.cc: This seems to need stdint.h, so add it. Also, on line
794 there is another place where a non-constant is used in an
automatic array. Replace it with a vector.
testgen.cc: fix another four instances of std::numeric_limits::max, min.
unittest.cc: fix another two instances of std::numeric_limits::max, min.

11. Changes in CMakeLists.txt (This is my first use of CMake so I am
expecting some of this is not the way it should be done):
- Change how boost links:

if(MSVC)
set(Boost_USE_STATIC_LIBS ON)
set(Boost_USE_MULTITHREADED ON)
set(Boost_USE_STATIC_RUNTIME OFF)
endif()

- Add the two items needed by tests to the list of Boost components:

find_package (Boost 1.38 REQUIRED
    COMPONENTS regex filesystem system program_options thread date_time)

- Allow the compile to find the things put under "msvc_extras" and
under the Cygwin root. When the compiler finds, say <stdint.h>, it
finds it under msvc_extras because of the first of these, then it
finds the real one under Cygwin because of the second. Also add a flag
that tells the FlexLexer header we have no unistd.h.

if(MSVC)
include_directories ( "msvc_extras" "$ENV{CYGWIN_ROOT}" )
add_definitions(-DYY_NO_UNISTD_H)
endif()

- Change the linkage of avrocpp to static, otherwise it is necessary
to add "export" macros everywhere. That can be done but then you get
many, many warnings from using templates in a DLL interface.

if(MSVC)
add_library (avrocpp STATIC ${SOURCE_FILES})
else()
add_library (avrocpp SHARED ${SOURCE_FILES})
endif()

- Add a flag to the buffertest compile so it can use asynchronous
error handling:
if(MSVC)
	set_target_properties(buffertest PROPERTIES COMPILE_FLAGS "/EHa")
endif()

12. Open CMake. Point it at the source, set the output folder to
"builder" under the same root, click Configure and then Generate.

13. Using Visual Studio 2008, open the sln file in the "builder"
directory. Build the ALL_BUILD target.

That's it. The build still generates a lot of warnings, which it would
be good to remove. Any comments on how to do the CMake stuff better
would be appreciated.

Cheers

Gary Price