You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Joan Touzet <wo...@apache.org> on 2015/07/12 02:14:58 UTC
Windows build blocked by khash NIF
Hello everyone,
Progress on the Windows build was surprisingly smooth, thanks to native
MSVC build chain support in rebar, and improved in the version I am
running locally (2.6.0).
Unfortunately, we have a release blocker on Windows in the khash NIF.
This NIF expects to call an Erlang BEAM VM internal function,
make_hash2(). This function is *not* exported in erl_interface, and only
works on *NIX platforms because the khash .so NIF is loaded into a
calling binary that has make_hash2 defined internally.
It is very naughty that we do this, and I'm disappointed this code made
it into the tree in its current state. But such is life ^_^;
On Windows, the BEAM VM is self-contained in a dll (beam.smp.dll) which
is called by a nearly stub-sized Erlang executable (erl.exe). The
beam.smp.dll exports only 3 functions, none of which are make_hash2().
Further complicating things, on Windows any DLL that calls an
external requires that external function to be declared in a .lib
or .def file sourced from the original. I attempted faking a
beam.smp.def/beam.smp.lib of the form:
EXPORTS
make_hash2
and linking, but to no avail - again because make_hash2 is optimized
away inside of the beam.smp.dll, not externally referenceable.
In short, we have three options:
1) Build a custom beam.smp.dll on Windows that exports make_hash2
2) Revert to the old-style ets approach on Windows, conditionally
using khash only on *NIX.
3) Ship CouchDB 2.0 without a Windows build.
My research about this time last year indicated that Windows downloads
of CouchDB actual exceeded downloads of couchdb.tar.gz from
http://couchdb.apache.org/, on the order of ~10000 DLs per month total
(between 500-2000 per month more than the tarball.) My guess is that a
lot of developers use it for local developing or prototyping purposes.
Either way, it seems the Windows build is popular, and we drop support
for it in 2.0 at our peril.
Thoughts?
Re: Windows build blocked by khash NIF
Posted by Paul Davis <pa...@gmail.com>.
Also, for the record I'd like to stick with phash2 rather than the
term_to_binary approach and our own hash function as Erlang has done a
fair amount of tweaking to make their hash function work well on terms
as I understand it.
On Mon, Jul 13, 2015 at 11:50 AM, Paul Davis
<pa...@gmail.com> wrote:
> Sorry, was away on vacation last week.
>
> The back story on that make_hash2 is at [1].
>
> Given that the patch never landed we can fall back to the proposed
> work around there. We can just call erlang:phash2/1 on each key and
> pass that into the NIF. While it won't allow us to easily reimplement
> ETS it should be more than fine for this case.
>
> [1] http://erlang.org/pipermail/erlang-patches/2013-February/003532.html
>
> On Sun, Jul 12, 2015 at 5:30 PM, Robert Samuel Newson
> <rn...@apache.org> wrote:
>> How about this;
>>
>> 1) When built on Windows, the khash_hash_fun insists that the key term is an erlang binary and we hash that.
>>
>> 2) When built on Windows, the functions in khash.erl call term_to_binary before calling down to the NIF functions.
>>
>> B.
>>
>> sketch;
>>
>> #ifndef _WIN32
>> hash_val_t
>> khash_hash_fun(const void* obj)
>> {
>> khnode_t* node = (khnode_t*) obj;
>> return (hash_val_t) make_hash2(node->key);
>> }
>> #else
>> hash_val_t
>> khash_hash_fun(const void* obj)
>> {
>> khnode_t* node = (khnode_t*) obj;
>> Eterm term = node->key;
>> if (ERL_IS_BINARY(term)) {
>> void* data = ERL_BIN_PTR(term);
>> int len = ERL_BIN_SIZE(term);
>> return hash(data, len); // TODO, write hash function
>> }
>> else {
>> return (hash_val_t) make_hash2(node->key);
>> }
>> }
>> #endif
>>
>>
>>> On 12 Jul 2015, at 09:14, Nick North <no...@gmail.com> wrote:
>>>
>>> I'm with Dave - it would be acceptable to ship a modified BEAM if it's a
>>> short-term measure. It's not great, but better than not having Windows
>>> support.
>>>
>>> Nick
>>>
>>> On Sun, 12 Jul 2015 at 08:35 Dave Cottlehuber <dc...@apache.org> wrote:
>>>
>>>>
>>>> On Sun, 12 Jul 2015, at 09:21 AM, Dave Cottlehuber wrote:
>>>>> On Sun, 12 Jul 2015, at 08:35 AM, Joan Touzet wrote:
>>>>>> I scoured the mailing lists but was unable to find any sort of patch
>>>>>> for this, submitted or not. It's certainly not landed in v18.0.x.
>>>>>>
>>>>>> Perhaps Paul is aware of the patch as a starting place at least?
>>>>
>>>> Um I should also say that in the distant past, I built & shipped
>>>> modified BEAMs, although that was just to fix up patches that had been
>>>> accepted but not released. e.g.
>>>> http://people.apache.org/~dch/snapshots/1.1.1/
>>>>
>>>> So I'm not averse to that if we have a patch that's been submitted
>>>> upstream and a reasonable expectation of success. In my experience there
>>>> are only a handful of people who have built from source on Windows...
>>>>
>>>> —
>>>> Dave Cottlehuber
>>>> dch@apache.org
>>>> Sent from my Couch
>>>>
>>
Re: Windows build blocked by khash NIF
Posted by Paul Davis <pa...@gmail.com>.
Sorry, was away on vacation last week.
The back story on that make_hash2 is at [1].
Given that the patch never landed we can fall back to the proposed
work around there. We can just call erlang:phash2/1 on each key and
pass that into the NIF. While it won't allow us to easily reimplement
ETS it should be more than fine for this case.
[1] http://erlang.org/pipermail/erlang-patches/2013-February/003532.html
On Sun, Jul 12, 2015 at 5:30 PM, Robert Samuel Newson
<rn...@apache.org> wrote:
> How about this;
>
> 1) When built on Windows, the khash_hash_fun insists that the key term is an erlang binary and we hash that.
>
> 2) When built on Windows, the functions in khash.erl call term_to_binary before calling down to the NIF functions.
>
> B.
>
> sketch;
>
> #ifndef _WIN32
> hash_val_t
> khash_hash_fun(const void* obj)
> {
> khnode_t* node = (khnode_t*) obj;
> return (hash_val_t) make_hash2(node->key);
> }
> #else
> hash_val_t
> khash_hash_fun(const void* obj)
> {
> khnode_t* node = (khnode_t*) obj;
> Eterm term = node->key;
> if (ERL_IS_BINARY(term)) {
> void* data = ERL_BIN_PTR(term);
> int len = ERL_BIN_SIZE(term);
> return hash(data, len); // TODO, write hash function
> }
> else {
> return (hash_val_t) make_hash2(node->key);
> }
> }
> #endif
>
>
>> On 12 Jul 2015, at 09:14, Nick North <no...@gmail.com> wrote:
>>
>> I'm with Dave - it would be acceptable to ship a modified BEAM if it's a
>> short-term measure. It's not great, but better than not having Windows
>> support.
>>
>> Nick
>>
>> On Sun, 12 Jul 2015 at 08:35 Dave Cottlehuber <dc...@apache.org> wrote:
>>
>>>
>>> On Sun, 12 Jul 2015, at 09:21 AM, Dave Cottlehuber wrote:
>>>> On Sun, 12 Jul 2015, at 08:35 AM, Joan Touzet wrote:
>>>>> I scoured the mailing lists but was unable to find any sort of patch
>>>>> for this, submitted or not. It's certainly not landed in v18.0.x.
>>>>>
>>>>> Perhaps Paul is aware of the patch as a starting place at least?
>>>
>>> Um I should also say that in the distant past, I built & shipped
>>> modified BEAMs, although that was just to fix up patches that had been
>>> accepted but not released. e.g.
>>> http://people.apache.org/~dch/snapshots/1.1.1/
>>>
>>> So I'm not averse to that if we have a patch that's been submitted
>>> upstream and a reasonable expectation of success. In my experience there
>>> are only a handful of people who have built from source on Windows...
>>>
>>> —
>>> Dave Cottlehuber
>>> dch@apache.org
>>> Sent from my Couch
>>>
>
Re: Windows build blocked by khash NIF
Posted by Nick North <no...@gmail.com>.
I should have said I was thinking of the reimplementation as a temporary
measure with the assumption that the VM make_hash2 would be exported at
some point and we could throw away our version. But the phash2 option
sounds much better.
Nick
On Mon, 13 Jul 2015 at 18:41, Paul Davis <pa...@gmail.com>
wrote:
> It might be possible to reimplement make_hash2 but it feels a bit
> dirty, not to mention we'd have to assign someone to watching Erlang
> commits in case they ever change the implementation for any reason.
>
> As for calling phash2 overhead, I don't think it'll be significant. We
> have to do the hash either way, so its basically just the overhead
> from calling a BIF which doesn't even register on the radar for things
> that we think about for performance.
>
> On Mon, Jul 13, 2015 at 12:11 PM, Nick North <no...@gmail.com> wrote:
> > This feels conceptually nicer than shipping a modified VM. Any idea what
> > effect it would have on performance?
> >
> > Alternatively, putting my deep ignorance of the VM to use to make silly
> > suggestions: would it be possible to replicate (or even just copy the
> code
> > for) make_hash2? That would save the term_to_binary requirement, but
> maybe
> > make_hash2 needs knowledge from the inside of the VM, which might make
> the
> > idea impossible.
> >
> > Nick
> >
> > On Sun, 12 Jul 2015 at 23:32 Robert Samuel Newson <rn...@apache.org>
> > wrote:
> >
> >> How about this;
> >>
> >> 1) When built on Windows, the khash_hash_fun insists that the key term
> is
> >> an erlang binary and we hash that.
> >>
> >> 2) When built on Windows, the functions in khash.erl call term_to_binary
> >> before calling down to the NIF functions.
> >>
> >> B.
> >>
> >> sketch;
> >>
> >> #ifndef _WIN32
> >> hash_val_t
> >> khash_hash_fun(const void* obj)
> >> {
> >> khnode_t* node = (khnode_t*) obj;
> >> return (hash_val_t) make_hash2(node->key);
> >> }
> >> #else
> >> hash_val_t
> >> khash_hash_fun(const void* obj)
> >> {
> >> khnode_t* node = (khnode_t*) obj;
> >> Eterm term = node->key;
> >> if (ERL_IS_BINARY(term)) {
> >> void* data = ERL_BIN_PTR(term);
> >> int len = ERL_BIN_SIZE(term);
> >> return hash(data, len); // TODO, write hash function
> >> }
> >> else {
> >> return (hash_val_t) make_hash2(node->key);
> >> }
> >> }
> >> #endif
> >>
> >>
> >> > On 12 Jul 2015, at 09:14, Nick North <no...@gmail.com> wrote:
> >> >
> >> > I'm with Dave - it would be acceptable to ship a modified BEAM if
> it's a
> >> > short-term measure. It's not great, but better than not having Windows
> >> > support.
> >> >
> >> > Nick
> >> >
> >> > On Sun, 12 Jul 2015 at 08:35 Dave Cottlehuber <dc...@apache.org> wrote:
> >> >
> >> >>
> >> >> On Sun, 12 Jul 2015, at 09:21 AM, Dave Cottlehuber wrote:
> >> >>> On Sun, 12 Jul 2015, at 08:35 AM, Joan Touzet wrote:
> >> >>>> I scoured the mailing lists but was unable to find any sort of
> patch
> >> >>>> for this, submitted or not. It's certainly not landed in v18.0.x.
> >> >>>>
> >> >>>> Perhaps Paul is aware of the patch as a starting place at least?
> >> >>
> >> >> Um I should also say that in the distant past, I built & shipped
> >> >> modified BEAMs, although that was just to fix up patches that had
> been
> >> >> accepted but not released. e.g.
> >> >> http://people.apache.org/~dch/snapshots/1.1.1/
> >> >>
> >> >> So I'm not averse to that if we have a patch that's been submitted
> >> >> upstream and a reasonable expectation of success. In my experience
> there
> >> >> are only a handful of people who have built from source on Windows...
> >> >>
> >> >> —
> >> >> Dave Cottlehuber
> >> >> dch@apache.org
> >> >> Sent from my Couch
> >> >>
> >>
> >>
>
Re: Windows build blocked by khash NIF
Posted by Paul Davis <pa...@gmail.com>.
It might be possible to reimplement make_hash2 but it feels a bit
dirty, not to mention we'd have to assign someone to watching Erlang
commits in case they ever change the implementation for any reason.
As for calling phash2 overhead, I don't think it'll be significant. We
have to do the hash either way, so its basically just the overhead
from calling a BIF which doesn't even register on the radar for things
that we think about for performance.
On Mon, Jul 13, 2015 at 12:11 PM, Nick North <no...@gmail.com> wrote:
> This feels conceptually nicer than shipping a modified VM. Any idea what
> effect it would have on performance?
>
> Alternatively, putting my deep ignorance of the VM to use to make silly
> suggestions: would it be possible to replicate (or even just copy the code
> for) make_hash2? That would save the term_to_binary requirement, but maybe
> make_hash2 needs knowledge from the inside of the VM, which might make the
> idea impossible.
>
> Nick
>
> On Sun, 12 Jul 2015 at 23:32 Robert Samuel Newson <rn...@apache.org>
> wrote:
>
>> How about this;
>>
>> 1) When built on Windows, the khash_hash_fun insists that the key term is
>> an erlang binary and we hash that.
>>
>> 2) When built on Windows, the functions in khash.erl call term_to_binary
>> before calling down to the NIF functions.
>>
>> B.
>>
>> sketch;
>>
>> #ifndef _WIN32
>> hash_val_t
>> khash_hash_fun(const void* obj)
>> {
>> khnode_t* node = (khnode_t*) obj;
>> return (hash_val_t) make_hash2(node->key);
>> }
>> #else
>> hash_val_t
>> khash_hash_fun(const void* obj)
>> {
>> khnode_t* node = (khnode_t*) obj;
>> Eterm term = node->key;
>> if (ERL_IS_BINARY(term)) {
>> void* data = ERL_BIN_PTR(term);
>> int len = ERL_BIN_SIZE(term);
>> return hash(data, len); // TODO, write hash function
>> }
>> else {
>> return (hash_val_t) make_hash2(node->key);
>> }
>> }
>> #endif
>>
>>
>> > On 12 Jul 2015, at 09:14, Nick North <no...@gmail.com> wrote:
>> >
>> > I'm with Dave - it would be acceptable to ship a modified BEAM if it's a
>> > short-term measure. It's not great, but better than not having Windows
>> > support.
>> >
>> > Nick
>> >
>> > On Sun, 12 Jul 2015 at 08:35 Dave Cottlehuber <dc...@apache.org> wrote:
>> >
>> >>
>> >> On Sun, 12 Jul 2015, at 09:21 AM, Dave Cottlehuber wrote:
>> >>> On Sun, 12 Jul 2015, at 08:35 AM, Joan Touzet wrote:
>> >>>> I scoured the mailing lists but was unable to find any sort of patch
>> >>>> for this, submitted or not. It's certainly not landed in v18.0.x.
>> >>>>
>> >>>> Perhaps Paul is aware of the patch as a starting place at least?
>> >>
>> >> Um I should also say that in the distant past, I built & shipped
>> >> modified BEAMs, although that was just to fix up patches that had been
>> >> accepted but not released. e.g.
>> >> http://people.apache.org/~dch/snapshots/1.1.1/
>> >>
>> >> So I'm not averse to that if we have a patch that's been submitted
>> >> upstream and a reasonable expectation of success. In my experience there
>> >> are only a handful of people who have built from source on Windows...
>> >>
>> >> —
>> >> Dave Cottlehuber
>> >> dch@apache.org
>> >> Sent from my Couch
>> >>
>>
>>
Re: Windows build blocked by khash NIF
Posted by Nick North <no...@gmail.com>.
This feels conceptually nicer than shipping a modified VM. Any idea what
effect it would have on performance?
Alternatively, putting my deep ignorance of the VM to use to make silly
suggestions: would it be possible to replicate (or even just copy the code
for) make_hash2? That would save the term_to_binary requirement, but maybe
make_hash2 needs knowledge from the inside of the VM, which might make the
idea impossible.
Nick
On Sun, 12 Jul 2015 at 23:32 Robert Samuel Newson <rn...@apache.org>
wrote:
> How about this;
>
> 1) When built on Windows, the khash_hash_fun insists that the key term is
> an erlang binary and we hash that.
>
> 2) When built on Windows, the functions in khash.erl call term_to_binary
> before calling down to the NIF functions.
>
> B.
>
> sketch;
>
> #ifndef _WIN32
> hash_val_t
> khash_hash_fun(const void* obj)
> {
> khnode_t* node = (khnode_t*) obj;
> return (hash_val_t) make_hash2(node->key);
> }
> #else
> hash_val_t
> khash_hash_fun(const void* obj)
> {
> khnode_t* node = (khnode_t*) obj;
> Eterm term = node->key;
> if (ERL_IS_BINARY(term)) {
> void* data = ERL_BIN_PTR(term);
> int len = ERL_BIN_SIZE(term);
> return hash(data, len); // TODO, write hash function
> }
> else {
> return (hash_val_t) make_hash2(node->key);
> }
> }
> #endif
>
>
> > On 12 Jul 2015, at 09:14, Nick North <no...@gmail.com> wrote:
> >
> > I'm with Dave - it would be acceptable to ship a modified BEAM if it's a
> > short-term measure. It's not great, but better than not having Windows
> > support.
> >
> > Nick
> >
> > On Sun, 12 Jul 2015 at 08:35 Dave Cottlehuber <dc...@apache.org> wrote:
> >
> >>
> >> On Sun, 12 Jul 2015, at 09:21 AM, Dave Cottlehuber wrote:
> >>> On Sun, 12 Jul 2015, at 08:35 AM, Joan Touzet wrote:
> >>>> I scoured the mailing lists but was unable to find any sort of patch
> >>>> for this, submitted or not. It's certainly not landed in v18.0.x.
> >>>>
> >>>> Perhaps Paul is aware of the patch as a starting place at least?
> >>
> >> Um I should also say that in the distant past, I built & shipped
> >> modified BEAMs, although that was just to fix up patches that had been
> >> accepted but not released. e.g.
> >> http://people.apache.org/~dch/snapshots/1.1.1/
> >>
> >> So I'm not averse to that if we have a patch that's been submitted
> >> upstream and a reasonable expectation of success. In my experience there
> >> are only a handful of people who have built from source on Windows...
> >>
> >> —
> >> Dave Cottlehuber
> >> dch@apache.org
> >> Sent from my Couch
> >>
>
>
Re: Windows build blocked by khash NIF
Posted by Robert Samuel Newson <rn...@apache.org>.
How about this;
1) When built on Windows, the khash_hash_fun insists that the key term is an erlang binary and we hash that.
2) When built on Windows, the functions in khash.erl call term_to_binary before calling down to the NIF functions.
B.
sketch;
#ifndef _WIN32
hash_val_t
khash_hash_fun(const void* obj)
{
khnode_t* node = (khnode_t*) obj;
return (hash_val_t) make_hash2(node->key);
}
#else
hash_val_t
khash_hash_fun(const void* obj)
{
khnode_t* node = (khnode_t*) obj;
Eterm term = node->key;
if (ERL_IS_BINARY(term)) {
void* data = ERL_BIN_PTR(term);
int len = ERL_BIN_SIZE(term);
return hash(data, len); // TODO, write hash function
}
else {
return (hash_val_t) make_hash2(node->key);
}
}
#endif
> On 12 Jul 2015, at 09:14, Nick North <no...@gmail.com> wrote:
>
> I'm with Dave - it would be acceptable to ship a modified BEAM if it's a
> short-term measure. It's not great, but better than not having Windows
> support.
>
> Nick
>
> On Sun, 12 Jul 2015 at 08:35 Dave Cottlehuber <dc...@apache.org> wrote:
>
>>
>> On Sun, 12 Jul 2015, at 09:21 AM, Dave Cottlehuber wrote:
>>> On Sun, 12 Jul 2015, at 08:35 AM, Joan Touzet wrote:
>>>> I scoured the mailing lists but was unable to find any sort of patch
>>>> for this, submitted or not. It's certainly not landed in v18.0.x.
>>>>
>>>> Perhaps Paul is aware of the patch as a starting place at least?
>>
>> Um I should also say that in the distant past, I built & shipped
>> modified BEAMs, although that was just to fix up patches that had been
>> accepted but not released. e.g.
>> http://people.apache.org/~dch/snapshots/1.1.1/
>>
>> So I'm not averse to that if we have a patch that's been submitted
>> upstream and a reasonable expectation of success. In my experience there
>> are only a handful of people who have built from source on Windows...
>>
>> —
>> Dave Cottlehuber
>> dch@apache.org
>> Sent from my Couch
>>
Re: Windows build blocked by khash NIF
Posted by Nick North <no...@gmail.com>.
I'm with Dave - it would be acceptable to ship a modified BEAM if it's a
short-term measure. It's not great, but better than not having Windows
support.
Nick
On Sun, 12 Jul 2015 at 08:35 Dave Cottlehuber <dc...@apache.org> wrote:
>
> On Sun, 12 Jul 2015, at 09:21 AM, Dave Cottlehuber wrote:
> > On Sun, 12 Jul 2015, at 08:35 AM, Joan Touzet wrote:
> > > I scoured the mailing lists but was unable to find any sort of patch
> > > for this, submitted or not. It's certainly not landed in v18.0.x.
> > >
> > > Perhaps Paul is aware of the patch as a starting place at least?
>
> Um I should also say that in the distant past, I built & shipped
> modified BEAMs, although that was just to fix up patches that had been
> accepted but not released. e.g.
> http://people.apache.org/~dch/snapshots/1.1.1/
>
> So I'm not averse to that if we have a patch that's been submitted
> upstream and a reasonable expectation of success. In my experience there
> are only a handful of people who have built from source on Windows...
>
> —
> Dave Cottlehuber
> dch@apache.org
> Sent from my Couch
>
Re: Windows build blocked by khash NIF
Posted by Dave Cottlehuber <dc...@apache.org>.
On Sun, 12 Jul 2015, at 09:21 AM, Dave Cottlehuber wrote:
> On Sun, 12 Jul 2015, at 08:35 AM, Joan Touzet wrote:
> > I scoured the mailing lists but was unable to find any sort of patch
> > for this, submitted or not. It's certainly not landed in v18.0.x.
> >
> > Perhaps Paul is aware of the patch as a starting place at least?
Um I should also say that in the distant past, I built & shipped
modified BEAMs, although that was just to fix up patches that had been
accepted but not released. e.g.
http://people.apache.org/~dch/snapshots/1.1.1/
So I'm not averse to that if we have a patch that's been submitted
upstream and a reasonable expectation of success. In my experience there
are only a handful of people who have built from source on Windows...
—
Dave Cottlehuber
dch@apache.org
Sent from my Couch
Re: Windows build blocked by khash NIF
Posted by Dave Cottlehuber <dc...@skunkwerks.at>.
On Sun, 12 Jul 2015, at 08:35 AM, Joan Touzet wrote:
> I scoured the mailing lists but was unable to find any sort of patch
> for this, submitted or not. It's certainly not landed in v18.0.x.
>
> Perhaps Paul is aware of the patch as a starting place at least?
>
> ----- Original Message -----
> > From: "Adam Kocoloski" <ko...@apache.org>
> > To: dev@couchdb.apache.org, "Joan Touzet" <wo...@apache.org>
> > Sent: Saturday, July 11, 2015 10:08:43 PM
> > Subject: Re: Windows build blocked by khash NIF
> >
> > There’s a comment in the code where we declare make_hash2() which
> > states
> >
> > > // There's a pending patch to expose this in the NIF API in newer
> > > Erlangs.
> >
> > Did that patch ever land?
> >
> > Adam
Is this unrelated to erlang:phash/2 or erlang:phash2/1,2 ?
It sounds like a thing which if we begged might get into a 17.5.7 or
18.0.3 for example.
A+
Dave
Re: Windows build blocked by khash NIF
Posted by Joan Touzet <wo...@apache.org>.
I scoured the mailing lists but was unable to find any sort of patch
for this, submitted or not. It's certainly not landed in v18.0.x.
Perhaps Paul is aware of the patch as a starting place at least?
----- Original Message -----
> From: "Adam Kocoloski" <ko...@apache.org>
> To: dev@couchdb.apache.org, "Joan Touzet" <wo...@apache.org>
> Sent: Saturday, July 11, 2015 10:08:43 PM
> Subject: Re: Windows build blocked by khash NIF
>
> There’s a comment in the code where we declare make_hash2() which
> states
>
> > // There's a pending patch to expose this in the NIF API in newer
> > Erlangs.
>
> Did that patch ever land?
>
> Adam
>
> > On Jul 11, 2015, at 8:54 PM, Joan Touzet <wo...@apache.org> wrote:
> >
> > Alex says:
> >> On Sun, Jul 12, 2015 at 3:14 AM, Joan Touzet <wo...@apache.org>
> >> wrote:
> >>> 1) Build a custom beam.smp.dll on Windows that exports
> >>> make_hash2
> >>
> >> Sounds as the right solution, but how it will complicate build
> >> process?
> >
> > It means that Windows builds have to have a custom built Erlang
> > with patches that we write. Windows builds will never be able to
> > run
> > against stock Erlang builds from erlang.org. We will be responsible
> > for
> > rebuilding and patching as new versions of Erlang are released,
> > e.g. for
> > security patches.
> >
> >>> 2) Revert to the old-style ets approach on Windows,
> >>> conditionally
> >>> using khash only on *NIX.
> >>
> >> Sounds as the simplest solution, but won't it cause any issues?
> >
> > Bob Newson proposed this solution. It's going to take a bunch of
> > build
> > magic to make happen, as we have to load the khash app only when
> > not
> > running on a Windows platform. The old code should be around
> > somewhere,
> > but I've not looked at it in a long while.
> >
> >> Also, how about ask on erlang-questions@ about export make_hash2?
> >> Sure, it won't solve problems for now, but could help in the
> >> future.
> >
> > If you want to ask please be my guest, my time is limited.
> >
>
>
Re: Windows build blocked by khash NIF
Posted by Adam Kocoloski <ko...@apache.org>.
There’s a comment in the code where we declare make_hash2() which states
> // There's a pending patch to expose this in the NIF API in newer Erlangs.
Did that patch ever land?
Adam
> On Jul 11, 2015, at 8:54 PM, Joan Touzet <wo...@apache.org> wrote:
>
> Alex says:
>> On Sun, Jul 12, 2015 at 3:14 AM, Joan Touzet <wo...@apache.org>
>> wrote:
>>> 1) Build a custom beam.smp.dll on Windows that exports make_hash2
>>
>> Sounds as the right solution, but how it will complicate build
>> process?
>
> It means that Windows builds have to have a custom built Erlang
> with patches that we write. Windows builds will never be able to run
> against stock Erlang builds from erlang.org. We will be responsible for
> rebuilding and patching as new versions of Erlang are released, e.g. for
> security patches.
>
>>> 2) Revert to the old-style ets approach on Windows, conditionally
>>> using khash only on *NIX.
>>
>> Sounds as the simplest solution, but won't it cause any issues?
>
> Bob Newson proposed this solution. It's going to take a bunch of build
> magic to make happen, as we have to load the khash app only when not
> running on a Windows platform. The old code should be around somewhere,
> but I've not looked at it in a long while.
>
>> Also, how about ask on erlang-questions@ about export make_hash2?
>> Sure, it won't solve problems for now, but could help in the future.
>
> If you want to ask please be my guest, my time is limited.
>
Re: Windows build blocked by khash NIF
Posted by Joan Touzet <wo...@apache.org>.
Alex says:
> On Sun, Jul 12, 2015 at 3:14 AM, Joan Touzet <wo...@apache.org>
> wrote:
> > 1) Build a custom beam.smp.dll on Windows that exports make_hash2
>
> Sounds as the right solution, but how it will complicate build
> process?
It means that Windows builds have to have a custom built Erlang
with patches that we write. Windows builds will never be able to run
against stock Erlang builds from erlang.org. We will be responsible for
rebuilding and patching as new versions of Erlang are released, e.g. for
security patches.
> > 2) Revert to the old-style ets approach on Windows, conditionally
> > using khash only on *NIX.
>
> Sounds as the simplest solution, but won't it cause any issues?
Bob Newson proposed this solution. It's going to take a bunch of build
magic to make happen, as we have to load the khash app only when not
running on a Windows platform. The old code should be around somewhere,
but I've not looked at it in a long while.
> Also, how about ask on erlang-questions@ about export make_hash2?
> Sure, it won't solve problems for now, but could help in the future.
If you want to ask please be my guest, my time is limited.
Re: Windows build blocked by khash NIF
Posted by Alexander Shorin <kx...@gmail.com>.
On Sun, Jul 12, 2015 at 3:14 AM, Joan Touzet <wo...@apache.org> wrote:
> 1) Build a custom beam.smp.dll on Windows that exports make_hash2
Sounds as the right solution, but how it will complicate build process?
> 2) Revert to the old-style ets approach on Windows, conditionally
> using khash only on *NIX.
Sounds as the simplest solution, but won't it cause any issues?
Also, how about ask on erlang-questions@ about export make_hash2?
Sure, it won't solve problems for now, but could help in the future.
--
,,,^..^,,,