You are viewing a plain text version of this content. The canonical link for it is here.
Posted to erlang@couchdb.apache.org by Andrey Kuprianov <an...@gmail.com> on 2013/10/28 05:18:16 UTC
Hi
Hi all.
Im interested in being able to do some Erlang especially when writing
design docs. At very least it would be nice to follow some examples and
pick up some basic Erlang programming skills.
Btw (and probably everyone knows anyway), there's a great book on Erlang
and it's available online on http://learnyousomeerlang.com/.
Cheers,
Andrey
Re: Hi! Here is a Sample of A general purpose crypto map/reduce
Erlang function for you.
Posted by Andrey Kuprianov <an...@gmail.com>.
Cool :) thanks
On Mon, Oct 28, 2013 at 6:55 PM, david martin
<da...@lymegreen.co.uk>wrote:
> On 28/10/13 04:18, Andrey Kuprianov wrote:
>
>> Hi all.
>>
>> Im interested in being able to do some Erlang especially when writing
>> design docs. At very least it would be nice to follow some examples and
>> pick up some basic Erlang programming skills.
>>
>> Btw (and probably everyone knows anyway), there's a great book on Erlang
>> and it's available online on http://learnyousomeerlang.com/**.
>>
>> Cheers,
>>
>> Andrey
>>
>> Here is a generic piece of Erlang code I wrote some time ago. Paste into
> futon to run.
>
> %-----------------------------**------------------------------**
> -----------
> % A general purpose crypto map/reduce Erlang function pair.
> % This Function performs a crypto md5 hash sum of all the JSON keys in
> every document.
> % This enables verification of the existence of every key in every
> document or enables the
> % differing documents to be identified.
>
> fun({Doc}) ->
> Attachments = proplists:get_value(<<"_**attachments">>, Doc),
> %returns a proplist or the atom undefined
>
> DocKeys = proplists:get_keys(Doc),
> % returns list of all keys in doc
>
> CryptoContextDocKeys=crypto:**md5_init(),
>
> CryptoContextDocKeysFinal =
> lists:foldr(fun(Dk,**CryptoAccum)->
> crypto:md5_update(CryptoAccum,**crypto:md5(Dk))
> end, CryptoContextDocKeys,DocKeys),
>
> CryptoNameAllKeys =
> list_to_binary(lists:flatten(
> [io_lib:format("~2.16.0b",[N]) || <<N>>
> <= crypto:md5_final(**CryptoContextDocKeysFinal)])),
>
> Emit([CryptoNameAllKeys,**DocKeys] ,2),
>
> case Attachments of
>
> {Propslist1}-> PropsKeysList=proplists:get_**keys(Propslist1),
> lists:foreach(fun(K)->
> {[{<<"content_type">>,Content_**type1},{<<"revpos">>,Revpos1},
> {<<"digest">>,Mp5digest1},{<<"**length">>, Length1},{<<"stub">>, Stub1}]}
> = proplists:get_value(K, Propslist1) ,
> Emit([Content_type1,Stub1,**Length1,Mp5digest1,K,Revpos1], 1)
> end, PropsKeysList);
>
> undefined-> ok
>
> end %end of Attachments case
>
> end.%end of view map function
>
> %-----------------------------**------------------------------**
> -----------
>
> fun(Keys,Values,ReReduce)->
>
> case ReReduce of
> true->lists:sum(Values);
> false->length(Values)
> end
> end.%end of view reduce function
> %-----------------------------**------------------------------**
> -----------
>
> sample data
> {
> "_id": "Competition 300608.csv1",
> "_rev": "1-**e87429f1dbea2db5cfd242b3398e32**ee",
> "URN": "130908",
> "LA": "806",
> "LA NAME": "Middlesbrough",
> "ESTAB": "6907",
> "SCHOOL NAME": "Macmillan Academy",
> "STREET": "PO Box 8",
> "LOCALITY": "Stockton Road",
> "ADDRESS 3": "",
> " TOWN": "Middlesbrough",
> "COUNTY": "Cleveland",
> "POSTCODE": "TS5 4YU",
> "TEL STD": "01642",
> "TEL NO": "800800",
> "HEAD TITLE": "Mr",
> "HEAD FIRST NAME": "K U",
> "HEAD LAST NAME": "Junior",
> "HEAD HONOURS": "O.B.E",
> "TYPE OF ESTABLISHMENT": "Academies",
> "PHASE OF EDUCATION": "[Not Applicable]",
> "STAT LOW AGE": "11",
> "STAT HIGH AGE": "19"
> }
> %-----------------------------**------------------------------**
> -----------
>
> sample output
> Map Value
> ["**947fad4183f054d4f6c50bd956dbb5**76", ["_rev", "URN", "COUNTY", "_id",
> "POSTCODE", "LA", " TOWN", "HEAD LAST NAME", "TYPE OF ESTABLISHMENT", "STAT
> HIGH AGE", "ESTAB", "ADDRESS 3", "TEL STD", "SCHOOL NAME", "HEAD TITLE",
> "PHASE OF EDUCATION", "LOCALITY", "TEL NO", "LA NAME", "STREET", "HEAD
> FIRST NAME", "HEAD HONOURS", "STAT LOW AGE"]]
> ID: Competition 300608.csv1
>
> Exact Reduce value
> ["**947fad4183f054d4f6c50bd956dbb5**76", ["_rev", "URN", "COUNTY", "_id",
> "POSTCODE", "LA", " TOWN", "HEAD LAST NAME", "TYPE OF ESTABLISHMENT", "STAT
> HIGH AGE", "ESTAB", "ADDRESS 3", "TEL STD", "SCHOOL NAME", "HEAD TITLE",
> "PHASE OF EDUCATION", "LOCALITY", "TEL NO", "LA NAME", "STREET", "HEAD
> FIRST NAME", "HEAD HONOURS", "STAT LOW AGE"]]
> 22518 %no. docs in database.
> %-----------------------------**------------------------------**
> -----------
>
>
> --
> David Martin
>
>
Re: Hi! Here is a Sample of A general purpose crypto map/reduce
Erlang function for you.
Posted by david martin <da...@lymegreen.co.uk>.
On 28/10/13 04:18, Andrey Kuprianov wrote:
> Hi all.
>
> Im interested in being able to do some Erlang especially when writing
> design docs. At very least it would be nice to follow some examples and
> pick up some basic Erlang programming skills.
>
> Btw (and probably everyone knows anyway), there's a great book on Erlang
> and it's available online on http://learnyousomeerlang.com/.
>
> Cheers,
>
> Andrey
>
Here is a generic piece of Erlang code I wrote some time ago. Paste into futon to run.
%----------------------------------------------------------------------
% A general purpose crypto map/reduce Erlang function pair.
% This Function performs a crypto md5 hash sum of all the JSON keys in every document.
% This enables verification of the existence of every key in every document or enables the
% differing documents to be identified.
fun({Doc}) ->
Attachments = proplists:get_value(<<"_attachments">>, Doc),
%returns a proplist or the atom undefined
DocKeys = proplists:get_keys(Doc),
% returns list of all keys in doc
CryptoContextDocKeys=crypto:md5_init(),
CryptoContextDocKeysFinal =
lists:foldr(fun(Dk,CryptoAccum)->
crypto:md5_update(CryptoAccum,crypto:md5(Dk))
end, CryptoContextDocKeys,DocKeys),
CryptoNameAllKeys =
list_to_binary(lists:flatten(
[io_lib:format("~2.16.0b",[N]) || <<N>>
<= crypto:md5_final(CryptoContextDocKeysFinal)])),
Emit([CryptoNameAllKeys,DocKeys] ,2),
case Attachments of
{Propslist1}-> PropsKeysList=proplists:get_keys(Propslist1),
lists:foreach(fun(K)->
{[{<<"content_type">>,Content_type1},{<<"revpos">>,Revpos1},
{<<"digest">>,Mp5digest1},{<<"length">>, Length1},{<<"stub">>, Stub1}]}
= proplists:get_value(K, Propslist1) ,
Emit([Content_type1,Stub1,Length1,Mp5digest1,K,Revpos1], 1)
end, PropsKeysList);
undefined-> ok
end %end of Attachments case
end.%end of view map function
%----------------------------------------------------------------------
fun(Keys,Values,ReReduce)->
case ReReduce of
true->lists:sum(Values);
false->length(Values)
end
end.%end of view reduce function
%----------------------------------------------------------------------
sample data
{
"_id": "Competition 300608.csv1",
"_rev": "1-e87429f1dbea2db5cfd242b3398e32ee",
"URN": "130908",
"LA": "806",
"LA NAME": "Middlesbrough",
"ESTAB": "6907",
"SCHOOL NAME": "Macmillan Academy",
"STREET": "PO Box 8",
"LOCALITY": "Stockton Road",
"ADDRESS 3": "",
" TOWN": "Middlesbrough",
"COUNTY": "Cleveland",
"POSTCODE": "TS5 4YU",
"TEL STD": "01642",
"TEL NO": "800800",
"HEAD TITLE": "Mr",
"HEAD FIRST NAME": "K U",
"HEAD LAST NAME": "Junior",
"HEAD HONOURS": "O.B.E",
"TYPE OF ESTABLISHMENT": "Academies",
"PHASE OF EDUCATION": "[Not Applicable]",
"STAT LOW AGE": "11",
"STAT HIGH AGE": "19"
}
%----------------------------------------------------------------------
sample output
Map Value
["947fad4183f054d4f6c50bd956dbb576", ["_rev", "URN", "COUNTY", "_id", "POSTCODE", "LA", " TOWN", "HEAD LAST NAME", "TYPE OF ESTABLISHMENT", "STAT HIGH AGE", "ESTAB", "ADDRESS 3", "TEL STD", "SCHOOL NAME", "HEAD TITLE", "PHASE OF EDUCATION", "LOCALITY", "TEL NO", "LA NAME", "STREET", "HEAD FIRST NAME", "HEAD HONOURS", "STAT LOW AGE"]]
ID: Competition 300608.csv1
Exact Reduce value
["947fad4183f054d4f6c50bd956dbb576", ["_rev", "URN", "COUNTY", "_id", "POSTCODE", "LA", " TOWN", "HEAD LAST NAME", "TYPE OF ESTABLISHMENT", "STAT HIGH AGE", "ESTAB", "ADDRESS 3", "TEL STD", "SCHOOL NAME", "HEAD TITLE", "PHASE OF EDUCATION", "LOCALITY", "TEL NO", "LA NAME", "STREET", "HEAD FIRST NAME", "HEAD HONOURS", "STAT LOW AGE"]]
22518 %no. docs in database.
%----------------------------------------------------------------------
--
David Martin