Fwd: debugging

[Jan Lehnardt <ja...@apache.org>]

This is useful for any CouchDB developer :)

Thanks Dave!
Jan
--


Begin forwarded message:

> From: Dave Cottlehuber <dc...@jsonified.com>
> Subject: debugging
> Date: November 5, 2012 09:11:49 GMT+01:00
> To: erlang@couchdb.apache.org
> Reply-To: erlang@couchdb.apache.org
> 
> Hi there,
> 
> I wanted to throw in a brief set of notes on debugging stuff I learned
> about, some recently & some taken from the internets.
> 
> most of this is for running "couchdb -i" but you can also do this via
> remsh if you set a cookie and either sname/name up. Let me know if
> that could be covered too.
> 
> # tell a non-networked erlang vm (e.g. started by couchdb -i) to get connected
> 
>    net_kernel:start([<newname>, shortnames]).
>    erlang:set_cookie(node(), 'yumyumyum').
> 
> now you can remsh or redbug to it. (what's redbug??) ….
> 
> # killing off a process (e.g. a view server) that you don't want
> running but there's no API to allow you to do that. The erlang shotgun
> is here!
> 
>    i(). %% look for the process name. You could do this nicer with
> list comprehensions etc below.
>    exit(list_to_pid("<0.114.0>"), false). %% boom!
> 
> # get a list of all running processes and then do evil stuff
> 
> - in this case we find out who is the memory hog, using v(-1) to refer
> to the result of the last computation in the shell and keeping the
> info around in Procs just in case we want to check it later.
> 
>    %% this trick via Geoff Cant / @archaelus
>    Procs =  [ {Pid, element(2, process_info(Pid, memory))}
>            || Pid <- erlang:processes() ].
> 
>     hd( lists:reverse( lists:keysort( 2, v(-1) ) ) ).
> 
> # trace all calls going through a particular module
> 
> NB this is pretty savage if you get it wrong, you can *easily* crash
> the vm. redbug is usually a better option, but I don't see how to get
> it to do this nifty trick:
> 
>   open_tracer(Module) ->
>         dbg:tracer(),
>         dbg:p(all, [call]),
>         dbg:tpl(Module, [{'_', [], [{return_trace}]}]).
> 
> # redbug
> 
> If you're using erlang and you don't know about redbug you are doing it wrong!!
> 
> Install redbug as part of eper kit, build is straightforwards using
> rebar, add it into your ebin path. Again I can provide more details on
> that if needed.
> 
> https://github.com/massemanet/eper/blob/master/doc/redbug.txt
> 
> Let's do a simple example - tracing all calls to lists:max, from our shell:
> 
> (akai@akai)20> redbug:start("lists:max").
> ok
> (akai@akai)21> lists:max([100,900,300,100,720]).
> 
> 08:57:40 <{erlang,apply,2}> {lists,max,[[100,900,300,100,720]]}
> 900
> 
> 08:57:40 <{erlang,apply,2}> {lists,max,[[900,300,100,720],100]}
> 
> 08:57:40 <{erlang,apply,2}> {lists,max,[[300,100,720],900]}
> 
> 08:57:40 <{erlang,apply,2}> {lists,max,[[100,720],900]}
> 
> 08:57:40 <{erlang,apply,2}> {lists,max,[[720],900]}
> 
> 08:57:40 <{erlang,apply,2}> {lists,max,[[],900]}
> 
> cool! you can see your recursion in action.
> 
> Fancy,  how about doing that from a remote node?
> 
> switch to your shell & set up redbug to call from a normal prompt/terminal.
> 
> launch redbug from your terminal:
> 
> % redbug akai@akai 5000 10 "lists:max"
> 
> this tetls it to connect to erlang host akai@akai, with a timeout of 5
> seconds (you'll likely need longer the first few times), and collect a
> maximum of 10 debug traces before stopping. Trace only lists:max.
> 
> redbug also accepts cookies and a bunch of other self-explanatory options.
> 
> whip back quickly into your erl shell and up arrow to restart the
> lists:max function.
> 
> Your terminal will show the same goodies.
> 
> Now imagine doing the same thing on a live system, tracing a specific
> call through mochiweb layer, or something in couch docs storage
> system. Woot!!!
> 
> redbug can also optionally provide the call stack, arguments and
> results but that starts getting hefty. So then write the results out
> to a file.
> 
> Why redbug?
> 
> - ease of use compared to writing tracers yourself
> - excellent logging
> - filtering by time and number of results tends to avoid the "oops my
> real-time debugging killed your couches".
> 
> # stats inside the vm
> 
> A number of options - use a fancy tool like vmstats from mononcqc or
> for a more hands-on view use entop.
> 
> 
> # debugging erlang startup
> 
> sometime erlang barfs on startup. how to proceed?
> 
> erl -init_debug  -env ERL_LIB
> /usr/local/Cellar/couchdb/1.2.0/lib/couchdb/erlang/lib -couch_ini
> 
> And then hand-start each application:
> 
> application:start(crypto).
> application:start(couch).
> …
> 
> Sometimes this helps find the issue. A working sample is here
> https://friendpaste.com/6zett3aAHSafRNZbgfMBoD
> 
> I should try the last step broken down into subsidiary steps rather
> than a single application.
> 
> Anybody got other tips/tricks to share?
> 
> A+
> Dave