Awesome
Rexbug is a thin Elixir wrapper for :redbug
production-friendly Erlang tracing debugger.
It tries to preserve :redbug's simple and
intuitive interface while making it more convenient to use by Elixir developers.
README
What does it do?
It's an Elixir tracing - based debugger. It allows you to connect to a live Elixir system and get information when some code inside it is executed. The "some code" can be a whole module, a specific function in the module, or some function, but only if it's called with some specific arguments. The information you can get is the function arguments, its result and the stack trace.
If you want to you can narrow the tracing down to a specific process, investigate a remote node or look at the messages sent between processes.
Rexbug is also production-system-friendly. It has sensible limits for both time and amount of trace events after which it stops tracing. This means you won't accidentally overload the system and flood your console with debug information if your trace pattern wasn't specific enough.
It also provides Rexbug.dtop/1 -
a tool with much of the functionality of
observer,
with an interface similar to and Linux's htop
How does it work?
Rexbug uses unmodified :redbug library
underneath. It translates Elixir syntax to the Erlang format expected by
:redbug.
:redbug in turn interacts with the
Erlang trace facility.
It will instruct the Erlang VM to generate so called
"trace messages" when certain events (such as a particular
function being called) occur.
The trace messages are either printed (i.e. human readable)
to a file or to the screen; or written to a trc file.
Using a trc file puts less stress on the system, but
there is no way to count the messages (so the msgs opt
is ignored), and the files can only be read by special tools
(such as 'bread'). Printing and trc files cannot be combined.
By default (i.e. if the :file opt is not given), messages
are printed.
Installation
The package can be installed by adding Rexbug to your list of dependencies
in mix.exs:
def deps do
[{:rexbug, ">= 2.0.0-rc1"}]
end
After you've added Rexbug to your project, there's nothing left to do - you
can start debugging it at your convenience.
Examples
Tracing a single function
The general syntax is Rexbug.start("ModuleName.function_name/_").
The /_ tells Rexbug we're interested in any arity of the function.
iex(3)> Rexbug.start("Map.get/_ :: return") # asking for the return value too
{105, 2}
iex(4)> Map.get(%{}, :foo)
nil
# 10:49:02 #PID<0.1057.0> IEx.Evaluator.init/4
# Map.get(%{}, :foo)
# 10:49:02 #PID<0.1057.0> IEx.Evaluator.init/4
# Map.get(%{}, :foo, nil)
# 10:49:02 #PID<0.1057.0> IEx.Evaluator.init/4
# Map.get/3 -> nil
# 10:49:02 #PID<0.1057.0> IEx.Evaluator.init/4
# Map.get/2 -> nil
redbug done, timeout - 2
Tracing a whole module
iex> Rexbug.start("Map")
{82, 41}
iex> m = Map.put(%{}, :foo, :bar) # this could have been called in any process
%{foo: :bar}
# 18:51:55 #PID<0.150.0> IEx.Evaluator.init/4
# Map.__info__(:macros)
iex> Map.get(m, :foo)
:bar
# 18:51:57 #PID<0.150.0> IEx.Evaluator.init/4
# Map.__info__(:macros)
# 18:51:57 #PID<0.150.0> IEx.Evaluator.init/4
# Map.get(%{foo: :bar}, :foo)
# 18:51:57 #PID<0.150.0> IEx.Evaluator.init/4
# Map.get(%{foo: :bar}, :foo, nil)
iex> # Rexbug tracing is going to time out now
nil
redbug done, timeout - 4
iex>
Tracing with matching function arguments
iex> Rexbug.start("Enum.member?([_, _, _], \"foo\")")
{82, 1}
iex> Enum.member?([1, 2], "foo") # first argument doesn't match
false
iex> Enum.member?([1, 2, 3], "bar") # second argument doesn't match
false
iex> Enum.member?([1, 2, 3], "foo") # will match
false
# 18:55:44 #PID<0.150.0> IEx.Evaluator.init/4
# Enum.member?([1, 2, 3], "foo")
iex> Rexbug.stop()
:stopped
redbug done, local_done - 1
iex>
Tracing messages sent and received from a process
iex> s = self()
#PID<0.193.0>
iex> proc = Process.spawn(fn ->
...> receive do
...> anything -> send(s, {:got, anything})
...> end
...> end, [])
iex> Rexbug.start([:send, :receive], procs: [proc], time: 60_000)
{1, 0}
iex> send(proc, :foo)
:foo
# 18:31:08 #PID<0.208.0> (:dead)
# <<< :foo
# 18:31:08 #PID<0.208.0> (:dead)
# #PID<0.193.0> IEx.Evaluator.init/4 <<< {:got, :foo}
iex> flush()
{:got, :foo}
:ok
redbug done, timeout - 2
Running dtop
iex(0)> Rexbug.dtop() # start dtop
{:ok, :started}
-------------------------------------------------------------------------------
nonode@nohost size: 41.6M(420.4G), cpu%: 2(0), procs: 378, runq: 0, 12:27:41
memory: proc 8.4M, atom 737.5k, bin 2.2M, code 15.9M, ets 1.7M
pid name current msgq mem cpu
<0.685.0> redbug_dtop redbug_dtop:prc_i 0 431.8k 2
<0.66.0> group:server/3 group:more_data/6 0 176.4k 0
<0.64.0> user_drv user_drv:server_l 0 26.5k 0
<0.10.0> erl_prim_loader erl_prim_loader:l 0 142.9k 0
<0.50.0> code_server code_server:loop/ 0 284.7k 0
<0.456.0> inet_gethost_native inet_gethost_nati 0 18.8k 0
<0.437.0> Elixir.DBConnection.Connecti gen_server:loop/7 0 26.8k 0
<0.440.0> Elixir.DBConnection.Connecti erlang:hibernate/ 0 3.3k 0
<0.448.0> Elixir.DBConnection.Connecti erlang:hibernate/ 0 3.3k 0
<0.447.0> Elixir.DBConnection.Connecti erlang:hibernate/ 0 3.3k 0
<0.446.0> Elixir.DBConnection.Connecti erlang:hibernate/ 0 3.3k 0
<0.445.0> Elixir.DBConnection.Connecti erlang:hibernate/ 0 3.3k 0
<0.444.0> Elixir.DBConnection.Connecti erlang:hibernate/ 0 3.3k 0
<0.443.0> Elixir.DBConnection.Connecti erlang:hibernate/ 0 3.3k 0
<0.442.0> Elixir.DBConnection.Connecti erlang:hibernate/ 0 3.3k 0
<0.441.0> Elixir.DBConnection.Connecti erlang:hibernate/ 0 3.3k 0
<0.439.0> Elixir.DBConnection.Connecti erlang:hibernate/ 0 3.3k 0
<0.538.0> cowboy_clock gen_server:loop/7 0 10.9k 0
<0.550.0> telemetry_poller:init/1 gen_server:loop/7 0 3.0k 0
iex(3)> Rexbug.dtop(sort: :mem) # sort by memory
{:ok, :reconfigured}
-------------------------------------------------------------------------------
nonode@nohost size: 43.4M(420.4G), cpu%: 1(0), procs: 378, runq: 0, 12:27:47
memory: proc 10.1M, atom 737.5k, bin 2.6M, code 15.9M, ets 1.7M
pid name current msgq mem cpu
<0.685.0> redbug_dtop redbug_dtop:prc_i 0 1.8M 1
<0.44.0> application_controller gen_server:loop/7 0 691.1k 0
<0.521.0> telemetry_poller_default gen_server:loop/7 0 407.4k 0
<0.168.0> 'Elixir.Hex.State' gen_server:loop/7 0 372.3k 0
<0.50.0> code_server code_server:loop/ 0 284.7k 0
<0.473.0> Elixir.Postgrex.TypeServer:i gen_server:loop/7 0 264.4k 0
<0.66.0> group:server/3 group:more_data/6 0 197.3k 0
<0.497.0> Elixir.FileSystem.Backends.F gen_server:loop/7 0 176.3k 0
<0.10.0> erl_prim_loader erl_prim_loader:l 0 142.9k 0
<0.496.0> phoenix_live_reload_file_mon gen_server:loop/7 0 142.8k 0
<0.82.0> disk_log disk_log:loop/1 0 109.7k 0
<0.165.0> 'Elixir.Hex.Supervisor' gen_server:loop/7 0 88.8k 0
<0.2.0> erts_literal_area_collector: erts_literal_area 0 77.7k 0
<0.0.0> init init:loop/1 0 42.3k 0
<0.1.0> erts_code_purger erts_code_purger: 0 35.5k 0
<0.682.0> Elixir.IEx.Evaluator:init/4 Elixir.IEx.Evalua 0 34.4k 0
<0.578.0> supervisor:ranch_acceptors_s erlang:hibernate/ 0 30.7k 0
<0.64.0> user_drv user_drv:server_l 0 26.8k 0
<0.456.0> inet_gethost_native inet_gethost_nati 0 26.7k 0
iex(4)> Rexbug.dtop() # stop dtop
{:ok, :stopped}
For more info and advanced usage see Rexbug.Dtop module docs.
Motivation
I was discussing investigating some unexpected behaviour in an Elixir project with
one of my colleagues and he rightfully suggested
using a tracing debugger to get to the bottom of it. The tool he had the most
experience with was :redbug and it soon turned out it's possible to use from
iex and with Elixir code, as long as you know some Erlang and are mindful
of some gotchas.
I really liked how :redbug was designed, but wished using it with Elixir was
more streamlined...
:redbug syntax comparison
If you want to move between :redbug and Rexbug or you're just curious how
they compare, here's some examples:
# tracing an Erlang module
Rexbug.start(":ets")
:redbug.start('ets')
# stopping
Rexbug.stop()
:redbug.stop()
# tracing with arguments matching (and strings)
Rexbug.start("String.starts_with?(_, \"foo\")") # you can use the ~s sigil so that you don't have to escape the quotes
:redbug.start('\'Elixir.String\':\'starts_with?\'(_, <<"foo">>)')
# selecting the actions
Rexbug.start("Map.new/_ :: return;stack")
:redbug.start('\'Elixir.Map\':new -> return;stack')
Known issues/limitations
- In the trace patterns
"Mod.fun"implicitly translates to"Mod.fun()", which is equivalent to"Mod.fun/0". To target the function with any arity, use"Mod.fun/_"or"Mod.fun/any"
FAQ
Which versions of Elixir and Erlang/OTP does Rexbug support?
- Elixir 1.11.4 and newer
- Erlang/OTP 24 and newer
If you're targeting an older system, try Rexbug 1.x, which handles Elixir 1.4 and newer
Make sure to check the general Erlang/Elixir compatibility table
My app is already running and it doesn't have Rexbug in its dependencies. Can I still debug it?
Yes! You can connect to it from a node that has Rexbug in its path and work from there.
The app:
nietaki@shiny:~$ iex --sname production --cookie monster
Erlang/OTP 20 [erts-9.0] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:10] [hipe] [kernel-poll:false]
Interactive Elixir (1.4.5) - press Ctrl+C to exit (type h() ENTER for help)
iex(production@shiny)1> Rexbug.help() # the node doesn't know about Rexbug
** (UndefinedFunctionError) function Rexbug.help/0 is undefined (module Rexbug is not available)
Rexbug.help()
iex(production@shiny)2> Stream.interval(1000) |> Enum.each(&Integer.mod(&1, 3))
Your local shell:
nietaki@shiny:~$ iex --sname investigator --cookie monster -pa ~/repos/rexbug/_build/dev/lib/rexbug/ebin/ -pa ~/repos/rexbug/_build/dev/lib/redbug/ebin/
Erlang/OTP 20 [erts-9.0] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:10] [hipe] [kernel-poll:false]
Interactive Elixir (1.4.5) - press Ctrl+C to exit (type h() ENTER for help)
iex(investigator@shiny)1> opts = [target: :production@shiny, msgs: 4]
[target: :production@shiny, msgs: 4]
iex(investigator@shiny)2> Rexbug.start("Integer.mod/2", opts)
{63, 1}
% 23:53:44 <9548.89.0>({'Elixir.IEx.Evaluator',init,4})
% 'Elixir.Integer':mod(46, 3)
% 23:53:45 <9548.89.0>({'Elixir.IEx.Evaluator',init,4})
% 'Elixir.Integer':mod(47, 3)
% 23:53:46 <9548.89.0>({'Elixir.IEx.Evaluator',init,4})
% 'Elixir.Integer':mod(48, 3)
% 23:53:47 <9548.89.0>({'Elixir.IEx.Evaluator',init,4})
% 'Elixir.Integer':mod(49, 3)
redbug done, msg_count - 4
iex(investigator@shiny)3>
Instead of pointing to the Rexbug and :redbug beam files you can just clone
this repo and run iex -S mix in the root directory:
nietaki@shiny:rexbug (master=)$ iex --sname investigator --cookie monster -S mix
Erlang/OTP 19 [erts-8.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false]
Interactive Elixir (1.4.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(investigator@shiny)1> opts = [target: :production@shiny, msgs: 4]
[target: :production@shiny, msgs: 4]
iex(investigator@shiny)2> Rexbug.start("Integer.mod/2", opts)
(...)
How does Rexbug compare with other Elixir debuggers?
Good question! There are other projects that give you similar capabilities, like dbg by @fishcakez or exrun, both of which look great and are definitely more battle-tested than Rexbug.
I'll try to add a brief and unbiased (as much as I can) comparison after I've spent some time playing with them so I can do make sure I know what I'm talking about.
Why "translate" the syntax instead of forking :redbug and caling its internals directly?
There's a number of reasons:
- The performance overhead should be irrelevant. You pay the small additional cost
once every time you run
Rexbug.start/2and it should be negligible compared to whatever system you're debugging. - Since
:redbugis included as-is you can still use it directly and benefit from any new features it might get. Also if your team is split between people more comfortable in Erlang and Elixir, everyone can use what they prefer. - "time to market" - doing this was the simplest way I could think of to get to a relatively polished library.
- This approach didn't seem to limit the possible featureset. All the
:redbugfeatures can still be provided.
In general there weren't enough reasons to do it the other way. I don't rule out the possibility of a future rewrite, which wouldn't be too drastic anyways.