Awesome
<div align="center">Phoenix + PaperTrail Demo
A showcase of using PaperTrail
in a simple Phoenix Todo List App.
Why? 🤷
We need a way of capturing the history of items in our App
to enable "undo" functionality.
What? 💭
PaperTrail:
github.com/izelnakri/paper_trail
Lets you track and record all the changes in your database
and revert back to anytime in history.
This repo demos using PaperTrail in a simple Todo List.
Who? 👤
This quick demo is aimed at people in the @dwyl team
who need to understand how PaperTrail is used in our App.
How? 👩💻
Prerequisites? 📝
This Demo builds upon the foundational work done
in our Phoenix Todo List Tutorial:
dwyl/phoenix-papertrail-demo
it is
assumed knowledge.
If you haven't been through it, we suggest taking a few minutes to get up-to-speed.
You should also be using Phoenix 1.7+, as we will use TailwindCSS in this tutorial to style the UI.
1. Borrow Baseline Code
Let's start by cloning the code from dwyl/phoenix-todo-list-tutorial. We are going to be using a version of the repo that does not have authentication nor API. This is to simplify the workflow and testing.
We are going to be using this version:
https://github.com/dwyl/phoenix-todo-list-tutorial/tree/ab4470d68f64ed8f5596dd09f3322193317b838e.
You can also find the initial version of this project in the following link:
https://github.com/dwyl/phoenix-papertrail-demo/tree/430e5626d4b8dc15a7d76d4fbae61fed5094f6a8.
In here, you can clone the files with the same initial version we are going to be starting this tutorial.
Just make sure these files look like the following:
/test/app_web/controllers/item_controller_test.exs/test/app/todo_test.exs/test/app_web/controllers/item_html_test.exs/lib/app_web/router.ex/lib/app_web/controllers/item_controller.ex
After cloning the code, run the following commands to fetch the dependencies and to set up databases.
mix deps.get
mix setup
After this, if you run mix phx.server,
you will be able to run the application.
As you can see, it's a simple todo-list application.
2. Installing PaperTrail
We want to see the changes made by the user
when interacting with its todo items.
As it stands, we have a simple items database table
in which the items are saved.
You can check the fields that are persisted in
lib/app/todo/item.ex.
schema "items" do
field :person_id, :integer, default: 0
field :status, :integer, default: 0
field :text, :string
timestamps()
end
To track changes to items within a table,
we are going to be using
PaperTrail.
Let's start by installing it.
Inside mix.exs,
add the following line to the list of dependencies.
def deps do
[
...
{:paper_trail, "~> 0.14.3"}
]
end
We need to change PaperTrail's configuration
inside config/config.exs.
We need to let PaperTrail know
the reference of the database repo
so it can save the changes correctly.
Open config/config.exs and add the following:
config :paper_trail, repo: App.Repo
After this, run the following two commands.
It will fetch the PaperTrail dependency
and recompile the project with the configuration we just added.
mix deps.get
mix compile
After this, we will run the following:
mix papertrail.install
The terminal will print the following information.
* creating priv/repo/migrations
* creating priv/repo/migrations/20221220130334_add_versions.exs
A migration is created. This file, upon execution, will create the table where the changes will be recorded. Let's give it a closer look!
2.1. Analysing the table migration
Here is what you should see in
priv/repo/migrations.
defmodule Repo.Migrations.AddVersions do
use Ecto.Migration
def change do
create table(:versions) do
add :event, :string, null: false, size: 10
add :item_type, :string, null: false
add :item_id, :integer
add :item_changes, :map, null: false
add :originator_id, references(:users) # you can change :users to your own foreign key constraint
add :origin, :string, size: 50
add :meta, :map
# Configure timestamps type in config.ex :paper_trail :timestamps_type
add :inserted_at, :utc_datetime, null: false
end
create index(:versions, [:originator_id])
create index(:versions, [:item_id, :item_type])
# Uncomment if you want to add the following indexes to speed up special queries:
# create index(:versions, [:event, :item_type])
# create index(:versions, [:item_type, :inserted_at])
end
end
As we can see, several fields are created
in a table called "versions".
You can find the details of each one in
https://github.com/izelnakri/paper_trail#papertrailversion-fields.
We are going to be focusing on the following five:
- event: either "insert", "update" or "delete" operation
- item_type: model name of the mutated record (in our case, it's
item) - item_id: the
idof the changed record. - item_changes:
jsonobject with the changed fields - originator_id: foreign key referencing the creator of the change.
This is usually a
userin aUserstable. We don't have this, so we are just going to reference theItemstable.
item_type,item_idanditem_changesare calleditemby default, not because our table is calledItem. It is pure coincidence.
2.2 Creating versions table
Before executing the generated migration,
we are going to change the originator_id
reference from users
(this table doesn't exist)
to items.
add :originator_id, references(:items) # you can change :users to your own foreign key constraint
If you now run mix ecto.migrate,
the table versions should be created.
The terminal will output these lines:
13:46:48.334 [info] == Running 20221220130334 Repo.Migrations.AddVersions.change/0 forward
13:46:48.337 [info] create table versions
13:46:48.346 [info] create index versions_originator_id_index
13:46:48.349 [info] create index versions_item_id_item_type_index
13:46:48.354 [info] == Migrated 20221220130334 in 0.0s
If you check the tables
using DBeaver,
you will note that a versions table
is created with the specified fields.
DBeaver is a PostgreSQL GUI. If you don't have this installed, we highly recommend you doing so.
3. Making PaperTrail handle mutations
We are ready to track these changes!
According to PaperTrail documentation,
only the insert, update and delete methods
are trackable.
For this, we just need to make Repo operations
pass through PaperTrail.
Head on to lib/app/todo.ex
and do just that!
Import PaperTrail
and change the create_item/1,
updated_item/2
and delete_item/1 functions
to the following.
defmodule App.Todo do
alias PaperTrail
def create_item(attrs \\ %{}) do
%Item{}
|> Item.changeset(attrs)
|> PaperTrail.insert()
end
def update_item(%Item{} = item, attrs) do
item
|> Item.changeset(attrs)
|> PaperTrail.update()
end
def delete_item(%Item{} = item) do
PaperTrail.delete(item)
end
end
Instead of calling Repo.insert(), for example,
we are using PaperTrail.insert() method,
which will handle tracking the changes
for us.
Do notice these operations will now return a different object. For example, on success, the return will be:
{:ok,
%{
model: %Item{__meta__: Ecto.Schema.Metadata<:loaded, "items"> ...},
version: %PaperTrail.Version{__meta__: Ecto.Schema.Metadata<:loaded, "versions">...}
}
}
It now returns the changeset
inside :model
and the referring versions record
inside :version.
If you run mix phx.server
and visit localhost:4000,
the app should work normally.
However, every time the user creates an item, edits an item or deletes one, this action is recorded.
3.1 Fixing tests
If you run mix test,
you will notice we broke quite a few tests. 😅
Finished in 0.2 seconds (0.08s async, 0.1s sync)
32 tests, 14 failures
Randomized with seed 149737
Do not worry, though!
We can fix it!
The reason these tests are failing
is because of what we said earlier:
the object returned from the repo
operation is no longer a changeset
but a map with {model: changeset, version: version_obj}.
Many of these tests is using the item_fixture/1
function inside test/support/fixtures/todo_fixtures.ex.
If instead of returning the map,
we return the :model property,
most of these failing tests should be corrected!
Open the file and change the function to this:
def item_fixture(attrs \\ %{}) do
{:ok, item} =
attrs
|> Enum.into(%{
person_id: 42,
status: 42,
text: "some text"
})
|> App.Todo.create_item()
Map.get(item, :model)
end
If we run mix test:
Finished in 0.2 seconds (0.09s async, 0.1s sync)
32 tests, 3 failures
We still got three tests to fix!
Luckily, it's also really simple!
Open the test/app/todo_test.exs
and change the following tests to.
test "create_item/1 with valid data creates a item" do
valid_attrs = %{person_id: 42, status: 42, text: "some text"}
assert {:ok, ret} = Todo.create_item(valid_attrs)
item = ret.model
assert item.person_id == 42
assert item.status == 42
assert item.text == "some text"
end
test "update_item/2 with valid data updates the item" do
item = item_fixture()
update_attrs = %{person_id: 43, status: 43, text: "some updated text"}
assert {:ok, ret} = Todo.update_item(item, update_attrs)
item = ret.model
assert item.person_id == 43
assert item.status == 43
assert item.text == "some updated text"
end
test "delete_item/1 deletes the item" do
item = item_fixture()
assert {:ok, %{model: %Item{}, version: %PaperTrail.Version{}}} = Todo.delete_item(item)
assert_raise Ecto.NoResultsError, fn -> Todo.get_item!(item.id) end
end
We are now checking the changeset
with item = ret.model,
because the object returned
from database operations have changed
since we are now using PaperTrail.
If we run mix test:
................................
Finished in 0.3 seconds (0.1s async, 0.2s sync)
32 tests, 0 failures
Everything is fixed! Hurray! 🎉
4. Showing changes to item in UI
So far, we can check these changes internally.
But it would be great to have the ability
to see the changes an item has been subjected to
by double-clicking them.
This is the "hardest" part of the tutorial but it will be well worth it!
4.1 Fetching last changes of an item
PaperTrail has a way to fetch the
last and first version
(the name of the change record inside versions table)
but no way of querying multiple or even filtering.
We will need to do that job ourselves.
For this, we are going to add a Changes schema
that will be then displayed on the UI.
inside lib/app/todo, create a file change.ex
and add the following code.
defmodule App.Todo.Change do
use Ecto.Schema
schema "versions" do
field :event, :string
field :item_type, :string
field :item_id, :integer
field :item_changes, :map
field :originator_id, :integer
field :origin, :string
field :meta, :map
field :inserted_at, :naive_datetime
end
end
We just added the same fields that are present
inside the versions table.
Inside lib/app, add a new file change.ex.
Inside this file,
we are going to add the function list_last_20_changes/1.
This function will return the last 20 changes
made on a given item.
defmodule App.Change do
@moduledoc """
The Todo context.
"""
import Ecto.Query, only: [from: 2]
alias App.Todo.Change
alias PaperTrail
alias App.Repo
@doc """
Returns the list of changes for a specific item.
## Examples
iex> list_changes(12)
[%Change{}, ...]
"""
def list_last_20_changes(id) do
from(record in Change, where: record.item_id == ^id, select: record, limit: 20, order_by: [desc: :inserted_at])
|> Repo.all()
end
end
4.2 Adding list of changes to connection assigns
To show an item's changes in the UI,
we need to make it available inside the view controller.
Currently, to edit an item, we double-click it.
We can leverage this behaviour
to also show the actions next to it.
When editing a timer,
the index function is called
inside lib/app/controllers/item_controller.ex.
Whenever an id is in the URL params,
it means an item is being edited.
Therefore, we can add a changes array
to the connection assigns
referring to the tracking changes of the item
that is being edited.
Inside item_controller.ex,
change the index/2 function to the following.
def index(conn, params) do
item =
if not is_nil(params) and Map.has_key?(params, "id") do
Todo.get_item!(params["id"])
else
%Item{}
end
changes = case Map.has_key?(params, "id") do
true ->
Change.list_last_20_changes(params["id"])
false -> []
end
items = Todo.list_items()
changeset = Todo.change_item(item)
render(conn, "index.html",
items: items,
changeset: changeset,
editing: item,
filter: Map.get(params, "filter", "all"),
changes: changes
)
end
We are now fetching the item changes
every time it is being edited!
4.3 Changing the UI
Now that we have the changes array assigned to the connection assigns, we can use them in our UI!
Since we are using Phoenix 1.7rc, it has TailwindCSS installed by default. So we can use it!
Find the lib/app_web/controllers/item_html/index.html.heex
file and make the following changes.
Locate <section class="todoapp">
and change it to:
<section class="todoapp h-fit w-[100%] mb-12 md:w-[50%] md:mb-0 md:ml-2">
Now wrap all the contents of the file in a <div>
with the following classes.
<div class='w-full flex flex-col md:flex-row md:justify-around'>
<section></section>
<!-- adding a section here -->
</div>
We are creating two flex columns: one for the section pertaining to adding/editing/deleting items (what we already have) and another section to showcase the changes of the item (what we are going to build).
Inside the wrapper div we have created,
add a second section,
next to the already existing one.
This will be the section displaying the changes.
Add the following code:
<section class="max-w-full md:w-[50%]">
<header class="header mt-12">
<h1 class="text-5xl text-center text-[#af2f2f26]">Actions</h1>
<h3 class="text-lg text-center text-[#af2f2f9f]">last 20 actions of chosen item</h3>
</header>
<div class="mt-12 pr-1 pl-4">
<%= for change <- @changes do %>
<div class="mb-4">
<div>
<span class="text-blue-500 "><b>event:</b> <%= change.event %></span>
<span class='ml-2 mr-2'>|</span>
<span><b>type:</b> <%= change.item_type %></span>
<span class='ml-2 mr-2'>|</span>
<span><b>item_id:</b> <%= change.item_id %></span>
</div>
<div>
<span class="text-xs"><b>item_changes:</b> <%= inspect(Map.take(change.item_changes, ["inserted_at", "status", "text"]))%></span>
</div>
</div>
<% end %>
</div>
</section>
Here we are iterating over the @changes array
inside the connection assigns
and displaying the information to the user.
The last change we need to do is to
remove the width constraints of the body
in the assets/css/todomvc-app.css file.
Locate body {}
and delete min-width and max-width fields.
It should now look like this.
body {
font: 14px 'Helvetica Neue', Helvetica, Arial, sans-serif;
line-height: 1.4em;
background: #f5f5f5;
color: #4d4d4d;
margin: 0 auto;
-webkit-font-smoothing: antialiased;
-moz-osx-font-smoothing: grayscale;
font-weight: 300;
}
If you run mix phx.server,
your app should now be working!
4.4 Running tests
If we run MIX_ENV=test mix coveralls.html,
we will observe this output:
................................
Finished in 0.2 seconds (0.08s async, 0.1s sync)
32 tests, 0 failures
Randomized with seed 852095
----------------
COV FILE LINES RELEVANT MISSED
100.0% lib/app/change.ex 24 1 0
100.0% lib/app/todo.ex 112 7 0
100.0% lib/app/todo/change.ex 15 1 0
100.0% lib/app/todo/item.ex 19 2 0
100.0% lib/app_web/controllers/error_html.ex 19 1 0
100.0% lib/app_web/controllers/error_json.ex 15 1 0
100.0% lib/app_web/controllers/item_controller. 106 34 0
100.0% lib/app_web/controllers/item_html.ex 40 12 0
0.0% lib/app_web/gettext.ex 24 0 0
100.0% lib/app_web/router.ex 27 5 0
[TOTAL] 100.0%
That's right! No need to add new tests!
We fixed all of them prior.
This shows that integrating PaperTrail
into your project should have minimal impact
and you can quickly get up-and-running
with a few lines of code!
5. Adding further information on version records
We touched on a basic usage of PaperTrail
in this walk-through.
However, there are a handful of features
that might be useful for other use-cases
to add extra information to each
change record.
5.1 Version origin references
Amongst the fields in the versions schema,
there is one called origin.
The user can add a string
pertaining to the origin of the mutation.
PaperTrail.update(changeset, origin: "migration")
5.2 Adding metadata
Similarly, there is a meta field
in which metadata can be added.
PaperTrail.update(changeset, meta: %{data: "some data"})
There are more advanced features that can be used
(e.g. multi-tenancy).
You can find more about these
in the PaperTrail docs