How to Use MongoDB with Elixir

Written by Pete Corey on Oct 31, 2016.

Many of the application’s I’ve developed for myself and for clients over the recent years are intimately tied to MongoDB. This means that any new technology stack I experiment with need to be able to work well with this database.

Elixir is no exception to this rule.

Thankfully, Elixir and the Phoenix framework are database agnostic. They don’t require you to be tied to a single database, and even offer options for interfacing with a wide variety of databases.

Let’s dig into how we can use MongoDB in an Elixir application.

This article was written with version ~0.1 of the MongoDB driver in mind. For instructions on using version 0.2 of the MongoDB driver, see How to Use MongoDB with Elixir - Revisited.

Ecto Adapter

A very common way of interacting with a database in an Elixir application is to use the Ecto package.

Ecto acts as a repository layer around your database. It lets you write queries in a unified languages, and supports communicating with many types of databases through “adapters”.

The Mongo.Ecto package is the Ecto adapter for MongoDB. Unfortunately, Mongo.Ecto is currently in a state of flux.

Mongo.Ecto is currently incompatible with Ecto 1.1. On top of that, work to support Ecto 2.0 is very much a work in progress.

All of this is to say that integrating with MongoDB through Ecto is not currently an option if you’re looking for a low-friction, fully supported solution.

MongoDB Driver

Under the hood, the Mongo.Ecto adapter makes use of Eric Meadows-Jönsson’s MongoDB driver package.

While Mongo.Ecto is in a state of flux, the MongoDB driver package seems to be stable and functional.

Setting up the MongoDB driver in your Elixir application is a simple process. Get started by following the documentation on GitHub. Once you’ve defined your MongoPool module, you can start the process and make your queries:


{:ok, _} = MongoPool.start_link(database: "test")

MongoPool
|> Mongo.find("collection", %{ "foo" => "bar" })
|> Enum.to_list

Database options can be passed into the connection pool when you start_link. For example, to connect to a "meteor" database on localhost:3001, you could initiate your connection pool with these options:


{:ok, _} = MongoPool.start_link(database: "meteor", port: 3001)

You can find all available options in the Mongo.Connection module, or through iex:


iex -S mix
> h Mongo.Connection.start_link

Final Thoughts

It’s a shame that the Mongo.Ecto adapter isn’t in a stable state. While the MongoDB driver works beautifully, it would be nice to leverage the unified interface of Ecto when building applications.

When using the MongoDB driver directly, you need to concern yourself with tightly coupling your application to your persistence method. Ecto provides a nice layer of decoupling between the two.

Additionally, using the MongoDB driver directly opens yourself up to the possibility of being vulnerable to NoSQL injection attacks. We’ll drive into that topic next week.

Phoenix Todos - Updating and Deleting

This post is written as a set of Literate Commits. The goal of this style is to show you how this program came together from beginning to end.

Each commit in the project is represented by a section of the article. Click each section's header to see the commit on Github, or check out the repository and follow along.

Written by Pete Corey on Oct 26, 2016.

Update List Name

The next piece of functionality we need to knock out is the ability to rename lists.

To do this, we’ll create a helper method on our List model called update_name. This method simply changes the name of the given List:


Repo.get(PhoenixTodos.List, id)
|> changeset(%{
  name: name
})
|> Repo.update!

We’ll create a new channel event, "update_name", to handle name change requests, and we’ll wire up a Redux thunk to push a "update_name" event onto our channel:


channel.push("update_name", { list_id, name })
  .receive("ok", (list) => {
    dispatch(updateNameSuccess());
  })
  .receive("error", () => dispatch(updateNameFailure()))
  .receive("timeout", () => dispatch(updateNameFailure()));

After wiring up the rest of the necessary Redux plumbing, we’re able to update the names of our lists.

web/channels/list_channel.ex

... + def handle_in("update_name", %{ + "list_id" => list_id, + "name" => name + }, socket) do + list = List.update_name(list_id, name) + |> Repo.preload(:todos) + + broadcast! socket, "update_list", list + + {:noreply, socket} + end + end

web/models/list.ex

... + def update_name(id, name) do + Repo.get(PhoenixTodos.List, id) + |> changeset(%{ + name: name + }) + |> Repo.update! + end + def set_checked_status(todo_id, checked) do

web/static/js/actions/index.js

... +export const UPDATE_NAME_REQUEST = "UPDATE_NAME_REQUEST"; +export const UPDATE_NAME_SUCCESS = "UPDATE_NAME_SUCCESS"; +export const UPDATE_NAME_FAILURE = "UPDATE_NAME_FAILURE"; + export function signUpRequest() { ... } + +export function updateNameRequest() { + return { type: UPDATE_NAME_REQUEST }; +} + +export function updateNameSuccess() { + return { type: UPDATE_NAME_SUCCESS }; +} + +export function updateNameFailure() { + return { type: UPDATE_NAME_FAILURE }; +} + +export function updateName(list_id, name) { + return (dispatch, getState) => { + const { channel } = getState(); + dispatch(updateNameRequest()); + channel.push("update_name", { list_id, name }) + .receive("ok", (list) => { + dispatch(updateNameSuccess()); + }) + .receive("error", () => dispatch(updateNameFailure())) + .receive("timeout", () => dispatch(updateNameFailure())); + } +}

web/static/js/components/ListHeader.jsx

... this.setState({ editing: false }); - updateName.call({ - listId: this.props.list._id, - newName: this.refs.listNameInput.value, - }, alert); + this.props.updateName(this.props.list.id, this.refs.listNameInput.value); }

web/static/js/pages/ListPage.jsx

... addTask, - setCheckedStatus + setCheckedStatus, + updateName } from "../actions"; ... <div className="page lists-show"> - <ListHeader list={list} addTask={this.props.addTask}/> + <ListHeader list={list} addTask={this.props.addTask} updateName={this.props.updateName}/> <div className="content-scrollable list-items"> ... return dispatch(setCheckedStatus(todo_id, status)); + }, + updateName: (list_id, name) => { + return dispatch(updateName(list_id, name)); }

Delete Lists

Let’s give users the ability to delete lists in our application.

We’ll start by creating a delete function in our List model. delete simply deletes the specified model object:


Repo.get(PhoenixTodos.List, id)
|> Repo.delete!

We’ll call List.delete from a "delete_list" channel event handler. Once deleted, we’ll also broadcast a "remove_list" event down to all connectd clients:


list = List.delete(list_id)
|> Repo.preload(:todos)

broadcast! socket, "remove_list", list

We’ll trigger this "delete_list" event with a Redux thunk:


channel.push("delete_list", { list_id, name })
  .receive("ok", (list) => {
    dispatch(deleteListSuccess());
  })
  .receive("error", () => dispatch(deleteListFailure()))
  .receive("timeout", () => dispatch(deleteListFailure()));

Lastly, we need to handle the new "remove_list" event that will be broadcast to all connected clients. We’ll set up a "remove_list" event listener on the client, and trigger a REMOVE_LIST action from the listener:


channel.on("remove_list", list => {
  dispatch(removeList(list));
});

The REMOVE_LISTENER action simply filters the specified list out of our application’s set of lists:


lists = state.lists.filter(list => {
  return list.id !== action.list.id
});

After combining all of that with some Redux plumbing, users can delete lists.

web/channels/list_channel.ex

... + def handle_in("delete_list", %{ + "list_id" => list_id, + }, socket) do + list = List.delete(list_id) + |> Repo.preload(:todos) + + broadcast! socket, "remove_list", list + + {:noreply, socket} + end + end

web/models/list.ex

... + def delete(id) do + Repo.get(PhoenixTodos.List, id) + |> Repo.delete! + end + def set_checked_status(todo_id, checked) do

web/static/js/actions/index.js

... export const UPDATE_LIST = "UPDATE_LIST"; +export const REMOVE_LIST = "REMOVE_LIST"; ... +export const DELETE_LIST_REQUEST = "DELETE_LIST_REQUEST"; +export const DELETE_LIST_SUCCESS = "DELETE_LIST_SUCCESS"; +export const DELETE_LIST_FAILURE = "DELETE_LIST_FAILURE"; + export function signUpRequest() { ... +export function removeList(list) { + return { type: REMOVE_LIST, list }; +} + export function connectSocket(jwt) { ... }) + channel.on("remove_list", list => { + dispatch(removeList(list)); + }); }; ... } + +export function deleteListRequest() { + return { type: DELETE_LIST_REQUEST }; +} + +export function deleteListSuccess() { + return { type: DELETE_LIST_SUCCESS }; +} + +export function deleteListFailure() { + return { type: DELETE_LIST_FAILURE }; +} + +export function deleteList(list_id, name) { + return (dispatch, getState) => { + const { channel } = getState(); + dispatch(deleteListRequest()); + channel.push("delete_list", { list_id, name }) + .receive("ok", (list) => { + dispatch(deleteListSuccess()); + }) + .receive("error", () => dispatch(deleteListFailure())) + .receive("timeout", () => dispatch(deleteListFailure())); + } +}

web/static/js/components/ListHeader.jsx

... if (confirm(message)) { // eslint-disable-line no-alert - remove.call({ listId: list._id }, alert); - /* this.context.router.push('/');*/ + this.props.deleteList(list.id); + this.context.router.push('/'); }

web/static/js/pages/ListPage.jsx

... setCheckedStatus, - updateName + updateName, + deleteList } from "../actions"; ... <div className="page lists-show"> - <ListHeader list={list} addTask={this.props.addTask} updateName={this.props.updateName}/> + <ListHeader list={list} + addTask={this.props.addTask} + updateName={this.props.updateName} + deleteList={this.props.deleteList}/> <div className="content-scrollable list-items"> ... return dispatch(updateName(list_id, name)); + }, + deleteList: (list_id) => { + return dispatch(deleteList(list_id)); }

web/static/js/reducers/index.js

... UPDATE_LIST, + REMOVE_LIST, JOIN_LISTS_CHANNEL_SUCCESS, ... return Object.assign({}, state, { lists }); + case REMOVE_LIST: + lists = state.lists.filter(list => { + return list.id !== action.list.id + }); + return Object.assign({}, state, { lists }); case CONNECT_SOCKET:

Delete Todos

Next, we’ll give users the ability to delete completed todos from their lists.

We’ll start by creating a delete_todo helper in our List model. This method deletes the specified Todo:


todo = Repo.get(PhoenixTodos.Todo, todo_id)
|> Repo.preload(:list)

Repo.delete!(todo)

It’s also interesting to note that the delete_todo helper returns the parent list of the task:


todo.list

We use this returned list in our "delete_todo" channel event handler to broadcast an "udpate_list" event to all connected clients:


list = List.delete_todo(todo_id)
|> Repo.preload(:todos)

broadcast! socket, "update_list", list

We’ll kick off this "delete_todo" event with a Redux thunk called deleteTodo:


channel.push("delete_todo", { todo_id, name })
  .receive("ok", (list) => {
    dispatch(deleteTodoSuccess());
  })
  .receive("error", () => dispatch(deleteTodoFailure()))
  .receive("timeout", () => dispatch(deleteTodoFailure()));

And with a little more Redux plumbing, users can remove completed todo items.

web/channels/list_channel.ex

... + def handle_in("delete_todo", %{ + "todo_id" => todo_id, + }, socket) do + list = List.delete_todo(todo_id) + |> Repo.preload(:todos) + + broadcast! socket, "update_list", list + + {:noreply, socket} + end + end

web/models/list.ex

... + def delete_todo(todo_id) do + todo = Repo.get(PhoenixTodos.Todo, todo_id) + |> Repo.preload(:list) + + Repo.delete!(todo) + + todo.list + end + def set_checked_status(todo_id, checked) do

web/static/js/actions/index.js

... +export const DELETE_TODO_REQUEST = "DELETE_TODO_REQUEST"; +export const DELETE_TODO_SUCCESS = "DELETE_TODO_SUCCESS"; +export const DELETE_TODO_FAILURE = "DELETE_TODO_FAILURE"; + export function signUpRequest() { ... } + +export function deleteTodoRequest() { + return { type: DELETE_TODO_REQUEST }; +} + +export function deleteTodoSuccess() { + return { type: DELETE_TODO_SUCCESS }; +} + +export function deleteTodoFailure() { + return { type: DELETE_TODO_FAILURE }; +} + +export function deleteTodo(todo_id, name) { + return (dispatch, getState) => { + const { channel } = getState(); + dispatch(deleteTodoRequest()); + channel.push("delete_todo", { todo_id, name }) + .receive("ok", (list) => { + dispatch(deleteTodoSuccess()); + }) + .receive("error", () => dispatch(deleteTodoFailure())) + .receive("timeout", () => dispatch(deleteTodoFailure())); + } +}

web/static/js/components/TodoItem.jsx

... deleteTodo() { - remove.call({ todoId: this.props.todo.id }, alert); + this.props.deleteTodo(this.props.todo.id); }

web/static/js/pages/ListPage.jsx

... updateName, - deleteList + deleteList, + deleteTodo } from "../actions"; ... setCheckedStatus={this.props.setCheckedStatus} + deleteTodo={this.props.deleteTodo} /> ... return dispatch(deleteList(list_id)); + }, + deleteTodo: (todo_id) => { + return dispatch(deleteTodo(todo_id)); }

Final Thoughts

These changes wrap up all of the list and task CRUD functionality in our application. Again, it’s interesting to notice that the vast majority of the work required to implement these features lives in the front-end of the application.

Next week, we’ll work on introducing the concept of private lists into our application. Stay tuned!

A Five Minute Introduction to NoSQL Injection

Written by Pete Corey on Oct 24, 2016.

NoSQL injection is one of the most common vulnerabilities I find in Meteor applications.

I’m even starting to notice NoSQL injection vulnerabilities in other stacks and frameworks as well, like vanilla Node.js applications, and even Phoenix/Elixir applications.

I’ve written a short one page, five minute introduction to NoSQL injection for applications using MongoDB. My goal is to give you a quick primer on what the vulnerability looks like and how to fix it. That way, you’ll be able to identify it and seal up the vulnerability in your own applications.

I’m giving out a PDF version of the introduction to everyone who signs up for my newsletter. To sign up, enter your first name and email address below and hit subscribe!

Along with the NoSQL injection primer, you’ll receive weekly updates on all East5th articles and projects. Each newsletter also has a small selection of links and resources that I’ve found helpful when building secure, robust, maintainable software.

After you’ve signed up and read the NoSQL introduction, respond to the email and let me know if there’s any way I can help you build better software!