How to use MongoDB With Elixir - Revisited

Written by Pete Corey on Dec 5, 2016.

I recently wrote an article on how to use MongoDB with Elixir. Since that article was released, changes have been made to the MongoDB Elixir package.

In many ways, these changes make the package more approachable and flexible for developers, but they left my old instructions outdated and incomplete.


Previous versions of the MongoDB driver (< 0.2) required that you build your own MongoPool module somewhere in your project:


defmodule MongoPool do
  use Mongo.Pool, name: __MODULE__, adapter: Mongo.Pool.Poolboy
end

This MongoPool module set up your connection pooling (using Mongo.Pool.Poolboy) and was the main process you would instantiate to establish a connection with your MongoDB database:


{:ok, _} = MongoPool.start_link(database: "test")

Once the connection was established, you could query the database through the connection pool, without specifying the PID if the pool’s process:


MongoPool
|> Mongo.find("collection", %{ "foo" => "bar" })
|> Enum.to_list

However, with version 0.2 of the MongoDB driver, things have changed.

Now, you instantiate your connection to your MongoDB database by spinning up the Mongo process directly:


{:ok, mongo_pid} = Mongo.start_link(database: "test")

The first argument to Mongo.find/Mongo.insert_one/etc… is now the Mongo process ID:


mongo_pid
|> Mongo.find("collection", %{ "foo" => "bar" })
|> Enum.to_list

Alternatively, you can name your Mongo process to avoid having to pass the mongo_pid around your application:


{:ok, _} = Mongo.start_link(database: "test", name: :mongo)

:mongo
|> Mongo.find("collection", %{ "foo" => "bar" })
|> Enum.to_list

Out of the box, this will establish a single connection to the database. To enable connection pooling, like we had with our old MongoPool, we need to specify how we want our pooling handled when we spin up our Mongo process:


{:ok, _} = Mongo.start_link(database: "test", name: :mongo, pool: DBConnection.Poolboy)

You then need to specify the pool module you’re using when running MongoDB operations:


:mongo
|> Mongo.find("collection", %{ "foo" => "bar" }, pool: DBConnection.Poolboy)
|> Enum.to_list

The heart of these changes is that the mongodb driver package is now using the DBConnection package, instead of wrapping Poolboy itself.

See the changelog and the GitHub documentation and the Hex documentation for more details.

Using Apollo Client with Elixir's Absinthe

Written by Pete Corey on Nov 21, 2016.

There’s no doubt that GraphQL has been making waves in the web development community since it was announced, and for good reason! GraphQL helps decouple an application’s front-end from its back-end in amazingly flexible ways.

Unfortunately, React and Redux, the current go-to front-end frameworks for handling client-side state and interacting with a GraphQL server are cumbersome to use at best. Thankfully, the Apollo client, a new project from the Meteor Development Group, is trying to offer a more straight-forward, batteries included option for interfacing with GraphQL and managing your client-side state.

Let’s dig into how to set up a basic GraphQL server in Elixir using Absinthe, and how to interact with that server using the Apollo client.

Elixir’s Absinthe

Absinthe is a GraphQL implementation for Elixir. It lets you set up a GraphQL endpoint on your Elixir/Phoenix server.

Setting up Absinthe is a straight-forward process. To start, we’ll add dependencies on the absinthe and absinthe_plug Mix packages and fire up their corresponding applications:


defp deps do
  [ ...
   {:absinthe, "~> 1.2.0"},
   {:absinthe_plug, "~> 1.2.0"}]

applications: [ … :absinthe, :absinthe_plug]

Just like in the Absinthe tutorial, our next step is to set up our GraphQL types. We’ll create simple schemas for an author and a post:


object :author do
  field :id, :id
  field :first_name, :string
  field :last_name, :string
  field :posts, list_of(:post) do
    resolve fn author, _, _ ->
      {:ok, HelloAbsinthe.Schema.find_posts(author.id)}
    end
  end
end

object :post do
  field :id, :id
  field :title, :string
  field :author, :author do
    resolve fn post, _, _ ->
      {:ok, HelloAbsinthe.Schema.find_author(post.author.id)}
    end
  end
  field :votes, :integer
end

Next, we’ll define the types of queries we support. To keep things simple, we’ll add two basic queries. The first, posts, will return all posts in the system, and the second, author, will return an author for a given id:


query do
  field :posts, list_of(:post) do
    resolve &get_all_posts/2
  end

  field :author, type: :author do
    arg :id, non_null(:id)
    resolve &get_author/2
  end
end

To cut down on the number of moving parts in this example, we’ll write our two resolver functions to return a set of hard-coded posts and authors, rather than pulling them from some external data source:


@posts [
  %{id: 1, title: "GraphQL Rocks",           votes: 3, author: %{id: 1}},
  %{id: 2, title: "Introduction to GraphQL", votes: 2, author: %{id: 2}},
  %{id: 3, title: "Advanced GraphQL",        votes: 1, author: %{id: 1}}
]

@authors [
  %{id: 1, first_name: "Sashko", last_name: "Stubailo"},
  %{id: 2, first_name: "Tom",    last_name: "Coleman"},
]

...

def get_all_posts(_args, _info) do
  {:ok, @posts}
end

def get_author(%{id: id}, _info) do
  {:ok, find_author(id)}
end

def find_author(id) do
  Enum.find(@authors, fn author -> author.id == id end)
end

def find_posts(author_id) do
  Enum.find(@posts, fn post -> post.author.id == author_id end)
end

Now all we need to do is tell Absinthe that we want our GraphQL endpoint to listen on the "/graphql" route and that we want it to use our newly defined schemas and queries:


forward "/graphql", Absinthe.Plug, schema: HelloAbsinthe.Schema

And that’s it! Now we can send our server GraphQL queries and it will process them and send back the result.

Let’s move on to setting up Apollo on the front-end.

Apollo Client

If you haven’t noticed already, we’re basing this example off of the query example on the Apollo Developer page.

Before we continue with their example, we need to set up React in our application. Since we started with a fresh Phoenix project (mix phoenix.new), we’ll need to install install some NPM dependencies to work with React, Apollo, etc…:


npm install --save react react-dom apollo-client react-apollo \
                         graphql-tag babel-preset-react

Next, we’ll need to tell Brunch how to we want our ES6 transpiled by tweaking our Babel options in brunch-config.js:


plugins: {
  babel: {
    presets: ["es2015", "react"],
    ...

The last thing we need to do is replace the HTML our Phoenix application generates (in app.html.eex) with an empty <div> to hold our React application:


 <div id="app"></div>

Now we can copy over the <PostList> component from the Apollo example. We’ll throw it in a file called PostList.jsx.

Lastly, we’ll create an instance of ApolloClient and wire up the <PostList> component to our container <div> in our app.js:


const client = new ApolloClient();

ReactDOM.render(
  <ApolloProvider client={client}>
    <PostList />
  </ApolloProvider>,
  document.getElementById("app")
);

And that’s it! When our application reloads, we’ll see all of the hard-coded author and post data from our server loaded up and rendered on the client.

How it Works

This is obviously a drastically over-simplified example of what GraphQL can do, but it’s a good jumping off point. Let’s see how all of it ties together, starting on the client.

The <PostList> component we pulled from the Apollo example is a simple component that expects to be passed a loading boolean and a list of posts inside of a data property.

If loading is true, we’ll show a loading message. Otherwise, we’ll render the list of posts:


function PostList({ data: { loading, posts } }) {
  if (loading) {
    return <div>Loading</div>;
  } else {
    return (<ul>{posts.map(post => … )} </ul>);
  }
}

Where do loading and posts come from? The loading field is controlled by the Apollo client. When we’re waiting on the response for a GraphQL query, loading will be true. The posts field actually comes directly from the response to our GraphQL query.

When we export PostList, we actually wrap it in a GraphQL query that describes the data this component needs to render:


export default graphql(gql`
  query allPosts {
    posts {
      id
      title
      votes
      author {
        id
        firstName
        lastName
      }
    }
  }
`)(PostList);

The shape of a GraphQL query’s response maps directly to the shape of the query itself. Notice how we’re asking for a set of posts. We want each post to be returned with an id, title, votes, and an author object, complete with id, firstName, and lastName.

Our response will look exactly like this:


{
  posts: [
    {
      id: 1,
      title: "GraphQL Rocks",
      votes: 3,
      author: {
        id: 1,
        firstName: "Sashko",
        lastName: "Stubailo"
      }
    },
    ...
  ]
}

This is the power of GraphQL. It inverts the normal query/result relationship between the client and the server. The client tells the server exactly what it needs, and that exact data is returned from the query. No more, no less.

Apollo takes that client-first mentality even further. With Apollo, each component tells the server exactly what it needs and manages it’s data lifecycle entirely on its own, independent from other components in the application.

Final Thoughts

I’m really excited about the combination of an Elixir/Absinthe back-end driving an Apollo-powered client front-end.

I’ve only just started playing with this combination, but I hope to start building out more complex and realistic applications to see if it lived up to my hopes and expectations.

Be sure to check out the entire project on GitHub. Have you used Absinthe or any part of the Apollo stack? If so, shoot me an email and let me know your opinions!

Phoenix Todos - Public and Private Lists

This post is written as a set of Literate Commits. The goal of this style is to show you how this program came together from beginning to end.

Each commit in the project is represented by a section of the article. Click each section's header to see the commit on Github, or check out the repository and follow along.

Written by Pete Corey on Nov 16, 2016.

Make Private

Now that our channel connection can be authenticated, we can gives users the ability to make their lists private.

To start, we’ll add a "make_private" channel event handler. This handler will call List.make_private and set the list’s user_id equal to the socket’s currently authenticated user:


list = get_user_id(socket)
|> List.make_private(list_id)
|> Repo.preload(:todos)

Once we’ve done that, we’ll broadcast a "update_list" event to all connected clients.

However, if a list becomes private, we’ll want to remove it from other users’ clients, instead of just showing the change. To do this, we’ll have to intercept all outbound "update_list" events:


intercept ["update_list"]

def handle_out("update_list", list, socket) do

If a user has permission to see the outgoing list, we’ll push another "update_list" event. Otherwise, we’ll push a "remove_list" event:


case List.canView?(get_user_id(socket), list) do
  true ->
    push(socket, "update_list", list)
  false ->
    push(socket, "remove_list", list)
end

After wiring up all of the necessary Redux plumbing to call our "make_private" event, the functionality it complete.

web/channels/list_channel.ex

... + intercept ["update_list"] + defp get_user_id(socket) do ... + def handle_in("make_private", %{ + "list_id" => list_id, + }, socket) do + list = get_user_id(socket) + |> List.make_private(list_id) + |> Repo.preload(:todos) + + broadcast! socket, "update_list", list + + {:noreply, socket} + end + def handle_in("delete_todo", %{ ... + def handle_out("update_list", list, socket) do + case List.canView?(get_user_id(socket), list) do + true -> + push(socket, "update_list", list) + false -> + push(socket, "remove_list", list) + end + {:noreply, socket} + end + end

web/models/list.ex

... @required_fields ~w(name incomplete_count) - @optional_fields ~w() + @optional_fields ~w(user_id) ... + def make_private(user_id, id) do + Repo.get(PhoenixTodos.List, id) + |> changeset(%{ + user_id: user_id + }) + |> Repo.update! + end + def delete_todo(todo_id) do ... + def canView?(_, %{user_id: nil}), do: true + def canView?(user_id, %{user_id: user_id}), do: true + def canView?(_, _), do: false + end

web/static/js/actions/index.js

... +export const MAKE_PRIVATE_REQUEST = "MAKE_PRIVATE_REQUEST"; +export const MAKE_PRIVATE_SUCCESS = "MAKE_PRIVATE_SUCCESS"; +export const MAKE_PRIVATE_FAILURE = "MAKE_PRIVATE_FAILURE"; + export const DELETE_TODO_REQUEST = "DELETE_TODO_REQUEST"; ... channel.on("update_list", list => { + console.log("update_list", list) dispatch(updateList(list)); ... +export function makePrivateRequest() { + return { type: MAKE_PRIVATE_REQUEST }; +} + +export function makePrivateSuccess() { + return { type: MAKE_PRIVATE_SUCCESS }; +} + +export function makePrivateFailure() { + return { type: MAKE_PRIVATE_FAILURE }; +} + +export function makePrivate(list_id) { + return (dispatch, getState) => { + const { channel } = getState(); + dispatch(makePrivateRequest()); + channel.push("make_private", { list_id }) + .receive("ok", (list) => { + dispatch(makePrivateSuccess()); + }) + .receive("error", () => dispatch(makePrivateFailure())) + .receive("timeout", () => dispatch(makePrivateFailure())); + } +} + export function deleteTodoRequest() {

web/static/js/components/ListHeader.jsx

... } else { - makePrivate.call({ listId: list._id }, alert); + this.props.makePrivate(list.id); }

web/static/js/pages/ListPage.jsx

... deleteList, + makePrivate, deleteTodo ... updateName={this.props.updateName} - deleteList={this.props.deleteList}/> + deleteList={this.props.deleteList} + makePrivate={this.props.makePrivate} + /> <div className="content-scrollable list-items"> ... }, + makePrivate: (list_id) => { + return dispatch(makePrivate(list_id)); + }, deleteTodo: (todo_id) => {

Make Public

Just as we let users make their lists private, we need to let them make their private lists public again.

We’ll do this by adding a "make_public" channel event that sets the user_id field on the specified list to nil and broadcasts an "update_list" event.


list = List.make_public(list_id)
|> Repo.preload(:todos)

broadcast! socket, "update_list", list

Unfortunately, this introduces a situation where lists are added back into the UI through a "update_list" event rather than a "add_list" event.

To handle this, we need to check if the "UPDATE_LIST" Redux reducer actually found the list it was trying to update. If it didn’t, we’ll push the list to the end of the list, adding it to the UI:


if (!found) {
  lists.push(action.list);
}

And with that, users can make their private lists public.

web/channels/list_channel.ex

... + def handle_in("make_public", %{ + "list_id" => list_id, + }, socket) do + list = List.make_public(list_id) + |> Repo.preload(:todos) + + broadcast! socket, "update_list", list + + {:noreply, socket} + end + def handle_in("delete_todo", %{

web/models/list.ex

... + def make_public(id) do + Repo.get(PhoenixTodos.List, id) + |> changeset(%{ + user_id: nil + }) + |> Repo.update! + end + def delete_todo(todo_id) do

web/static/js/actions/index.js

... +export const MAKE_PUBLIC_REQUEST = "MAKE_PUBLIC_REQUEST"; +export const MAKE_PUBLIC_SUCCESS = "MAKE_PUBLIC_SUCCESS"; +export const MAKE_PUBLIC_FAILURE = "MAKE_PUBLIC_FAILURE"; + export const DELETE_TODO_REQUEST = "DELETE_TODO_REQUEST"; ... channel.on("update_list", list => { - console.log("update_list", list) dispatch(updateList(list)); ... +export function makePublicRequest() { + return { type: MAKE_PUBLIC_REQUEST }; +} + +export function makePublicSuccess() { + return { type: MAKE_PUBLIC_SUCCESS }; +} + +export function makePublicFailure() { + return { type: MAKE_PUBLIC_FAILURE }; +} + +export function makePublic(list_id) { + return (dispatch, getState) => { + const { channel } = getState(); + dispatch(makePublicRequest()); + channel.push("make_public", { list_id }) + .receive("ok", (list) => { + dispatch(makePublicSuccess()); + }) + .receive("error", () => dispatch(makePublicFailure())) + .receive("timeout", () => dispatch(makePublicFailure())); + } +} + export function deleteTodoRequest() {

web/static/js/components/ListHeader.jsx

... if (list.user_id) { - makePublic.call({ listId: list._id }, alert); + this.props.makePublic(list.id); } else {

web/static/js/pages/ListPage.jsx

... makePrivate, + makePublic, deleteTodo ... makePrivate={this.props.makePrivate} + makePublic={this.props.makePublic} /> ... }, + makePublic: (list_id) => { + return dispatch(makePublic(list_id)); + }, deleteTodo: (todo_id) => {

web/static/js/reducers/index.js

... case UPDATE_LIST: + let found = false; let lists = state.lists.map(list => { - return list.id === action.list.id ? action.list : list; + if (list.id === action.list.id) { + found = true; + return action.list; + } + else { + return list; + } }); + if (!found) { + lists.push(action.list); + } return Object.assign({}, state, { lists });

Final Thoughts

At this point, we’ve roughly recreated all of the features of the Meteor Todos application in Phoenix and Elixir.

I’ll be the first to admit that there are many problems with the project as it currently stands. My solution to channel authentication isn’t the best, many channel events aren’t making proper authorization checks, the front-end Redux architecture is awful, etc… That being said, this was a fantastic learning experience.

Building out Meteor-esque functionality in Phoenix is definitely more work than using Meteor, but I still believe that the benefits of using an Elixir backend outweigh the drawbacks. With a little more effort, I think I’ll be able to reduce the upfront burden quite a bit through packages and libraries.

Expect many upcoming articles discussing what I’ve learned from this conversion and how to approach building Elixir and Phoenix applications from the perspective of a Meteor developer.