Exploring the Bitcoin Blockchain with Elixir and Phoenix

Written by Pete Corey on Sep 18, 2017.

Earlier this month we dove into the brave new world of Bitcoin development by writing an Elixir module that could communicate with a Bitcoin full node. At this end of the day, we had a small handful of Elixir functions that could retrieve some basic information about the Bitcoin blockchain.

Let’s expand on that idea a bit.

In this article we’re going to use the Phoenix framework to build a bare-bones blockchain viewer. Let’s get to it!

Project Scaffolding

First things first, let’s create our new Phoenix project. We’ll be using Phoenix 1.3 and the new phx generators that shipped with it:


mix phx.new hello_blockchain --no-ecto

Once we’ve gone ahead and set up our new project, let’s add two new routes to our application. One for viewing block headers, and another for viewing full blocks:


mix phx.gen.html Blockchain Header headers --no-schema

mix phx.gen.html Blockchain Blocks blocks --no-schema

Notice that we specified --no-ecto when generating our new project, and --no-schema when generating our block and header resources. We don’t be needing Ecto in our blockchain viewer. All of the data we’re rendering lives in our full node!

Our Blockchain Context

When we generated our header and block resources, we also generated a Blockchain context module. Our context will be our interfacing with our Bitcoin full node.

Sound familiar?

That’s because we implemented the Blockchain context module in our last article! Let’s remove the auto-generated contents of Blockchain and copy over the bitcoin_rpc function.


def bitcoin_rpc(method, params \\ []) do
  with url <- Application.get_env(:hello_bitcoin, :bitcoin_url),
       command <- %{jsonrpc: "1.0", method: method, params: params},
       {:ok, body} <- Poison.encode(command),
       {:ok, response} <- HTTPoison.post(url, body),
       {:ok, metadata} <- Poison.decode(response.body),
       %{"error" => nil, "result" => result} <- metadata do
    {:ok, result}
  else
    %{"error" => reason} -> {:error, reason}
    error -> error
  end
end

Be sure to add dependencies on :httpoison and :poison, and set up your :bitcoin_url in your configuration file.

Once we’ve finished that, we’ll add four helper functions to our Blockchain module. we’ll use these functions to fetch the data we want to render with our blockchain viewer:


def getbestblockhash, do: bitcoin_rpc("getbestblockhash")

def getblockhash(height), do: bitcoin_rpc("getblockhash", [height])

def getblock(hash), do: bitcoin_rpc("getblock", [hash])

def getblockheader(hash), do: bitcoin_rpc("getblockheader", [hash])

Now that we have a functional context module, we can start wiring it up to our routes.

Routing

Armed with our newly generated block and header resources, let’s add their new routes to our router:


scope "/", HelloBlockchainWeb do
  pipe_through :browser
  resources "/", PageController, only: [:index]
  resources "/blocks", BlockController, only: [:index, :show]
  resources "/headers", HeaderController, only: [:index, :show]
end

For now, we’re only going to be handling the :index and :show routes for our blocks and headers.

Now that our routes are established, let’s move on to refactoring our controller modules. We’ll start with the BlockController.

We’re going to remove all of our controller functions except the index and show functions, which we’ll heavily modify.

When we hit the /blocks/ route, we’ll fall into the index controller function. We want this function to redirect us to the most recent block in the blockchain:


def index(conn, _params) do
  with {:ok, hash} <- Blockchain.getbestblockhash() do
    redirect(conn, to: block_path(conn, :show, hash))
  end
end

We use getbestblockhash to get the hash of the most recently validated Bitcoin block, and we redirect the user to the :show route, providing the resulting hash.

Our show controller function accepts the provided hash as an argument, fetches more information about the block from our full node, and finally renders the block using the show.html template:


def show(conn, %{"id" => hash}) do
  with {:ok, block} <- Blockchain.getblock(hash) do
    render(conn, "show.html", block: block)
  end
end

Similarly, the index function in the HeaderController redirects the user to the :show route of the most recently verified block:


def index(conn, _params) do
  with {:ok, hash} <- Blockchain.getbestblockhash() do
    redirect(conn, to: header_path(conn, :show, hash))
  end
end

While the show function fetches the relevant information using Blockchain.getblockheader and passes it into its show.html template:


def show(conn, %{"id" => hash}) do
  with {:ok, block} <- Blockchain.getblockheader(hash) do
    render(conn, "show.html", block: block)
  end
end

There’s one final route we need to implement. When a user hits our application for the first time, they’ll land on the Phoenix landing page. Instead, let’s show them the most recent block header:


def index(conn, _params) do
  with {:ok, hash} <- Blockchain.getbestblockhash() do
    redirect(conn, to: header_path(conn, :show, hash))
  end
end

Once again, we use Blockchain.getbestblockhash to fetch the most recently verified block hash from our Bitcoin full node. We use that hash to redirect the user to the header’s :show route.

With our routes properly configured, we can fetch data about any full block or block header just by knowings its hash.

Let’s more on to the final piece of the puzzle: rendering that data.

Templates

Our blockchain viewer now correctly routes the user to the appropriate :show route for either the block or block header they’re trying to view, and passes all relevant data to the corresponding template to render.

Now all we need to do is build out our templates!

To keep the scope of this article manageable, we’ll keep our user interface as minimal as possible. The more bare-bones way to render our blocks and headers, while still being meaningful to our users, is to render the data received from our controllers as JSON code blocks.

The simplest way to do this would be to dump the output of Poison.encode! into the DOM:


<code><%= Poison.encode!(@block, pretty: true) %></code>

The pretty: true option passed into Poison.encode! ensures that the resulting JSON string is nicely formatted. In our app.css file, we should set the white-space rule on <code> blocks to preserve this formatting:


code {
  white-space: pre !important;
}

Beautiful.

A basic block header.

A bit minimal, but beautiful none-the-less.


While dumping raw JSON into the DOM is informative, it’s not especially user-friendly. Let’s add an extra layer of interactivity to our blockchain viewer.

The blocks and block headers we receive from our Bitcoin full node come with previousblockhash field and (usually) a nextblockhash field. These hashes, as we would expect, point to the previous and next blocks in the blockchain, respectively. Let’s transform these hashes into links so users can easily navigate through the blockchain.

The first thing we’ll do is write a function in the corresponding view file that converts hashes into links. In our HeaderView module, our hash_link function would look like this:


defp hash_link(hash), do: "<a href='/headers/#{hash}'>#{hash}</a>"

Using this function, we can write a function that modifies our block header. It replaces the hashes in previousblockhash and nextblockhash with links to those blocks, and JSON encodes the resulting object:


def mark_up_block(block) do
  block
  |> Map.replace("previousblockhash", hash_link(block["previousblockhash"]))
  |> Map.replace("nextblockhash", hash_link(block["nextblockhash"]))
  |> Poison.encode!(pretty: true)
end

In our HTML template, we can replace the contents of our <code> block with the result of mark_up_block:


<code><%= raw(mark_up_block(@block)) %></code>

Notice that we’re wrapping mark_up_block with raw. We want to HTML being injected into our JSON to be interpreted as raw HTML, and not encoded as special characters.

After carrying out the same changes in our BlockView and our block HTMl template, cleaning up our layout template, and adding a few final styling touches, we have our result.

A decorated full block.

Behold, the most basic of blockchain explorers!

Final Thoughts

Obviously, this is just the tip of the iceberg when it comes to building a Bitcoin blockchain explorer.

I’m super excited about Bitcoin and development projects related to Bitcoin. If you’ve made it this far, I assume you are as well! If you’re looking for a deep dive into Bitcoin development, I recommend you check out the fantastic Mastering Bitcoin (affiliate link) book by Andreas Antonopoulos.

Expect more updates to this blockchain explorer in the future, and more Bitcoin focused projects. In the meantime, check out this project on Github.

Inject Detect is Live!

Written by Pete Corey on Sep 11, 2017.

My latest project, Inject Detect is officially live and open to the public! If you have a Meteor application and you’re concerned about the threat of NoSQL Injection, I made this for you!

As I mentioned last week, it’s been a long and winding road to the launch of Inject Detect, but I couldn’t be happier with the results.

Your Inject Detect dashboard.

My original goals in building Inject Detect were to help safeguard Meteor application owners and developers from the most common vulnerability I’ve seen in Meteor applications: NoSQL Injection. I wanted to offer a more affordable and effective alternative to my hands-on security assessment process. Inject Detect is the result.

Inject Detect is designed to detect and notify you of potential NoSQL Injection attacks as they happen in your application. It does this by building up a set of “expected queries” and monitoring for any queries that are structurally dissimilar from any of the queries in that set.

Unexpected queries in an application.

In an “always on” utility service, affordability is key. Inject Detect uses a “pay-as-you-go” pricing structure where every processed query costs a mere one one hundredth of a cent ($0.0001). This means that the peace of mind afforded by Inject Detect’s watchful eye will never break your project’s bank.

In addition to using a “pay-as-you-go” pricing structure, I’m also giving new users an initial ten dollar ($10.00) account credit so you can try out Inject Detect on the house for as long as that balance lasts you.

An application’s set of expected queries.

If you’re interested in learning more about Inject Detect and NoSQL Injection, or are just curious about what I’ve been working on for the past few months, check out these related articles:

If you’re eager to cut to the chase, you can sign up for Inject Detect here!

An unexpected query.

I’m incredibly excited to see how Inject Detect can safeguard your application and your data. Don’t hesitate to reach out and let me know if you have any comments, feedback, or questions about the project.

Controlling a Bitcoin Node with Elixir

Written by Pete Corey on Sep 4, 2017.

I’ve been bit by the Bitcoin bug, and I’ve been bit hard. To satiate my thirst for knowledge, I’ve been reading the fantastic Mastering Bitcoin (affiliate link) book by Andreas Antonopoulos, and diving into the brave new world of Bitcoin development.

Mastering Bitcoin does a fantastic job of outlining the technical underpinnings of Bitcoin, but I wanted to solidify my understanding with some hands-on experience.

Writing a simple Elixir application to communicate and control a Bitcoin Core full node through its JSON-RPC interface seems like a fantastic “hello world” exercise. Let’s get to it!

You’ll Need a Full Node

The first step in communicating with a Bitcoin Core full node is getting our hands on one. While publicly available nodes with wide open JSON-RPC interfaces are few and far between, it’s fairly simple to run our own Bitcoin Core node locally.

Assuming we’ve installed the bitcoind daemon on our system, we’ll need to configure it with a bitcoin.config file:


rpcuser=<username>
rpcpassword=<password>

The <username> and <password> values we define in our configuration will be used to authenticate ourselves when making requests to the Bitcoin node.

Once we’ve created our configuration file, we can spin up our full node:


bitcoind -conf=<path to bitcoin.config> -daemon

Once started, our full node daemon will begin connecting to peer nodes, downloading, and verifying blocks from the blockchain.

We can verify that everything is working as expected:


bitcoin-cli getinfo

This command should return some basic information about the node, including the node’s "version", and the number of "blocks" it’s received and verified. It may take several days to download and verify the entire blockchain, but we can keep continue on with our project in the meantime.

The Bitcoin Node’s JSON-RPC

Our Bitcoin full node implements a JSON-based RPC API which can be used to retrieve information about the Bitcoin blockchain, and to interact with the node itself.

Interestingly, the bitcoin-cli tool that we used to get information about the node leverages this JSON-RPC API. You can fetch a list of all of the available RPC commands on the node by calling bitcoin-cli help, or by browsing through the Bitcoin Wiki.

The node’s JSON-RPC accepts incoming commands through an HTTP server, which means that we can manually craft these RPC commands and bypass the bitcoin-cli tool entirely.

For example, we can run getinfo manually with curl:


curl --data-binary '{"jsonrpc":"1.0","method":"getinfo","params":[]}' \
     http://<user>:<pass>@localhost:8332/

Similarly, we can execute these commands from any programming environment with an HTTP client, like Elixir!

Setting Up Our Elixir Application

Now that we have a strategy for communicating with our Bitcoin full node, let’s start building out our Elixir application.

First, we’ll create a new Elixir project and update our mix.exs to add dependencies on poison, which we’ll need to encode and decode JSON objects, and httpoison, our go-to Elixir HTTP client.


defp deps do
  [
    {:httpoison, "~> 0.13"},
    {:poison, "~> 3.1"}
  ]
end

Now that we’ve laid out the scaffolding for our application, let’s turn our attention towards talking with our Bitcoin node.

We’ll start by gutting our HelloBitcoin module, and stubbing out a new getinfo function:


defmodule HelloBitcoin do

  def getinfo do
    raise "TODO: Implement getinfo"
  end

end

To keep things simple, we’ll interact with this module through iex -S mix. As a sanity check, let’s verify that everything is working correctly before moving on to the next section.

Calling our HelloBitcoin.getinfo stub should raise a runtime exception:


iex(1)> HelloBitcoin.getinfo
HelloBitcoin.getinfo
** (RuntimeError) TODO: Implement getinfo
    (hello_bitcoin) lib/hello_bitcoin.ex:4: HelloBitcoin.getinfo/0

Perfect. Progress through failure.

Constructing the GetInfo Command

Let’s start to flesh out our getinfo function.

To recap, our goal is to send a POST HTTP request to our Bitcoin node’s HTTP server (usually listening on http://localhost:8332), passing in a JSON object that holds the command we’re trying to execute and any required parameters.

It turns out this is incredibly easy with httpoison:


def getinfo do
  with url     <- Application.get_env(:hello_bitcoin, :bitcoin_url),
       command <- %{jsonrpc: "1.0", method: "getinfo", params: []},
       body    <- Poison.encode!(command),
       headers <- [{"Content-Type", "application/json"}] do
    HTTPoison.post!(url, body, headers)
  end
end

We start by pulling our url from the bitcoin_url key in our application’s configuration. This needs to be set in config/config.exs and should point to your local node:


config :hello_bitcoin, bitcoin_url: "http://<user>:<password>@localhost:8332"

Next, we build a map that represents our JSON-RPC command. In this case, our method is "getinfo", which requires no params. Finally, we construct the body of our request by JSON encoding our command with Poison.encode!.

Calling HelloBitcoin.getinfo should give us a successful 200 response from the Bitcoin node, along with the JSON encoded response to our getinfo command:


%HTTPoison.Response{
  body: "{\"result\":{\"version\":140200,\"protocolversion\":70015,\"walletversion\":130000,\"balance\":0.00000000,\"blocks\":482864,\"timeoffset\":-1,\"connections\":8,\"proxy\":\"\",\"difficulty\":888171856257.3206,\"testnet\":false,\"keypoololdest\":1503512537,\"keypoolsize\":100,\"paytxfee\":0.00000000,\"relayfee\":0.00001000,\"errors\":\"\"},\"error\":null,\"id\":null}\n",
  headers: [{"Content-Type", "application/json"}, {"Date", "Thu, 31 Aug 2017 21:27:02 GMT"}, {"Content-Length", "328"}],
  request_url: "http://localhost:8332",
  status_code: 200
}

Beautiful.

Let’s decode the resulting JSON in body and return the result:


HTTPoison.post!(url, body)
|> Map.get(:body)
|> Poison.decode!

Now our call to HelloBitcoin.getinfo returns the result returned by bitcoind in a more usable format:


%{"error" => nil, "id" => nil,
  "result" => %{"balance" => 0.0, "blocks" => 483001, "connections" => 8,
    "difficulty" => 888171856257.3206, "errors" => "",
    "keypoololdest" => 1503512537, "keypoolsize" => 100, "paytxfee" => 0.0,
    "protocolversion" => 70015, "proxy" => "", "relayfee" => 1.0e-5,
    "testnet" => false, "timeoffset" => -1, "version" => 140200,
    "walletversion" => 130000}}

You’ll notice that the "result", the data we actually want, is wrapped in a map containing metadata about the request itself. This metadata includes a potential error string, and the id of the request.

Let’s refactor our getinfo function to include some error handling, and to return the actual data we care about in the case of an error-free response:


with url <- Application.get_env(:hello_bitcoin, :bitcoin_url),
     command <- %{jsonrpc: "1.0", method: "getinfo", params: []},
     {:ok, body} <- Poison.encode(command),
     {:ok, response} <- HTTPoison.post(url, body),
     {:ok, metadata} <- Poison.decode(response.body),
     %{"error" => nil, "result" => result} <- metadata do
  result
else
  %{"error" => reason} -> {:error, reason}
  error -> error
end

Now our getinfo function will return an {:ok, result} tuple containing the result of our getinfo RPC call if everything goes well. In the case of an error we’ll receive an {:error, reason} tuple, explaining the failure.

Generalizing Commands

We could implement another Bitcoin RPC command, like getblockhash, in a nearly identical fashion:


def getblockhash(index) do
  with url <- Application.get_env(:hello_bitcoin, :bitcoin_url),
       command <- %{jsonrpc: "1.0", method: "getblockhash", params: [index]},
       {:ok, body} <- Poison.encode(command),
       {:ok, response} <- HTTPoison.post(url, body),
       {:ok, metadata} <- Poison.decode(response.body),
       %{"error" => nil, "result" => result} <- metadata do
    {:ok, result}
  else
    %{"error" => reason} -> {:error, reason}
    error -> error
  end
end

Calling our new getblockhash with an index of 0 gives us the hash of the Bitcoin genesis block, as we would expect.


HelloBitcoin.getblockhash(0)

{:ok, "000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f"}

While it’s great that this works, you’ll notice that there’s a huge amount of code duplication going on here. Our getblockhash function is nearly identical to our getinfo function.

Let’s abstract out the common functionality into a new bitcoin_rpc helper function:


defp bitcoin_rpc(method, params \\ []) do
  with url <- Application.get_env(:hello_bitcoin, :bitcoin_url),
       command <- %{jsonrpc: "1.0", method: method, params: params},
       {:ok, body} <- Poison.encode(command),
       {:ok, response} <- HTTPoison.post(url, body),
       {:ok, metadata} <- Poison.decode(response.body),
       %{"error" => nil, "result" => result} <- metadata do
    {:ok, result}
  else
    %{"error" => reason} -> {:error, reason}
    error -> error
  end
end

Now we can redefine our getinfo and getblockhash functions in terms of this new bitcoin_rpc helper function:


def getinfo, do: bitcoin_rpc("getinfo")

def getblockhash(index), do: bitcoin_rpc("getblockhash", [index])

Our bitcoin_rpc now acts as a fully functional and complete Bitcoin RPC interface. We can easily implement any of the Bitcoin RPC commands using this helper function.

If you’re curious and want to interact with a Bitcoin node yourself, the full source for this HelloBitcoin project is available on GitHub.

Wrap Up

In hindsight, this was a long article explaining a relatively simple idea. The Bitcoin full node software exposes a JSON-RPC interface that can easily be accessed by your favorite language or stack, such as Elixir.

I’m incredibly excited about Bitcoin development, and I’m planning on spending more time diving deeper into this world in the future.

If you’re interested in the technical ideas behind Bitcoin, or are interested in Bitcoin development, I highly recommend you read Mastering Bitcoin (affiliate link).