Offline GraphQL Mutations with Redux Offline and Apollo

Written by Pete Corey on Jul 31, 2017.

Last week we started a deep dive into adding offline support to a React application using a GraphQL data layer powered by Apollo.

Thanks to out of the box features provided by Apollo Client, and a little extra help provided by Redux Offline and Redux Persist, we’ve managed to get our Apollo queries persisting through page loads and network disruptions.

Now let’s turn our attention to mutations.

How do we handle Apollo mutations made while our client is disconnected from the server? How do we store data and mutations locally and later sync those changes to the server once we regain connectivity?

Defining the Problem

Last week, we dealt with mostly infrastructure-level changes to add offline support for our Apollo queries. Adding support for offline mutations requires a more hands-on approach.

To help explain, let’s pretend that we’re building a survey application. After a user has filled out the questions in a survey, they can submit the survey to the server through a completeSurvey mutation:


mutation completeSurvey($surveyId: ID!, $answers: [String]) {
    completeSurvey(surveyId: $surveyId, answers: $answers) {
        _id
        answers
        completedAt
    }
}

We’re passing this mutation into a component and calling it as you would any other mutation in an Apollo-based application:


onCompleteSurvey = () => {
    let surveyId = this.props.data.survey._id;
    let answers = this.state.answers;
    this.props.completeSurvey(surveyId, answers);
};

export default graphql(gql`
    ...
`, {
    props: ({ mutate }) => ({
        completeSurvey: (surveyId, answers) => mutate({
            variables: { surveyId, answers } 
        })
    })
})(Survey);

Unfortunately, this mutation will fail if the client attempts to submit their survey while disconnected from the server.

To make matters worth, we can’t even capture these failures by listening for a APOLLO_MUTATION_ERROR action in a custom reducer. Network-level errors are swallowed before an APOLLO_MUTATION_INIT is fired and results in an exception thrown by your mutation’s promise.

This is a problem.

Defining Success

Now that we’ve defined our problem, let’s try to define what a solution to this problem would look like.

“Offline support” is an amorphous blob of features, held together by a fuzzy notion of what the system “should do” while offline, and torn apart by what’s “actually possible”. What does it mean for our mutations to support network disruptions? Getting down to the details, what exactly should happen in our application when a user attempts to submit a survey while offline?

In our situation, it would be nice to mark these surveys as “pending” on the client. Once the user reconnects to the server, any pending surveys should automatically be completed in order via completeSurvey mutations. In the meantime, we could use this “pending” status to indicate the situation to the user in a friendly and meaningful way.

Now that we know what a successful offline solution looks like, let’s build it!

Enter Redux Offline

When it came to supporting offline queries, Redux Offline largely worked under the hood. None of the components within our application needed any modifications to support offline querying.

Unfortunately, that’s not the case with offline mutations.

To support offline mutations through Redux Offline, we’ll need to wrap all of our mutations in plain old Redux actions. These actions should define a meta field that Redux Offline uses to reconcile the mutations with the server, once reconnected.

Let’s add offline support to our completeSurvey mutation.


First, we’ll set up the Redux action and an action creator that we’ll use to complete our survey:


export const COMPLETE_SURVEY = 'COMPLETE_SURVEY';

export const completeSurvey = (survey, answers) => {
    const mutation = gql`
        mutation completeSurvey($surveyId: ID!, $answers: [String]) {
            completeSurvey(surveyId: $surveyId, answers: $answers) {
                _id
                answers
                completedAt
            }
        }
    `;
    return {
        type: COMPLETE_SURVEY,
        payload: { ...survey },
        meta: {
            offline: {
                effect: { mutation, variables: { surveyId: survey._id, answers } }
            }
        }
    };
};

The offline effect of this action contains our completeSurvey Apollo mutation, along with the surveyId and answers variables needed to populate the mutation.


To tell Redux Offline how to handle this offline effect object, we’ll need to add an effect callback to the Redux Offline configuration we previously defined in our store:


offline({
    ...config,
    ...,
    effect: (effect, action) => {
        return client.mutate({ ...effect }).then(({ data }) => data);
    }
})

At this point, we’ve instructed Redux Offline to manually trigger an Apollo mutation whenever we dispatch an action with an offline effect.

If Redux Offline detects that the client is disconnected from the server, it will throw this effect into a queue to be retried later.

Let’s refactor our Survey component to use our new action!


Now that our COMPLETE_SURVEY action is completed, we’ll inject a dispatcher for our new action into our React component and use it to replace our direct call to the completeSurvey mutation:


onCompleteSurvey = () => {
    let surveyId = this.props.data.survey._id;
    let answers = this.state.answers;
    this.props.completeSurvey(surveyId, answers);
};

export default connect(null, { completeSurvey })(Survey);

Now, instead of manually triggering the completeSurvey mutation through Apollo, we’re dispatching our COMPLETE_SURVEY action, which contains all of the information needed for Redux Offline to trigger our mutation either now or at some point in the future.


Once dispatched, if Redux Offline detects an active connection to the server, it will immediately carry out the effect associated with our COMPLETE_SURVEY action. This would kick off our completeSurvey mutation, and all would be right in the world.

However, if Redux Offline detects that it’s disconnected from the server, it stores the action in a queue it refers to as its “outbox” (offline.outbox in the Redux store). Once we reconnect to the server, Redux Offline works through its outbox in order, carrying out each queued mutation.

Now that we’ve refactored our survey application to manage submissions through an action managed by Redux Offline, we’ve successfully added partial offline support to our application.

Now we need to indicate what’s happening to the user.

Managing Offline Data Locally

In order to inform the user that their survey is in a “pending” state, we’ll need to store these updated surveys somewhere on the client until their status is reconciled with the server.

Our first instinct might be to keep our updated surveys where we’re keeping the rest of our application’s data: in Apollo’s store. This would let us neatly retrieve our data through Apollo queries! Unfortunately, it’s very difficult to directly and arbitrarily update the contents of our Apollo store.

Instead, let’s store the surveys in a new section of our Redux store (under the surveys key) that lives along side our Apollo store.

Creating and populating our new store with pending surveys is actually incredibly easy. First things first, let’s create a new reducer that listens for our COMPLETE_SURVEY action and stores the corresponding survey in our new store:


export default (state = [], action) => {
    switch (action.type) {
        case COMPLETE_SURVEY:
            return [...state, action.payload];
        default:
            return state;
    }
};

If you remember our action creator function, you’ll remember that the payload field of our COMPLETE_SURVEY action contains the entire survey object. When we handle the COMPLETE_SURVEY action, we simple concatenate that survey into our list of already completed surveys.

Next, we’ll need to wire this new reducer into our Redux store:


import SurveyReducer from "...";

export const store = createStore(
    combineReducers({
        surveys: SurveyReducer,
        ...
    }),
    ...

Perfect. Now every survey submitted while offline will be added to the surveys array living in our Redux store.

Displaying Pending Surveys

We can display these pending surveys to our users by subscribing to the surveys field of our Redux store in our React components:


export default connect(state => {
    return {
        pendingSurveys: state.survey
    };
})(PendingSurveyList);

Now we can access this tangential list of pendingSurveys from our components props and render them just as we would any other survey in our application:


render() {
    let pending = this.props.pendingSurveys;
    return (
        

Pending Surveys:

{pending.map(survey => <Survey survey={survey}/>)}
); }

When we render this component, our users will see their list of pending surveys that have yet to be submitted to the server.

Great! 🎉

Manually Managing Survey State

Unfortunately, there’s a problem with this solution. Even if we’re online when we submit our survey, it will be added to our surveys store and shown as a pending survey in our UI.

This makes sense.

Because we’re using the COMPLETE_SURVEY action to handle all survey completions, our action will fire on every survey we submit. These surveys pile up in our surveys list and never get removed. Because we’re persisting and rehydrating our store to localStorage, these surveys will persist even through page reloads!

We need a way to remove surveys from our surveys store once they’ve been submitted to the server.

Thankfully, Redux Offline has a mechanism for handling this.

Let’s make a new action called COMPLETE_SURVEY_COMMIT. We can instruct Redux Offline to dispatch this action once our mutation has been executed by specifying it in the commit field of the meta.offline portion of our action creator function:


meta: {
    offline: {
        effect: { mutation, variables: { surveyId: survey._id, answers } },
        commit: { type: COMPLETE_SURVEY_COMMIT, meta: { surveyId: survey._id } }
    }
}

Now we need to update our surveys reducer to remove a survey from our surveys store whenever a COMPLETE_SURVEY_COMMIT action is handled:


switch (action.type) {
    ...
    case COMPLETE_SURVEY_COMMIT:
        return _.chain(state).clone().filter(survey => survey._id !== action.meta.surveyId).value();
}

That’s it!

Now our application is adding surveys to the surveys store when they’re submitted (or marked as submitted while offline), and removing them once our completeSurvey mutation is successfully executed.

With that, we’ve achieved our definition of success.

If submitted offline, surveys will go into a “pending” state, which is visible to the user, and will eventually be synced with the server, in order, once a connection is re-established.

Credit Where Credit is Due

With a little bit of up-front planning and elbow grease, we’ve managed to add support for offline mutations to our React and Apollo-powered application. Combined with support for offline querying, we’ve managed to build out a reasonably powerful set of offline functionality!

To get a more wholistic understanding of the overall solution described here, be sure to check out Manur’s fantastic “Redux Offline Examples” project on Github. The apollo-web project, in particular, was a major inspiration for this post and an invaluable resource for adding feature rich offline support to my Apollo application.

He even includes more advanced features in his apollo-web project, such as reconciling locally generated IDs with server-generated IDs after a sync. Be sure to give the project a read through if you’re hungry for more details.

Thanks Manur, Apollo, Redux Offline, and Redux Persist!

Offline GraphQL Queries with Redux Offline and Apollo

Written by Pete Corey on Jul 24, 2017.

Ironically, in our ever more connected world, demands for offline capabilities of web applications are growing. Our users (and clients) expect to use rich internet applications while online, offline, and in areas of questionable connectivity.

This can be… difficult.

Let’s dive into how we can build a reasonably powerful offline solution using React and a GraphQL data layer powered by Apollo Client. We’ll split this article into two parts. This week, we’ll discuss offline querying. Next week, we’ll tackle mutations.

Redux Persist and Redux Offline

Under the hood, Apollo Client is powered by Redux. This means that the entire Redux ecosystem of tools and libraries are available for us to use in our Apollo application.

In the world of Redux offline support, there are two major players: Redux Persist, and Redux Offline.

Redux Persist is a fantastic, but bare bones, tool designed to store and retrieve (or “rehydrate”) a redux store to and from localStorage (or any other supported storage engine).

Redux Offline expands on Redux Persist and adds additional layers of functionality and utility. Redux Offline automatically detects network disconnections and reconnections, lets you queue up actions and operations while offline, and automatically retries those actions once reconnected.

Redux Offline is the batteries-included option for offline support. 🔋

Offline Queries

Out of the box, Apollo Client works fairly well in partially connected network situations. Once a query is made by the client, the results of that query are saved to the Apollo store.

If that same query is made again with any fetchPolicy other than network-only, the results of that query will be immediately pulled from the client’s store and returned to the querying component. This means that even if our client is disconnected from the server, repeated queries will still be resolved with the most recent results available.

Unfortunately, as soon as a user closes our application, their store is lost. How can we persist the client’s Apollo store through application restarts?

Redux Offline to the rescue!

The Apollo store actually exists within our application’s Redux store (under the apollo key). By persisting the entire Redux store to localStorage, and rehydrating it every time the application is loaded, we can carry over the results of past queries through application restarts, even while disconnected from the internet!

Using Redux Offline with an Apollo Client application doesn’t come without its kinks. Let’s explore how to get these two libraries to work together.

Manually Building a Store

Normally, setting up an Apollo client is fairly simple:


export const client = new ApolloClient({
    networkInterface
});

The ApolloClient constructor would create our Apollo store (and indirectly, our Redux store) automatically for us. We’d simply drop this new client into our ApolloProvider component:


ReactDOM.render(
    <ApolloProvider client={client}>
        <App />
    </ApolloProvider>,
    document.getElementById('root')
);

When using Redux Offline, we’ll need to manually construct our Redux store to pass in our Redux Offline middleware. To start, let’s just recreate what Apollo does for us:


export const store = createStore(
    combineReducers({ apollo: client.reducer() }),
    undefined,
    applyMiddleware(client.middleware())
);

Our new store uses the reducer and middleware provided by our Apollo client, and initializes with an initial store value of undefined.

We can now pass this store into our ApolloProvider:


<ApolloProvider client={client} store={store}>
    <App />
</ApolloProvider>

Perfect. Now that we have control over the creation of our Redux store, we can wire in offline support with Redux Offline.

Basic Query Persistence

Adding Redux Offline into the mix, in its simplest form, consists of adding a new piece of middleware to our store:


import { offline } from 'redux-offline';
import config from 'redux-offline/lib/defaults';

export const store = createStore(
    ...
    compose(
        applyMiddleware(client.middleware()),
        offline(config)
    )
);

Out of the box, this offline middleware will automatically start persisting our Redux store into localStorage.

Don’t believe me?

Fire up your console and pull up this localStorage entry:


localStorage.getItem("reduxPersist:apollo");

You should be given a massive JSON blob that represents the entire current state of your Apollo application.

Awesome!

Redux Offline is now automatically saving snapshots of our Redux store to localStorage. Any time you reload your application, this state will be automatically pulled out of localStorage and rehydrated into your Redux store.

Any queries that have resolutions living in this store will return that data, even if the application is currently disconnected from the server.

Rehydration Race Conditions

Unfortunately, store rehydration isn’t instant. If our application tries to make queries while Redux Offline is rehydrating our store, Strange Things™ can happen.

If we turn on autoRehydrate logging within Redux Offline (which is an ordeal in and of itself), we’d see similar errors when we first load our application:

21 actions were fired before rehydration completed. This can be a symptom of a race condition where the rehydrate action may overwrite the previously affected state. Consider running these actions after rehydration: …

The creator of Redux Persist acknowledges this and has written a recipe for delaying the rendering of your application until rehydration has taken place. Unfortunately, his solution relies on manually calling persistStore, which Redux Offline does for us behind the scenes.

Let’s come up with another solution.

We’ll start by creating a new Redux action called REHYDRATE_STORE, and a corresponding reducer that sets a rehydrated flag in our Redux store to true:


export const REHYDRATE_STORE = 'REHYDRATE_STORE';

export default (state = false, action) => {
    switch (action.type) {
        case REHYDRATE_STORE:
            return true;
        default:
            return state;
    }
};

Now let’s add our new reducer to our store and tell Redux Offline to trigger our action when it finishes rehydrating the store:


export const store = createStore(
    combineReducers({
        rehydrate: RehydrateReducer,
        apollo: client.reducer()
    }),
    ...,
    compose(
        ...
        offline({
            ...config,
            persistCallback: () => {
                store.dispatch({ type: REHYDRATE_STORE });
            },
            persistOptions: {
                blacklist: ['rehydrate']
            }
        })
    )
);

Perfect. When Redux Offline finishes hydrating our store, it’ll trigger the persistCallback function, which dispatches our REHYDRATE_STORE action, and eventually updates the rehydrate field in our store.

Adding rehydrate to our Redux Offline blacklist ensures that that piece of our store will never be stored to, or rehydrated from localStorage.

Now that our store is accurately reflecting whether or not rehydration has happened, let’s write a component that listens for changes to our new rehydrate field and only renders its children if rehydrate is true:


class Rehydrated extends Component {
    render() {
        return (
            <div className="rehydrated">
                {this.props.rehydrated ? this.props.children : <Loader />}
            </div>
        );
    }
}

export default connect(state => {
    return {
        rehydrate: state.rehydrate
    };
})(Rehydrate);

Finally, we can wrap our <App /> component in our new <Rehydrate> component to prevent our application from rendering until rehydration has taken place:


<ApolloProvider client={client} store={store}>
    <Rehydrated>
        <App />
    </Rehydrated>
</ApolloProvider>

Whew.

Now our application will happily wait until Redux Offline has completely rehydrated our store from localStorage before continuing on to render and make any subsequent GraphQL queries or mutations.

Quirks and Notes

There are a few quirks and things to take note of when using Redux Offline with Apollo client.

First, it’s important to note that the examples in this article are using version 1.9.0-0 of the apollo-client package. Fixes were introduced to Apollo Client in version 1.9 that resolved some strange behaviors when combined with Redux Offline.

Another oddity related to this setup is that Redux Offline doesn’t seem to play nicely with the Apollo Client Devtools. Trying to use Redux Offline with the Devtools installed can sometimes lead to unexpected, and seemingly unrelated errors.

These errors can be easily avoided by not connecting to the Devtools when creating your Apollo client instance:


export const client = new ApolloClient({
    networkInterface,
    connectToDevTools: false
});

Stay Tuned

Redux Offline should give you basic support for query resolution for your Apollo-powered React application, even if your application is re-loaded while disconnected from your server.

Next week we’ll dive into handling offline mutations with Redux Offline.

Stay tuned!

Recurring Tasks in Elixir

Written by Pete Corey on Jul 17, 2017.

Running periodic, or recurring tasks is a common undertaking for any web application. The stacks I’ve used in the past have all relied heavily on external databases and job queues to accomplish this task.

Elixir is a little different.

Thanks to Elixir’s Erlang heritage and the power of OTP, we’re given the option to opt out of relying on an external database and manage our recurring tasks from entirely within our application.

The Fruit Printer

Before we implement our recurring task runner, we should have a task that we want to repeat. Let’s pretend that we want to print out a random item from a list of @fruits:


@fruits ["🍉", "🍊", "🌽", "🍒", "🍇", "🌶"] # TODO: Is corn a fruit?

def print_fruit, do: IO.puts("fruit: #{Enum.fetch!(@fruits, :random.uniform(6))}")

To keep up with our voracious appetite for fruit, we want to print a fruit emoji to the console every two seconds. This is our recurring task.

How do we accomplish this?

The first thing we need to do is build our fruit printer into a GenServer process:


defmodule HelloRecurring.FruitPrinter do
  use GenServer

  @fruits ["🍉", "🍊", "🌽", "🍒", "🍇", "🌶"] # TODO: Is corn a fruit?

  def start_link, do: GenServer.start_link(__MODULE__, [])

  def print_fruit, do: IO.puts("fruit: #{Enum.fetch!(@fruits, :rand.uniform(6))}")

end

We’ll also want to supervise our new fruit printing operation:


defmodule HelloRecurring do
  use Application

  import Supervisor.Spec

  def start(_type, _args) do
    Supervisor.start_link([worker(HelloRecurring.FruitPrinter, [])],
                          [strategy: :one_for_one, name: HelloRecurring.Supervisor])
  end

end

Here we’re simply adding a FruitPrinter as a child of our supervision tree and telling the supervisor to restart the FruitPrinter child process if it ever dies for any reason.

At this point, our FruitPrinter GenServer isn’t doing us much good. It’s running, but it’s not printing fruit. We can still manually print fruit by calling FruitPrinter.print_fruit, but this would run the print_fruit function within the current process, not the GenServer’s process.

Not good enough!

We want our FruitPrinter to automatically print its own fruit every two seconds!

Our solution comes in the form of standard process messages. Let’s wire our FruitPrinter up to print_fruit whenever it receives a :print_fruit message:


def handle_info(:print_fruit, state) do
  print_fruit()
  {:noreply, state}
end

Now we can send a :print_fruit message to our FruitPrinter process with either Process.send/3, or Process.send_after/4.

Printing Fruit Forever and Ever

Sending delayed messages with Process.send_after/4 will be the key component to implementing our recurring task.

The general idea behind building out a recurring task runner in Elixir is that the task itself should be a GenServer process that schedules sending messages to itself signaling it to carry out its task.

Putting that plan into action, once our FruitPrinter is started, we can schedule a :print_fruit message to be sent to itself in two seconds:


def init(state) do
  schedule()
  {:ok, state}
end

def schedule, do: Process.send_after(self(), :print_fruit, 2000)

It’s important to note that we need to schedule our :print_fruit message in the GenServer’s init callback, rather than the start_link callback, because start_link is called under the context of the supervising process. The init callback is called once the process is created, and self() will point to our FruitPrinter, not the supervisor.

Next, we’ll add another call to schedule() in our handle_info callback. This will ensure that every handled :print_fruit message will schedule another :print_fruit message to be sent two seconds in the future:


def handle_info(:print_fruit, state) do
  schedule()
  print_fruit()
  {:noreply, state}
end

Spinning up the application, you’ll notice a constant stream of fruit being printed at a steady rate of once every two seconds.

Delicious victory.

Even When Things Go Wrong

Astute readers may have noticed a bug in our initial print_fruit function.

We’re using :rand.uniform(6) to pick a random index out of our list of @fruits. Unfortunately, :rand.uniform/1 produces a random number between 1 and n, not 0 and n - 1, as we assumed. This means that any given call to print_fruit has a one in six chance of crashing with an out of bounds error.

Whoops!

Interestingly, this bug hasn’t affected our recurring task. If we run our application for long enough to see this error raise its head, we’ll notice that two seconds after our FruitPrinter process crashes, it’s up and running again trying to print another random fruit.

Because our FruitPrinter is being supervised, any failures that result in a crash of the process will cause the supervising process to create a new FruitPrinter in its place. This new FruitPrinter will schedule a :print_fruit message in its init callback, and will continue working as expected.

Back to the problem at hand, the proper way to implement our print_fruit function would be with Elixir’s Enum.random/1 function:


def print_fruit, do: IO.puts("fruit: #{Enum.random(@fruits)}")

That’s better. We certainly don’t want bugs in our fruit.

Final Thoughts

While this type of entirely in-application recurring process may not be a solution for every problem out there, it’s a powerful option in the Elixir environment.

The robustness given to us by the concept of supervisors and the “let it crash” mentality gives us a clear advantage over similar patterns in other languages (i.e. setTimeout in Node.js).

Before you go reaching for an external tool, I’ve found that it’s often beneficial to ask yourself, “can I do this with just Elixir?”