Querying Non-Existent MongoDB Fields

Written by Pete Corey on Sep 5, 2016.

We were recently contacted by one of our readers asking about a security vulnerability in one of their Meteor applications.

They noticed that when they weren’t authenticated, they were able to pull down a large number of documents from a collection through a publication they thought was protected.

The Vulnerability

The publication in question looked something like this:


Meteor.publish("documents", function() {
  return Documents.find({ 
    $or: [
      { userId: this.userId },
      { sharedWith: this.userId }
    ]
  });
});

When an unauthenticated user subscribes to this publication, their userId would be undefined, or null when it’s translated into a MongoDB query.

This means that the query passed into Documents.find would look something like this:


{
  $or: [
    { userId: null },
    { sharedWith: null }
  ]
}

The $or query operator means that if either of these sub-queries match a document, that document will be returned to the client.

The first sub-query, { userId: null }, mostly likely won’t match any documents. It’s unlikely that a Document will be created without a userId, so there will be no Documents with a null or nonexistent userId field.

The second sub-query is more interesting. Due to a quirk with how MongoDB handles null equality queries, the { sharedWith: null } sub-query will return all documents who’s sharedWith field is either null, or unset. This means the query will return all unshared documents.

An un-authenticated user subscribing to the "documents" publication would be able to view huge amounts of private data.

This is a bad thing.

Fixing the Query

There are several ways we can fix this publication. The most straight-forward is to simply deny unauthenticated users access:


Meteor.publish("documents", function() {
  if (!this.userId) {
    throw new Meteor.Error("permission-denied");
  }
  ...
});

Another fix would be to conditionally append the sharedWith sub-query if the user is authenticated:


Meteor.publish("documents", function() {
  let query = {
    $or: [{ userId: this.userId }]
  };
  if (this.userId) {
    query.$or.push({ sharedWith: this.userId });
  }
  return Documents.find(query);
});

This will only add the { sharedWith: this.userId } sub-query if this.userId resolves to a truthy value.

Final Thoughts

This vulnerability represents a larger misunderstanding about MongoDB queries in general. Queries searching for a field equal to null will match on all documents who’s field in question equals null, or who don’t have a value for this field.

For example, this query: { imaginaryField: null } will match on all documents in a collection, unless they have a value in the imaginaryField field that is not equal to null.

This is a very subtle, but very dangerous edge case when it comes to writing MongoDB queries. Be sure to keep it in mind when designing the MongoDB queries used in your Meteor publications and methods.

Phoenix Todos - Static Assets

This post is written as a set of Literate Commits. The goal of this style is to show you how this program came together from beginning to end.

Each commit in the project is represented by a section of the article. Click each section's header to see the commit on Github, or check out the repository and follow along.

Written by Pete Corey on Aug 31, 2016.

Mix Phoenix.New

For this project we’ll be recreating Meteor’s Todos application using a React front-end and an Elixir/Phoenix powered backend. With limited experience with both React and Elixir/Phoenix, this should definitely be an interesting leaning experience.

This will be the largest literate commits project we’ve done to date, and will span several articles published over the upcoming weeks. Be sure to stay tuned for future posts!

This first commit creates a new Phoenix project called phoenix_todos.

Adding mix_test_watch

Moving forward, we’ll be writing tests for the Elixir code we create. Being the good developers that we are, we should test this code.

To make testing a more integrated process, we’ll add the mix_text_watch dependency, which lets us run the mix test.watch command to continuously watch our project for changes and re-run our test suite.

mix.exs

... {:gettext, "~> 0.9"}, - {:cowboy, "~> 1.0"}] + {:cowboy, "~> 1.0"}, + {:mix_test_watch, "~> 0.2", only: :dev}] end

mix.lock

"mime": {:hex, :mime, "1.0.1"}, + "mix_test_watch": {:hex, :mix_test_watch, "0.2.6"}, "phoenix": {:hex, :phoenix, "1.1.6"},

Hello React

Laying the groundwork for using React in a Phoenix project is fairly straight-forward.

We kick things off by installing our necessary NPM dependencies (react, react-dom, and babel-preset-react), and then updating our brunch-config.js to use the required Babel preset, and whitelisting our React NPM modules.

Once that’s finished, we can test the waters by replacing our app.html.eex layout template with a simple React attachment point:


<div id="hello-world"></div>

Finally, we can update our app.js to create and render a HelloWorld component within this new element:


class HelloWorld extends React.Component {
  render() {
    return (<h1>Hello World!</h1>)
  }
}
 
ReactDOM.render(
  <HelloWorld/>,
  document.getElementById("hello-world")
)

For a more detailed rundown of this setup process, be sure to read this fantastic article by Brandon Richey that walks you throw the process step by step.

brunch-config.js

... babel: { + presets: ["es2015", "react"], // Do not use ES6 compiler in vendor code ... npm: { - enabled: true + enabled: true, + whitelist: ["phoenix", "phoenix_html", "react", "react-dom"] }

package.json

{ - "repository": { - }, + "repository": {}, "dependencies": { "babel-brunch": "^6.0.0", + "babel-preset-react": "^6.11.1", "brunch": "^2.0.0", "javascript-brunch": ">= 1.0 < 1.8", + "react": "^15.3.1", + "react-dom": "^15.3.1", "uglify-js-brunch": ">= 1.0 < 1.8"

web/static/js/app.js

-// Brunch automatically concatenates all files in your -// watched paths. Those paths can be configured at -// config.paths.watched in "brunch-config.js". -// -// However, those files will only be executed if -// explicitly imported. The only exception are files -// in vendor, which are never wrapped in imports and -// therefore are always executed. +import React from "react" +import ReactDOM from "react-dom" -// Import dependencies -// -// If you no longer want to use a dependency, remember -// to also remove its path from "config.paths.watched". -import "deps/phoenix_html/web/static/js/phoenix_html" +class HelloWorld extends React.Component { + render() { + return (

Hello World!

) + } +} -// Import local files -// -// Local files can be imported directly using relative -// paths "./socket" or full ones "web/static/js/socket". - -// import socket from "./socket" +ReactDOM.render( + , + document.getElementById("hello-world") +)

web/templates/layout/app.html.eex

<body> - <div class="container"> - <header class="header"> - <nav role="navigation"> - <ul class="nav nav-pills pull-right"> - <li><a href="http://www.phoenixframework.org/docs">Get Started</a></li> - </ul> - </nav> - <span class="logo"></span> - </header> - - <p class="alert alert-info" role="alert"><%= get_flash(@conn, :info) %></p> - <p class="alert alert-danger" role="alert"><%= get_flash(@conn, :error) %></p> - - <main role="main"> - <%= render @view_module, @view_template, assigns %> - </main> - - </div> <!-- /container --> + <div id="hello-world"></div> <script src="<%= static_path(@conn, "/js/app.js") %>"></script>

Static Assets

Now we can start working in broad strokes. Since this project is a direct clone of the Meteor Todos application, we’re not planning on modifying the application’s stylesheets.

This means that we can wholesale copy the contents of ~/todos/.meteor/.../merged-stylesheets.css into our web/static/css/app.css file.

We can also copy all of the Meteor application’s static assets into our web/static/assets folder, and update our Phoenix Endpoint to make them accessible.

Reloading our application should show us a nice gradient background, and we shouldn’t see any Phoenix.Router.NoRouteError errors when trying to access our static assets.

lib/phoenix_todos/endpoint.ex

... at: "/", from: :phoenix_todos, gzip: false, - only: ~w(css fonts images js favicon.ico robots.txt) + only: ~w(css font js icon favicon.ico favicon.png apple-touch-icon-precomposed.png logo-todos.svg)

Layouts and Containers

Now that we have our basic React functionality set up and our static assets being served, it’s time to start migrating the React components from the Meteor Todos application into our new Phoenix application.

We’ll start this process by changing our app.html.eex file to use the expected "app" ID on its container element.


<div id="app"></div>

Next, we can update our app.js file, removing our HelloWorld component, and replacing it with the setup found in the Todos application. We need to be sure to remove the Meteor.startup callback wrapper, as we won’t be using Meteor:


import { render } from "react-dom";
import { renderRoutes } from "./routes.jsx";

render(renderRoutes(), document.getElementById("app"));

Now we port over the routes.jsx file. We’ll put this directly into our web/static/js folder, next to our app.js file.

We’ll keep things simple at first by only defining routes for the AppContainer and the NotFoundPage.


import AppContainer from './containers/AppContainer.jsx';
import NotFoundPage from './pages/NotFoundPage.jsx';

export const renderRoutes = () => (
  <Router history={browserHistory}>
    <Route path="/" component={AppContainer}>
      <Route path="*" component={NotFoundPage}/>
    </Route>
  </Router>
);

The AppContainer in the Meteor application defines a reactive container around an App component. This is very Meteor-specific, so we’ll gut this for now and replace it with a simple container that sets up our initial application state and passes it down to the App component.

Next comes the process of migrating the App component and all of its children components (UserMenu, ListList, ConnectionNotification, etc…).

This migration is fairly painless. We just need to be sure to remove references to Meteor-specific functionality. We’ll replace all of the functionality we remove in future commits.

After all of these changes, we’re greeted with a beautifully styled loading screen when we refresh our application.

package.json

"react": "^15.3.1", + "react-addons-css-transition-group": "^15.3.1", "react-dom": "^15.3.1", + "react-router": "^2.7.0", "uglify-js-brunch": ">= 1.0 < 1.8"

web/static/js/app.js

-import React from "react" -import ReactDOM from "react-dom" +import { render } from "react-dom"; +import { renderRoutes } from "./routes.jsx"; -class HelloWorld extends React.Component { - render() { - return (<h1>Hello World!</h1>) - } -} - -ReactDOM.render( - <HelloWorld/>, - document.getElementById("hello-world") -) +render(renderRoutes(), document.getElementById("app"));

web/static/js/components/ConnectionNotification.jsx

+import React from 'react'; + +const ConnectionNotification = () => ( + <div className="notifications"> + <div className="notification"> + <span className="icon-sync"></span> + <div className="meta"> + <div className="title-notification">Trying to connect</div> + <div className="description">There seems to be a connection issue</div> + </div> + </div> + </div> +); + +export default ConnectionNotification;

web/static/js/components/ListList.jsx

+import React from 'react'; +import { Link } from 'react-router'; + +export default class ListList extends React.Component { + constructor(props) { + super(props); + + this.createNewList = this.createNewList.bind(this); + } + + createNewList() { + const { router } = this.context; + // TODO: Create new list + } + + render() { + const { lists } = this.props; + return ( + <div className="list-todos"> + <a className="link-list-new" onClick={this.createNewList}> + <span className="icon-plus"></span> + New List + </a> + {lists.map(list => ( + <Link + to={`/lists/${ list._id }`} + key={list._id} + title={list.name} + className="list-todo" + activeClassName="active" + > + {list.userId + ? <span className="icon-lock"></span> + : null} + {list.incompleteCount + ? <span className="count-list">{list.incompleteCount}</span> + : null} + {list.name} + </Link> + ))} + </div> + ); + } +} + +ListList.propTypes = { + lists: React.PropTypes.array, +}; + +ListList.contextTypes = { + router: React.PropTypes.object, +};

web/static/js/components/Loading.jsx

+import React from 'react'; + +const Loading = () => ( + <img src="/logo-todos.svg" className="loading-app" /> +); + +export default Loading;

web/static/js/components/Message.jsx

+import React from 'react'; + +const Message = ({ title, subtitle }) => ( + <div className="wrapper-message"> + {title ? <div className="title-message">{title}</div> : null} + {subtitle ? <div className="subtitle-message">{subtitle}</div> : null} + </div> +); + +Message.propTypes = { + title: React.PropTypes.string, + subtitle: React.PropTypes.string, +}; + +export default Message;

web/static/js/components/MobileMenu.jsx

+import React from 'react'; + +function toggleMenu() { + // TODO: Toggle menu +} + +const MobileMenu = () => ( + <div className="nav-group"> + <a href="#" className="nav-item" onClick={toggleMenu}> + <span className="icon-list-unordered" title="Show menu"></span> + </a> + </div> +); + +export default MobileMenu;

web/static/js/components/UserMenu.jsx

+import React from 'react'; +import { Link } from 'react-router'; + +export default class UserMenu extends React.Component { + constructor(props) { + super(props); + this.state = { + open: false, + }; + this.toggle = this.toggle.bind(this); + } + + toggle(e) { + e.stopPropagation(); + this.setState({ + open: !this.state.open, + }); + } + + renderLoggedIn() { + const { open } = this.state; + const { user, logout } = this.props; + const email = user.emails[0].address; + const emailLocalPart = email.substring(0, email.indexOf('@')); + + return ( + <div className="user-menu vertical"> + <a href="#" className="btn-secondary" onClick={this.toggle}> + {open + ? <span className="icon-arrow-up"></span> + : <span className="icon-arrow-down"></span>} + {emailLocalPart} + </a> + {open + ? <a className="btn-secondary" onClick={logout}>Logout</a> + : null} + </div> + ); + } + + renderLoggedOut() { + return ( + <div className="user-menu"> + <Link to="/signin" className="btn-secondary">Sign In</Link> + <Link to="/join" className="btn-secondary">Join</Link> + </div> + ); + } + + render() { + return this.props.user + ? this.renderLoggedIn() + : this.renderLoggedOut(); + } +} + +UserMenu.propTypes = { + user: React.PropTypes.object, + logout: React.PropTypes.func, +};

web/static/js/containers/AppContainer.jsx

+import App from '../layouts/App.jsx'; +import React from 'react'; + +export default class AppContainer extends React.Component { + constructor(props) { + super(props); + this.state = { + user: undefined, + loading: true, + connected: true, + menuOpen: false, + lists: [] + }; + } + + render() { + return (<App {...this.state}/>); + } +};

web/static/js/layouts/App.jsx

+import React from 'react'; +import ReactCSSTransitionGroup from 'react-addons-css-transition-group'; +import UserMenu from '../components/UserMenu.jsx'; +import ListList from '../components/ListList.jsx'; +import ConnectionNotification from '../components/ConnectionNotification.jsx'; +import Loading from '../components/Loading.jsx'; + +const CONNECTION_ISSUE_TIMEOUT = 5000; + +export default class App extends React.Component { + constructor(props) { + super(props); + this.state = { + menuOpen: false, + showConnectionIssue: false, + }; + this.toggleMenu = this.toggleMenu.bind(this); + this.logout = this.logout.bind(this); + } + + componentDidMount() { + setTimeout(() => { + /* eslint-disable react/no-did-mount-set-state */ + this.setState({ showConnectionIssue: true }); + }, CONNECTION_ISSUE_TIMEOUT); + } + + componentWillReceiveProps({ loading, children }) { + // redirect / to a list once lists are ready + if (!loading && !children) { + const list = Lists.findOne(); + this.context.router.replace(`/lists/${ list._id }`{:.language-javascript}); + } + } + + toggleMenu(menuOpen = !Session.get('menuOpen')) { + Session.set({ menuOpen }); + } + + logout() { + Meteor.logout(); + + // if we are on a private list, we'll need to go to a public one + if (this.props.params.id) { + const list = Lists.findOne(this.props.params.id); + if (list.userId) { + const publicList = Lists.findOne({ userId: { $exists: false } }); + this.context.router.push(`/lists/${ publicList._id }`{:.language-javascript}); + } + } + } + + render() { + const { showConnectionIssue } = this.state; + const { + user, + connected, + loading, + lists, + menuOpen, + children, + location, + } = this.props; + + const closeMenu = this.toggleMenu.bind(this, false); + + // clone route components with keys so that they can + // have transitions + const clonedChildren = children && React.cloneElement(children, { + key: location.pathname, + }); + + return ( + <div id="container" className={menuOpen ? 'menu-open' : ''}> + <section id="menu"> + <UserMenu user={user} logout={this.logout}/> + <ListList lists={lists}/> + </section> + {showConnectionIssue && !connected + ? <ConnectionNotification/> + : null} + <div className="content-overlay" onClick={closeMenu}></div> + <div id="content-container"> + <ReactCSSTransitionGroup + transitionName="fade" + transitionEnterTimeout={200} + transitionLeaveTimeout={200} + > + {loading + ? <Loading key="loading"/> + : clonedChildren} + </ReactCSSTransitionGroup> + </div> + </div> + ); + } +} + +App.propTypes = { + user: React.PropTypes.object, // current meteor user + connected: React.PropTypes.bool, // server connection status + loading: React.PropTypes.bool, // subscription status + menuOpen: React.PropTypes.bool, // is side menu open? + lists: React.PropTypes.array, // all lists visible to the current user + children: React.PropTypes.element, // matched child route component + location: React.PropTypes.object, // current router location + params: React.PropTypes.object, // parameters of the current route +}; + +App.contextTypes = { + router: React.PropTypes.object, +};

web/static/js/pages/NotFoundPage.jsx

+import React from 'react'; +import MobileMenu from '../components/MobileMenu.jsx'; +import Message from '../components/Message.jsx'; + +const NotFoundPage = () => ( + <div className="page not-found"> + <nav> + <MobileMenu/> + </nav> + <div className="content-scrollable"> + <Message title="Page not found"/> + </div> + </div> +); + +export default NotFoundPage;

web/static/js/routes.jsx

+import React from 'react'; +import { Router, Route, browserHistory } from 'react-router'; + +// route components +import AppContainer from './containers/AppContainer.jsx'; +import NotFoundPage from './pages/NotFoundPage.jsx'; + +export const renderRoutes = () => ( + <Router history={browserHistory}> + <Route path="/" component={AppContainer}> + <Route path="*" component={NotFoundPage}/> + </Route> + </Router> +);

web/templates/layout/app.html.eex

<body> - <div id="hello-world"></div> + <div id="app"></div> <script src="<%= static_path(@conn, "/js/app.js") %>"></script>

Final Thoughts

This first installment of “Phoenix Todos” mostly consisted of “coding by the numbers”. Migrating over the styles, static assets, and front-end components of our Meteor application into our Phoenix application is tedious work to say the least.

Expect the excitement levels to ramp up in future posts. We’ll be implementing a Phoenix-style authentication system, replacing Meteor publications with Phoenix Channels, and re-implementing Meteor methods as REST endpoints in our Phoenix application.

This conversion process will open some very interesting avenues of comparison and exploration, which I’m eager to dive into.

Be sure to check back next week for Phoenix Todos - Part 2!

Assessing Mobile Meteor Applications

Written by Pete Corey on Aug 29, 2016.

Some Meteor applications are released solely as mobile applications. They’re intended to be experienced natively as a Cordova-powered application, rather than on the web through a browser.

From a security perspective, does this matter? Are security assessments for mobile-only applications approached differently than web-only, or web and mobile applications?

The answer to both questions is a resounding no!

Web is Always an Option

An interesting side-effect of the Meteor build process means that the “web” version of an application is always accessible, even if you intended to release it exclusively as a native mobile application.

During the mobile build process, you point your application at a hosted Meteor server. The mobile application communicates with the server, pulling down data and updated application code.


> meteor help build
...
Options:
  --server  Location where mobile builds connect to the Meteor server.
            Defaults to localhost:3000. Can include a URL scheme
            (for example, --server=https://example.com:443).

As expected, the application can also be accessed by navigating to this server URL directly with a browser.

This browser build can’t be disabled with current versions of Meteor. Trying to remove the “browser” platform results in an error:


> meteor remove-platform browser

While removing platforms:
error: browser: cannot remove platform in this version of Meteor

This means that the front-end of a Meteor application can always be seen by prying eyes.

Unzipping The Application

Let’s imagine that we’re trying to assess a Meteor mobile application called FooApp.

When we only have access to the compiled mobile application, how can we discover the Meteor server’s URL?

It turns out this is a fairly straight-forward process. We’ll dig into it for iOS applications (*.ipa archive files), but the same process applies to Android applications (*.apk archive packages).

Once we’ve downloaded FooApp through iTunes, its *.ipa file can usually be found at ~/Music/iTunes/iTunes Media/Mobile Applications/FooApp<version>.ipa.

Interestingly, iOS application archives can be unzipped using a standard archiving tool. The first step to discovering our server URL is to unzip the archive:


unzip FooApp<version>.ipa -d FooApp

We can now peruse through the contents of the FooApp bundle in the FooApp folder.

Finding the Server

Once we’ve unzipped our application, the server URL is within our reach.

To discover the server URL, open FooApp/Payload/FooApp.app/www/application/index.html. In that file, you’ll find a URL-encoded __meteor_runtime_config__ variable.

You can copy and paste that __meteor_runtime_config__ declaration into a browser console, and then print it in a more human friendly format:


__meteor_runtime_config__ = JSON.parse(decodeURIComponent("..."));
JSON.stringify(__meteor_runtime_config__, null, 2);

The result should look something like this:


{
  "meteorRelease": "METEOR@1.3.3-ddp-batching-beta.0",
  "ROOT_URL": "https://www.fooapp.com/",
  "ROOT_URL_PATH_PREFIX": "",
  "DDP_DEFAULT_CONNECTION_URL": "https://www.fooapp.com/",
  "autoupdateVersionCordova": "86e83cfe388118db86733f1333e3a2962fcad1b6",
  "appId": "ABCDEFGHIJ1234567890",
  "meteorEnv": {
    "NODE_ENV": "production"
  }
}

You’ll notice that both ROOT_URL and DDP_DEFAULT_CONNECTION_URL point to "https://www.fooapp.com/". This is the server URL that we’ve been searching for!

Navigating to the server would deliver all of the client-side code to our browser (even if it’s guarded by a Meteor.isCordova check), and give us access to call all Meteor methods and publications.

Now we can assess our mobile Meteor application just like any other Meteor application!