Meteor and Mongod.lock

Written by Pete Corey on Feb 16, 2015.

When Mongo closes incorrectly, (due to things like crashes, hard reboots, etc…), it leaves behind a non-zero byte mongod.lock file. The presence of this file indicates that Mongo wasn’t cleaned up correctly and will prevent Mongo from starting normally.

In a Meteor project, meteor will fail with the following messages if it detects that Mongo was not correctly shut down:

=> Started proxy.
Unexpected mongo exit code 100. Restarting.
Unexpected mongo exit code 100. Restarting.
Unexpected mongo exit code 100. Restarting.
Can't start Mongo server.
MongoDB had an unspecified uncaught exception.
This can be caused by MongoDB being unable to write to a local database.
Check that you have permissions to write to .meteor/local. MongoDB does
not support filesystems like NFS that do not allow file locking.

To fix this, two things need to happen. First, remove the lock file:

% rm .meteor/local/db/mongod.lock

Not that the lock file is deleted, Meteor should start without any complaints from Mongo. To be safe, Mongo’s repair routine should be run on your Meteor database. Normally, this is done using a mongod command (mongod --repair), but since Meteor doesn’t use mongod, we need to kick off the repair from within the Mongo shell:

% meteor mongo
> db.repairDatabase()
{ “ok” : 1}

That’s it! You can read more about how to recover data after an unexpected shutdown in the Mongo docs.

Meteor Velocity: Down the Debugging Rabbit Hole

Written by Pete Corey on Feb 9, 2015.

Meteor Requires Node v0.10.33 or Later

The other day I added Velocity to a Meteor app (meteor add mike:mocha) and I was greeted with a strange error when the server restarted:

W20150130-22:24:55.285(-8)? (STDERR) [velocity-mirror] Meteor requires Node v0.10.33 or later.

Hmm. Doesn’t the current version of Meteor (1.0.3) use its own instance of Node v0.10.33? Let’s check…

% meteor shell
> process.argv[0]
% /home/pcorey/.meteor/packages/meteor-tool/.1.0.40.moil5k++os.linux.x86_32+web.browser+web.cordova/meteor-tool-os.linux.x86_32/dev_bundle/bin/node --version

Just like I thought, Meteor is using Node v0.10.33. So where is this error coming from? Let’s do some monkey patching to find out!

Monkey Patching to the Rescue

I started by editing Meteor’s boot.js (/home/pcorey/.meteor/packages/meteor-tool/.1.0.40.moil5k++os.linux.x86_32+web.browser+web.cordova/meteor-tool-os.linux.x86_32/tools/server/boot.js). I overwrote console.error with my own custom function which prints the current call stack and then logs the error as usual:

oldConsoleError = console.error;
console.error = function() {
    var orig = Error.prepareStackTrace;
    Error.prepareStackTrace = function(_, stack){ return stack; };
    var err = new Error;
    Error.captureStackTrace(err, arguments.callee);
    var stack = err.stack;
    Error.prepareStackTrace = orig;

    oldConsoleError.apply(this, arguments);

So now what happens when we start the Meteor server?

(STDERR) Socket.<anonymous> (/home/pcorey/velocity-test/.meteor/local/build/programs/server/packages/velocity_node-soft-mirror.js:493:17)
(STDERR) Socket.emit (events.js:95:17)
(STDERR) Socket.<anonymous> (_stream_readable.js:764:14)
(STDERR) Socket.emit (events.js:92:17)
(STDERR) emitReadable_ (_stream_readable.js:426:10)
(STDERR) emitReadable (_stream_readable.js:422:5)
(STDERR) readableAddChunk (_stream_readable.js:165:9)
(STDERR) Socket.Readable.push (_stream_readable.js:127:10)
(STDERR) Pipe.onread (net.js:528:21)
(STDERR) [velocity-mirror] Meteor requires Node v0.10.33 or later.

Interesting… It looks like we should take a look at velocity_node-soft-mirror.js:493:

mirrorChild.getChild().stderr.on('data', function (data) {
    console.error('[velocity-mirror]', data.toString());

So this is where the error is coming from. mirrorChild is a child node process spawned a few lines earlier:

    command: 'node',
    args: [mainJs],
    options: {
        silent: true,
        detached: true,
        cwd: process.env.PWD,
        env: _.defaults(environment, process.env)

It looks like it’s simply running node, which will use the version of node on the system’s PATH, not Meteor’s bundled version of node. Let’s check what version of node lives on the PATH:

% node --version

Mystery solved. Velocity is spawning a “mirror” instance of Meteor but it’s using whatever instance of node is installed on the machine, not the version bundled with Meteor.

Fixing the Problem

Interestingly, if we look through the node-soft-mirror package on Github, we can see that this problem exists in the tagged version 0.2.6 and below, but it was fixed in version 0.2.7. If we take a look at the meteor-mocha-web project on Github, we can see that it’s explicitly depending on version 0.2.4 of node-soft-mirror.

An easy way to fix this is to explicitly add version 0.2.7 or higher of node-soft-mirror to our project.

meteor add velocity:node-soft-mirror@0.3.1

After that, the Meteor server should start without issue and everything should work as expected.

Well, that was definitely an adventure. It was pretty obvious what was going on after I dug into the internals of the error and how Velocity uses its “mirror”, but hindsight is always 20/20. I’ve filed a bug on the meteor-mocha-web project to update their node-soft-mirror package.


The current version of mike:mocha uses the version installed on the system, not the version packaged with Meteor. Explicitly add velocity:node-soft-mirror v0.2.7 or above to fix the problem.

Announcing East5th!

Written by Pete Corey on Feb 4, 2015.

I’ve decided to take an enormous step in my career. At the end of 2014 I left my job working as a contact software developer, and decided to forge my own path by starting a small web consultancy and development shop. After going through the hoops of incorporation, I can happily say that my company, East5th, is now alive, well and accepting clients!

Throughout my career as a software developer, I’ve come to believe in many ideas and ideals related to the software creation process. I believe that software should be tailor made to meet a client’s needs. Discovering what those needs are is fundamentally as important, or more important than implementing a solution. I believe that software should be developed iteratively. A minimum viable product will soothe a vast majority of pains at the lowest cost to the client. I believe in using value based pricing as a means of focusing on quality over commodity. More than anything, I believe that clients are looking for a professional’s expertise to guide them through the process of building a software solution.

I couldn’t be more excited to put these beliefs into practice with East5th.

1pxsolidtomato will live on! I’ll continue blogging on a variety of technical topics, but I hope to mix it up with updates on East5th and my experiences as a business owner. Stay tuned!