When a Meteor finally hits production
I am talking about a Meteor app of course.
In the last 5 months or so, I have been creating Meteor environments from scratch on AWS, using a bunch of different tools and I would like to share what I’ve learned with you.
Here is what I will write about:
- Environment Setup Tools
- Setting up database for production
- Redundancy
- Performance
- Mobile App Build
- Database Migrations
- Continuos Deployment
We will start at the end of writing your demo
Production is not like the demo app it took a week to write. Meteor is one of the best platforms out there to write demo apps with. But, what happens after your app have some clients? Now you have a bit (or allot) of responsibility for making your app as reactive, available and fresh as possible. With these new requirements the old ‘meteor deploy’ action is not an option anymore for a couple of reasons, one (and most convincing) being that there are no guarantees that your app is even running.
With that said, you will still want to have that kind of simplicity when you deploy… (fast bug fixes?)
Best option out there right now for deployment with ease, into your own AWS servers, would be to use meteor-up. The link I have provided has everything you need in order to install and configure your servers within minutes. However, I will try to create some best practices about configuring your environment.
The base mup.json file looks like this:
{
// Server authentication info
"servers": [
{
"host": "hostname",
"username": "root",
"password": "password"
// or pem file (ssh based authentication)
// WARNING: Keys protected by a passphrase are not supported
//"pem": "~/.ssh/id_rsa"
// Also, for non-standard ssh port use this
//"sshOptions": { "port" : 49154 },
// server specific environment variables
"env": {}
}
],
"setupMongo": true,
"setupNode": true,
"nodeVersion": "0.10.36",
"setupPhantom": true,
"enableUploadProgressBar": true,
"appName": "meteor",
"app": "/Users/arunoda/Meteor/my-app",
"env": {
"PORT": 80,
"ROOT_URL": "http://myapp.com",
"MONGO_URL": "mongodb://arunoda:fd8dsjsfh7@hanso.mongohq.com:10023/MyApp",
"MAIL_URL": "smtp://postmaster%40myapp.mailgun.org:adj87sjhd7s@smtp.mailgun.org:587/"
},
"deployCheckWaitTime": 15
}You most likely create this file within the root folder of your project and set the “app” value to be “.” — which will work perfectly, but. Once you make the decision to create a production environment, the staging env is inevitable. For that reason, I would set up the mup.json file in a directory on the project tree that is dedicated for deployment configuration and create a dir for each environment. So ultimately you will have:
- PROJECT_ROOT/.config/deployment/production/mup.json
- PROJECT_ROOT/.config/deployment/staging/mup.json
Note that I have introduced a dot on the name of my config dir. This will tell the Meteor build system the directory and it’s subtree should not be included in the different apps (mobile, browser, server).
The base mup.json file will tell mup to install a local mongo instance within the servers you setup. This will not create a mongo replica set for you and because of that I prefer to turn this feature off.
The recommended Mongo setup for production use is a replica set with at least 2 active nodes and an arbiter. This information could be useless to you if you chose one of the hosted solutions of mongo (i.e MongoLab). But the dedicated machines option could be much pricier than creating your own replica set with (at least) two dedicated EC2 machines. So as I have ultimately concluded for standard to medium scale (Someone said startups?) the economic choice would be to create your own setup.
A bigger scale project can choose from a wider range of options and should factor in the amount of time spent on maintenance of your own machines so I will not address these right now.
Setting up a replica set
I chose to create two EC2 instances dedicated for two active mongo nodes and used the web server as an arbiter. This will be a good starting point and could always joined with another node for reading on other geographical locations. You can go as big as you want with these machines, I went with the t2.medium as my current needs are narrowed to reactivity on the east coast mostly and the size of the data is not very large. I have created a dedicated security group with the default mongo port (27017) that is restricted to this security group owners. This will later let the mongo nodes and the app server an access to each other while restrict access from the WAN.
After that I’ve installed MongoDB version 2.6 on each of my servers (including the web server). After that I’ve used the tutorial available on docs.mongodb.org to create a secure replica set.
After the replica set is up and running there is one crucial thing to do, in order to enable the ability of Meteor to read changes from mongo’s oplog. To create a user with read permissions on the oplog you need to run these commands:
use admin;
db.auth(ADMIN_USER, ADMIN_PASSWORD);
db.createUser({user: “oplogger”, pwd: NEW_PASSWORD, roles: [{role: “read”, db: “local”}]});
Make sure to run this command on your primary node of the replica set.
In order to configure Meteor to use the oplogger user you created just now, you need to update the mup.json env section to include the following line:
“MONGO_OPLOG_URL” : “mongodb://oplogger:OPLOGGER_PASSWORD@first_server:27017,second_server:27017/local?authSource=admin”
Mup deploy time!
Now, after you finished configuring the your replica set it’s time to deploy your app using the simple and awesome mup deploy. As I said before you should create a directory for each environment so in order to deploy into production what you need to do is, cd into the dir that corresponds to your production and run `mup deploy`.
As meteor-up’s README states — you should first run `mup setup` in order for mup to install node, phantomJS (optional) and stud for SSL. It will also create the directories where your app will be deployed to.
Redundancy
For a production environment, it will be wise to setup at least two web servers right from the start and using mup it is really easy. Just add another server object on the servers field of your mup.json file, and change the instance address.
A low cost and simple solution for routing would be to configure a second A Record on your DNS so it will forward some of your clients to that second server. I am unsure though of how this will behave for hot code push to mobile apps. What I know about this matter is that the versioning of the mobile app bundle is signed on the build process with a version id and that the mobile cordova client will only download a the bundle from the version if it is different. So, theoretically speaking, as long as you deploy with mup and both servers get updated successfully, there should not be any problems. One more concern of mine is with sticky sessions and web sockets. Static content is return via address 1 and web sockets connection is established with address 2. That should not be a problem for Meteor I would think. Please tell me if you have any experience with that kind of setup. I will definitely play around with it in the near future.
A more common solution will be to put a load balancer in front of your web server. This load balancer should support sticky sessions and web sockets, but you can read more about it on Meteor Hacks post.
Performance
First of all, let’s agree that trying to handle performance issues without profiling tools is close to imposible. I have no intention to promote Kadira by any way (mostly because I am not payed to do so). That being said, I’ve had a very positive experience using it and I strongly encourage you to use it for some time and check out the info provided from it on your app.
I will note that if any of you know of a better product (maybe an open source product) that is better I would love to hear about it!
Using Kadira you can pin point the areas of your code base that has performance issues. After those areas are found you should start thinking about what will be the strategy to handle with it. If its a publish function that takes too much time try adding mongo indexes on the fields that you use. Also, think of handling the addition, change and remove events of the publish function yourself. In some cases doing so could cause worse performance than just using the default “return cursor” method, so make sure to check the differences and decide accordingly. In some cases, you will have long running methods that serve a purpose on your app and cannot be reduced into shorter running times. Because of the single threaded nature of Node (and because of that, Meteor) these methods should not run on the same process as your front end server. You can create a companion Meteor app that will work on your mongo collections as you need or you could also use the micro services feature of meteorhacks:cluster package.
If your problem seems to be related with real load than you should consider scale horizontally and get a load balancer in front of your web server. Try this post (mentioned before) for more information on the matter.
If your current code base does not let you scale horizontally than you should definitely prioritise this over other features as this is a necessity with Meteor apps as you grow your client base. Try to export the problematic code into a micro service (with cluster) that will run only on a single cluster node and scale the rest of your server to more nodes.
Mobile App
It is so easy to create a mobile app with Meteor and Cordova. This will get a bit messier as you move to a production build. One thing you should do is configure your mobile app build with your DNS address instead of the server hostname. This will put you in a state that no matter what you will decide to do with your mobile app’s codebase (Beside adding another cordova plugin or changing your DNS address), You will not have to upgrade your mobile app version. If you are new to apple’s iOS release process, just try it once and behold this feature’s mighty power!
How to configure the mobile app build process:
meteor build --server http://your.dns.address TARGET_DIRAfter doing this once, all you have to do for pushing an update (that will not change any cordova plugin) is to publish a new build to your server with ‘mup deploy’
Database Migrations
Ok, so regarding this subject I am a bit inclined to recommend my own package: davidyaha:collection2-migrations.
This package is a complementing package for aldeed:collection2 and will use your, already written, simple schema objects to track changes in the database structure, and migrate your existing data accordingly. Just make sure to create decent auto-values and default-values for any required field.
To be fair, this is the competition:
I have not tried either because I was already using collection2 and wanted a compatible solution. Although, I would love to hear suggestions for improvements.
Continuos Deployment
So we are using CircleCI. I do not know how to cofigure it for other CI’s but I want to share what I’ve done and it will possibly be similar to do the same on other CI environments.
My circle.yml:
dependencies:pre:
- npm install -g mup # install meteor-up for deploymentdeployment:production:
branch: production
commands:
- cd PROJECT_DIR/.build/deployment/production && mup deploystaging:
branch: staging
commands:
- cd PROJECT_DIR/.build/deployment/staging && mup deploy
This will deploy my app each time I merge into branch “staging” or “production” (each to it’s environment). Simple and easy :)
Other Solutions
Dedicated cloud solutions:
Upcoming:
