Skip to main content

Biggest project management mistake

This holds true for developers, project managers, entrepreneurs, everybody who has shipped software to the real world. The biggest mistake one can make while shipping software is not shipping buggy code or half-baked features (and they are all easy to uncover given sufficient QA), or not planning for failure. The biggest mistake that folks make is - not planning for success.

Trust me - shit breaks loose when your app gets hot over the night! So here's a list of concerns one needs to plan for, before launching a site:

Caching
Cache everything - images, javascript, style sheets, html templates - everything. I use nginx at the frontend proxy layer and it does a full page caching for entire html pages. Works like a charm!
At the application layer, use Redis as a TTL-based cache.

Deliver static content really fast
Images, JS, stylesheets - they need to be rendered by a specialized static files host - a CDN in essence. Use a managed service provider (Amazon CloudFront), or deploy nginx (with local caching) over a separate box and serve yourself.

Scaling load-balancers/frontend proxies
Plan your deployment in such a way that adding and removing first layer application servers should be ridiculously simple. This means a load-balancer is an absolute must. Here again - you can use Amazon ELB or deploy nginx or haproxy.  But if you happen to deploy a load-balancer yourself, keep in mind that servers die all the time and you don't want your release sabotaged by a dead load-balancer. Plan for failure. What happens when the load-balancer dies? You spin up a new one! And that should be ridiculously simple - use AMIs, docker, vagrant, puppet, chef, anything.

Scaling application servers
Keep a couple of application servers in standby so that when the load comes in, you can just plow them in quickly. A better strategy is to use docker or other packaging tools which let you spawn applications quickly.
Use event-based I/O at application server layer. For python (flask, django) applications, I use uwsgi as the app container. Gunicorn is also a good choice.

Scaling Databases
Index your tables and views correctly. Shard. This machine needs to be high on memory. Increase max parallel connections setting before going live.

Logging and alerts
You cannot improve something you cannot measure. And your users mailing you about app crashes is not the best idea of usability testing. Use NewRelic agents on your web server components, use Crashlytics in your mobile app. If hell is breaking loose - you should be the first to know, not your users. ELK and ElastAlert make a good choice too, but for beginners, a NewRelic based alerting mechanism is good enough. Amazon users can very well use CloudWatch.


This is part one of the post. In part two, I will talk about:
  1. Automation and tests
  2. Blue-green deployments
  3. Reporting
  4. Backups
  5. Logging


Comments

Popular posts from this blog

सूनापन

मुद्दत हो गयी उन तन्हायियो को गुजरे , फिर भी इन आँखों में नमी क्यों है  ? तोड़ दिया मोहब्बत पर से यकीन मेरा, फिर भी मेरी दुनिया में तेरी कमी क्यों है ? हसरत है क्यों आज भी तेरी चाहत की मुझे, क्यों याद तेरी जेहेन से मिटती नहीं ? जलजला क्यों उमड़ता है ख्वाबो में मेरे, उस आशिकी की आगज़नी क्यों है  ? सन्नाटो में भी क्यों सुनता हू तुझे मेरी परछाई से क्यों तू जाती नहीं ? इन डबडबाती आँखों को तलाश तेरी, आज भी कहीं क्यों है   ?

How the Python import system works

How the Python import system works From:  https://tenthousandmeters.com/blog/python-behind-the-scenes-11-how-the-python-import-system-works/ If you ask me to name the most misunderstood aspect of Python, I will answer without a second thought: the Python import system. Just remember how many times you used relative imports and got something like  ImportError: attempted relative import with no known parent package ; or tried to figure out how to structure a project so that all the imports work correctly; or hacked  sys.path  when you couldn't find a better solution. Every Python programmer experienced something like this, and popular StackOverflow questions, such us  Importing files from different folder  (1822 votes),  Relative imports in Python 3  (1064 votes) and  Relative imports for the billionth time  (993 votes), are a good indicator of that. The Python import system doesn't just seem complicated – it is complicated. So even though the  documentation  is really good, it d

I was fat once! Now I run!

On Aug 12, 2013 I weighed 107 kg, which, for a 5'4'' guy, is 47 kg more than the Indian Army standards. But today, I weigh 82 kg, and I continue to lose a few pounds every week! At one point of time, running 100 metres would make me all tired and sweaty, but today I can easily run 10 km at a stretch! So this guide is for all those couch potatoes who have lost all hopes of getting a good body! I was at that stage myself. I know how it feels like. I know the bullshit that you keep telling yourself as to how you are not all that bad, and how the world should accept you the way you are! Let me tell you this straight! You are NOT special. You are just another lazy bum who wants things to change but does not want to endure any suffering! Or even worse, you are one of those who act on an impulse every few months. You hit the gym for a week and boom!, you lose it all again. The first step is to accept that you have a pathetic body and that this needs to change! Start Sm