Saturday, January 21, 2017

Before you "judge" a python programmer

If you're coming from a compiled language context, using exceptions for flow control would look odd to you. But here's the thing - in the python world, exceptions are super cheap and using them for flow control is the "idiomatic python" way!
In fact, exception-driven flow control is built right into the language itself e.g. the "for" loop in python terminates when the iterable raise a "StopIteration" exception!

There is a lot of material on the internet already on the topic, so if there's one thing you take out of this post, it's this - in python, it's "Easier to Ask for Forgiveness than Permission"!

Sunday, January 15, 2017

Capture and compare stdout in python unit tests

A recent fan of TDD, I set out to write tests for whatever comes my way. And there was one feature where the code would print messages to the console. Now - I had tests written for the API but I could not get my head around ways to capture these messages in my unittests.
After some searching and some stroke of genius, here's how I accomplished capturing stdout.

Tuesday, January 03, 2017

Biggest project management mistake

This holds true for developers, project managers, entrepreneurs, everybody who has shipped software to the real world. The biggest mistake one can make while shipping software is not shipping buggy code or half-baked features (and they are all easy to uncover given sufficient QA), or not planning for failure. The biggest mistake that folks make is - not planning for success.

Trust me - shit breaks loose when your app gets hot over the night! So here's a list of concerns one needs to plan for, before launching a site:

Cache everything - images, javascript, style sheets, html templates - everything. I use nginx at the frontend proxy layer and it does a full page caching for entire html pages. Works like a charm!
At the application layer, use Redis as a TTL-based cache.

Deliver static content really fast
Images, JS, stylesheets - they need to be rendered by a specialized static files host - a CDN in essence. Use a managed service provider (Amazon CloudFront), or deploy nginx (with local caching) over a separate box and serve yourself.

Scaling load-balancers/frontend proxies
Plan your deployment in such a way that adding and removing first layer application servers should be ridiculously simple. This means a load-balancer is an absolute must. Here again - you can use Amazon ELB or deploy nginx or haproxy.  But if you happen to deploy a load-balancer yourself, keep in mind that servers die all the time and you don't your want your release sabotaged by a dead load-balancer. Plan for failure. What happens when the load-balancer dies? You spin up a new one! And that should be ridiculously simple - use AMIs, docker, vagrant, puppet, chef, anything.

Scaling application servers
Keep a couple of application servers in standby so that when load comes in, your can just plow them in quickly. A better strategy is to use docker or other packaging tools which let you spawn applications quickly.
Use event-based I/O at application server layer. For python (flask, django) applications, I use uwsgi as the app container. Gunicorn is also a good choice.

Scaling Databases
Index your tables and views correctly. Shard. This machine needs to be high on memory. Increase max parallel connections setting before going live.

Logging and alerts
You cannot improve something you cannot measure. And your users mailing you about app crashes is not the best idea of usability testing. Use NewRelic agents on your web server components, use crack analytics in your mobile app. If hell is breaking loose - you should be the first to know, not your users. ELK and ElasticAlerts make a good choice too; but for beginners a NewRelic based alerting mechanism is good enough. Amazon users can very well use CloudWatch.

This is part one of the post. In part two, I will talk about:
  1. Automation and tests
  2. Blue-green deployments
  3. Reporting
  4. Backups
  5. Logging

Sunday, January 01, 2017

When you are a one-man startup

Everyone needs a hobby. And some take it more seriously than others. Here's some cliched startup gyaan for someone wanting to take the plunge!
  1. Time is money
  2. Have a time table and follow it, specially if you have a day job
  3. Sharpen your tools
  4. Don't spend time on anything you are not good at, because #1
  5. Automation is the key. if it can be automated, it should be automated
  6. Health comes first
  7. Develop to sell. If it cannot be sold, it should not be built
  8. Have an exit plan and follow it. "I'll shut shop if I cannot get 10 paying customers in 6 months"
  9. You are the average of five people you spend most of your time with
  10. Don't stop having fun!

Thursday, December 15, 2016

Learn frontend to get good at backend : lessons for my 20 year old self

Only when you are the consumers of your own products, can you design the best ones! Same philosophies go for API design and programming in general.

This advice may seem counter-intuitive at first, but trust me, the best way to get better at backend programming is to spend some time doing frontend! It will teach you a lot about how your APIs are being used in the "real world". A first-hand experience of API consumption will tell you a lot about your design philosophies!