I started my career at a Big 4 consulting firm in the mid-90s. First year analysts at the firm were referred to as “green beans” and boy was I green. Educated as a Materials Science Engineer and without any real programming education in college, I had a lot to learn and I had to learn it fast. I was also lucky enough to be part of an emerging technology group called the Internet Center of Excellence or "ICE".
With the retail adoption of the Internet, companies were investing heavily to get their first generation websites off the ground. At the same time, internet technologies were changing rapidly with very little standardization; which equated to our group working our butts off trying to get dot-com and brick-n-mortar sites launched on rickety app servers, poorly performing databases, 10 MB networks and limited, expensive, hardware.
Needless to say, our biggest problems were usually around performance. Each and every problem was unique, sometimes having to do with the App Server waiting for a database call, a Web Server waiting for an App Server to finish processing or a web browser waiting for some HTML to be downloaded. There was never a silver bullet to these problems, but as a trained engineer, I had a pretty good knack for using the scientific method to isolate variables and determine what the route causes were, so I got assigned a lot of the performance problems to debug.
I was an applications guy and I would work with the guys on the infrastructure side to figure stuff out. One day I was racking my brain on a particularly tough performance problem. We were working on a large greeting card company’s e-Commerce site and the ‘Checkout’ function was taking forever. The lead infrastructure engineer on our team was a guy by the name of Garren Du. Garren was a cocky, but likeable, New Yorker who was very smart and swore a lot. He always seemed to have a can of soda in his hand and wore a sly smile on his face when he talked, often making brash statements, hoping to strike up a good-natured argument. Garren was one of the more senior guys on the project (and when I say senior, I mean like, 25). Being a 22-year old green bean I looked up to him and thought him wise beyond his years. Turns out, he actually was.

Struggling with a particularly tough issue, trying to figure yet ANOTHER application server, I asked Garren, “How do you keep up with the Internet technology changing so fast? It makes it so hard to pinpoint problems. Every project is a different database, a different app server a different web server. It’s impossible to know it all!”
His response was simple, “It’s all TCP/IP, brother.”
I wasn’t shy about my greenness in those days, “What’s TCP/IP”?
“It’s a standard networking protocol, it’s the way all this data is packaged up and sent flying around the network and the Internet. I don’t care what the server is, call it what you want, Informix, Blue Martini, BroadVision. It’s all the same sh*t. It’s just TCP/IP man. I can write a shell script in Unix, pop open a TCP/IP socket connection and do the same thing that any of those overpriced software packages claim to do.”
“Yeah, but how’s that going to help me fix these performance problems?” I asked.
“Because it’s simple man. SMALL DATA. Small data means fast performance. You keep your data small, your application is going to be quick. You got big data, you got big problems.”
“OK well how do I keep data small?”
“Small and infrequent queries to your database. Getting only what you need. Small content output from your app server so it doesn’t take forever to get to the web server. Small images on your webserver and small includes on your web pages, so it doesn’t take forever to download stuff. And here’s the other thing. Keep the data
close to your end users. The further away data is, the longer it’s going to take to get it to them. It’s simple physics man. If the data is on the web server, it'll be fast; if it has to go to the app server, slower; back to the DB, slower; to some other external systems, SLOWER. So you've GOT to cache that sh*t, on every tier. Limit that TCP/IP traffic. . .But not too much or you’ll f*ck up your memory.”
Garren was smiling, while my head was spinning, Garren said, “You can comb through that code for hours looking for the needle that’s causing those problems. Me, I packet sniff the network stream and I can tell you exactly why something is slow. It’s big data man.” He smiled, knowingly. “Now come on let’s go get a coke.”
Garren's simplistic view of the Internet holds all the more true for mobile applications today. Now more than ever, we’re dealing with limited hardware (cell phones), limited network speed (cell networks) and rickety new technology (iOS, Android, etc.). So how do we deal with it? Keep it small, brother. There's a million manifestations or practical tips on this, here are a few of my favorites:
- Limit the download footprint of your mobile web pages/services by minifying the whitespace in your code. Don't include content that is not going to be displayed or used. This includes heavyweight javascript libraries or irrelevant CSS files.
- Limit your use of cookies. Cookies get transferred with every request/response. That's extra data being transported over TCP/IP that probably isn't being used.
- Avoid redirects in your codebase. Redirects cause additional TCP/IP roundtrips between the client and the server. They're brutal over slow connections.
- Batch your data requests. Limit the amount of times you need to go back to the database for any given page or service. This can apply to the front end as well by using CSS sprites instead of having individual images downloaded. That's less data being downloaded in fewer TCP/IP trips.
- Cache cache cache. Use Cache Control headers on your services to your advantage, and use the power of the internet to cache relevant parts of your app. If you want more details, read this. And cache on other tiers as well. Use HTML5 DOM Storage to cache information locally in the browser. Pre-generate dynamic content asynchronously and store it on your web server instead of having your app server generate it on-demand everytime. Store common database queries in your app server's application scope, and share it amongst users. Limit that TCP/IP traffic!
Regardless if you're building mobile apps, or web apps, or client server or Minority Report apps, this premise will always hold true. Keep it small. It's all TCP/IP Brother. Big Data means Big Problems.
Let's help each other out and put some other performance tips and tricks in the comments below, eh?
Happy coding,
-J