Building SHOPULARITY: One Dude, A Lot of Coding, and a Rock Star's Couch in LA. Nuff?

October 20, 2011 - by alexsalkever

I tend to build little one-off, ridiculous web projects all the time in an effort to learn new skills or techniques that can help me in larger projects. Because I’ve been a “lifetime” customer of Joyent for the past ten years, I’ve been very satisfied using a shared hosting environment where I can pretty much go to town and build whatever I want in my preferred stack (LAMP) without too many limitations. When I started working SHOPULARITY, an online popularity contest where high-end stuff is given away, totally for free to the player that convinces enough other people they deserve the prize more than anyone else before time runs out, I realized it had some real legs and would only be suitable as a real-world application in a more scalable, dedicated environment. Joyent Cloud provides the perfect combination of performance, stability, scalability and pricing that made development of SHOPULARITY a lot less stressful than it could have been. (Note: this is a shameless but well deserved and totally heartfelt plug).

From Blank Slate to Fully Stocked Dev Environment

Joyent Cloud starts you off with, essentially, a blank slate. Of course, that blank slate includes SmartOS (an open source OS derived from OpenSolaris and optimized for the cloud), SFTP, SSH and several other essential packages that, if you know what you're doing, get you from zero to dev in no time. My workflow of choice is Notepad++, WinSCP, and Pandora|One tuned to the Phil Collins channel. SHOPULARITY's proof-of-concept came together rapidly over a weekend where I integrated with a social API aggregator to start hashing out the game mechanics and general feasibility of social sign-on as a login mechanism for an entire site. After showing the idea to a few friends and getting some feedback—and not being called a complete idiot—it was time to build.

Version one of SHOPULARITY (we're now at version two) was ugly to say the least. There was little distinction between the on-site Google Adsense ads and the actual content. The number one user complaint was that the site and concept was too good to be true (we give away awesome stuff for free, after all) and that SHOPULARITY had to be a front for some sort of spammy data-mining scheme that would steal your children and torture your pet (I promise it's not!). It was more important, in my mind, to build quickly, have the functionality down and prove there was a market and user base first before getting bogged down in the look and feel. Consequently, the bulk of my time was spent on coding, game mechanics, voting logic, database design and automation of processes. There's an entire CMS that was also constructed to manage SHOPULARITY without having to dive in to configuration files or the database layer directly. Everything was written completely from scratch in PHP, Javascript and various shell scripting languages, with no frameworks used to offer maximum flexibility and minimum code bloat. The social sign-on aggregation API from the proof of concept was abandoned for more control by doing it myself, which ultimately proved to be the right decision.

Code, Code, More Code, Sleep, Crash on Rock Star's Couch, More Code

While I'd like to say development happened very quickly, the consequences of building while having a full-time day job got in the way. What I didn't have to worry about and therefore saved me a ton of time was the hardware stack or hosting. While it's true one must have some level of sysadmin skills to build on Joyent Cloud, I am by no means an old-timey neck-beard and am able to fake my way through most everything with the occasional help from Joyent's incredible support team. I ended up taking a vacation from work, traveled from my home base of New York to LA, crashed on a rock-star friend's couch, and finished the bulk of code within a few weeks of coming back. The site went live early in the morning on May 25th and has had zero downtime since, unless I've broken something.

While developing, I was operating on one single 1/4 GB "staging" server with an onboard database. When it came time to go live, code was pushed to a 1GB SmartMachine, and the database separated to its own 1GB Percona appliance. I'm not using a load balancer yet, but Zeus is ready to roll at a moment's notice. Once things were live and stable, it was time for New Relic for application monitoring, which has been absolutely invaluable in identifying bottlenecks and inefficiencies in the system that are 100% my own fault. Insights into script execution times, poor database design and slow queries were infinitely easier to identify and rectify with New Relic. The setup also took like five minutes. It's a really amazing feeling when you install something and it does exactly what it says it's going to do.

As SHOPULARITY Grew Up...The Smart Way to Scale

As traffic grew and the user base has expanded from my initial seed group of personal contacts to the whole world, it seemed necessary from the New Relic reports on page rendering times that a CDN would be needed to deliver an optimal experience to my end-users. SHOPULARITY has large, high quality images and custom Cufon enabled fonts on every page and when combined with dense AJAX calls taking up lots of lines of Javascript, leveraging Joyent's relationship with CDNetworks was a no-brainer. The move to the CDN was a breeze that took, literally, a few hours and the performance improvement was immediate. Since the application layer and database are on separate tracks, scaling is very easy. Throwing a dedicated load-balancer appliance in front of it all and then having a fully redundant and completely scalable architecture are completely achievable, all without me ever having to spill whiskey on a piece of hardware.

These days, my time is spent working on new features, building more robust CMS tools to deal with the element of fraud that exists in every type of contest application (and I am proud to say the algorithms and tools are pretty sophisticated and seem to be working very well!) and planning for the future. Of course, as SHOPULARITY is still in its nascent stages, I haven't lost sight of the business goals or overall strategy that will make it a major player in the advertising market. I look forward to scaling and the challenges it brings and feel confident that the infrastructure behind the application will support me and my users for years to come.

Jordan Holberg



Sign up now for Instant Cloud Access Get Started