Node.js on the Road: David Corona

Node.js on the Road is an event series aimed at sharing Node.js production user stories with the broader community. Watch for key learnings, benefits, and patterns around deploying Node.js.

David Corona, Software Development Engineer
NBC News Digital

All right, how is everyone doing?

Good. Good? All right, my name is David Corona. I work at NBC News Digital. I'm on the team that works on Have you ever heard of The Today Show or if you watch it? Probably not. We don't target developers. But it's a really cool team; I really enjoy working there; we have a lot of versatility at NBC News.

We used to be a part of Microsoft, but we've broken off about two years ago, and now we're just NBC, and we've got a lot more allowance as far as using new technologies, exploring new things besides just working using Microsoft only tools. So it's been fun working there the past year. I've been there for a little over a year, so it's been fun. A little bit about me, I already covered the first line. Been programming since I was really young, I always enjoyed it. I'm mostly a .Net Developer. I also love JavaScript and Node.js. I've been using Node for the past couple of years, and those are things that I enjoy.

So NBC and Node.js, at NBC we're a little new to using Node.js. We've just kind of gotten into the mindset of exploring new things, and Node's been one of the things that we've started moving towards, experimenting, trying it out and, so far it's worked really well. So far NBC Plus Node.js has been pretty sweet, and that's the name of my talk.

So we're using it in lots of ways. One of the ways that we are using it is in, like, external services. It's really good at, and it's really flexible at doing lots of different things. One of the cool things about Node is that, it's so easy to just get up and running. You can make an app, you can deploy it, you can have it running on whatever cloud service or internal server you want to, and it's been really good in that way, and we've got several services running right now on Node.js.

I'm going to kind of go through basically the ways that we're using Node, and the kind of things we're doing with it. One app we have running right now is called Asset Watcher, and basically what this does is, it watches a bunch of folders, like from—we have a big video service for and for

We have lots of videos from the show, from different journalists, and people in the field. And we have a lot of videos coming in from all kinds of sources whether they are internal sources, they are from the Today Show, TV show, they're from people on the field. And so we have a lot of places where videos are being dropped to, and it's been a challenge to take all the videos and make them accessible to the people who are actually writing content for the website and embedding videos onto stories.

So one thing that Asset Watcher does, is it just basically watches lots of locations that videos can drop to, and it then sends messages and updates to our internal API to make those videos accessible to lots of places: our mobile apps, our editors who are writing content, and just lots of things.

We have one called Mavis, it's not the typing tutor, but it's another service that basically we have a video player platform and this thing listens to the video player. It's kind of a CMS tool for videos, and it listens to the CMS and any updates it gets, it just processes those and sends them to our APIs that can send out that data to all of our different apps and platforms that use it.

So it works really good. Another one is called Bottle Rocket Feed, and this one is used by all of our mobile apps. We have lots of different mobile apps. We got some for today, we got some for NBC News, on all the different platforms, and those all need certain formats of data, and every one is different, so another challenge was, how do we get all these different mobile apps that need these different formats to actually consume our videos.

So Bottle Rocket Feed basically listens to our CMS and it takes all that data, kind of caches it in memory, there's no actual database for it, and it provides an end point that all this places can kind of hit and get this data from externally. Another really cool service.

And the last one is called Autopilot.
At NBC we have a pretty big API system, And we have all these different tools. We have mobile apps, we have the websites, we have a bunch of people in the field, and they all need to access all of our data. So we have one big central API that everything flows through and it has to work. Autopilot basically, it's kind of like a continuous testing tool. It just constantly hits the API, constantly testing it, running a big suite of unit tests, and any time something breaks, it sends an email.

So we always have something constantly watching the API, making sure it's always working, and it doesn't go down, and that's pretty important whenever you have something as big as our API that has to power so many things. So, those are ways that we're using it externally. These apps sit on, they're all hosted on Amazon, they're not on our internal servers, but those things that we're running, one-off applications that just kind of run and do their own thing, and no one actually has to touch them, or look at them, they just kind of run, they just work. So it's really good in that way.

So another thing we are using it for is This is our new website. I don't know if anyone has actually seen it, but this launched a couple of months ago. Brand new website, brand new platform, pretty much everything new. So this itself is not actually running on Node.js, but the way we're using Node.js for this website is that we're using it for our actual dev environment.

And it's been really, really cool. Since we've had these big platforms, these big stacks using all Microsoft products, it's been a real struggle working on these projects. Because we've got so many things we have to tie in to, and our dev environments are pretty massive. Like whenever I first started, it literally took me a week and a half to get my environment up and running, so I could actually write code.

Yeah, it was pretty insane. We have got multiple VMs, we have got multiple databases, and it's crazy. So,, this is a really cool new website. The entire thing is one big single page app. So it's all client side, I think it's running in Ember.js, but as far as the dev environment, it's been really awesome using Node.js, because
to onboard a new Dev and to start working on it, you basically download the repo, you run, you run grunt, and you're up and running in 10 minutes, not even 10 minutes. So it's been really, really cool doing that. So dev environment for all of our front-end devs, since they're really only working on the single page app, it's the perfect use case. What the heck? Oh, that's my machine. Never mind. It uses cool tools like grunt. They're using stuff like bower, they're using live-reload which is really awesome, whenever you're working on front-end stuff, you make a change in Node, you save it, and then your site is already reloaded and up and running, it's awesome.

Another cool thing that we do with Node is that we can actually mimic our production environment only using Node.js, so this new website, it's all in .Net. There's a baseline rendering in .Net, it renders a page, and from that point the single page app takes over from there, and it's off and running, but we have to actually render a baseline. So with Node.js we can easily just, in a dev environment, you make a call to the production server, get the HTML from the baseline rendering, cache it, and then you have your rendering mimicked.

So you're basically running as if you're in production, but you're in a dev environment, and it's really awesome. So it makes devs happy. In the beginning when I was setting my environment, that's pretty much what I wanted to do, I wanted to flip over my table and just "screw this, I'm out of here, I don't want to spend a week setting up my environment." I actually punched the guy in the face actually, and now it's really easy, everyone's happy, so far it's been awesome.

So, this is what I do. This is kind of the focus of what I wanted to present. Last Friday, we just launched our new homepage on mobile using Node.js, which was really awesome. It's our actual first production app that's running on Node, that people can actually see, and it's not just something running in the cloud that no one really cares about, but that's been fun. If you go to right now, you can see it on your phone.

OK, if you don't want to go, I have a thing here that you can look at, this is how it looks, it's pretty nice. I didn't design it, but I worked on the code. So this is actually running on Node.js, it's not much to look at, it's pretty simple, it's only one page, but the awesome thing with this is that everything is in place, we have all the foundation work there, and it's a really cool stepping stone for us moving forward and doing all kinds of cool things with Node.js at NBC. This one, it doesn't rely on a database, it doesn't have to access a database whenever it's running on the server, it doesn't have to have a bunch of permissions, it strictly uses our API. So whenever it renders, it gets all the data from the API and it just renders it, and it just works, it's really cool.

So baby steps. We only launched our new cover on mobile. We really wanted to kind of take baby steps, we released one piece then if it works well, we do the next piece, we do the next piece, and then next thing you know our entire site is running on Node.js and that's kind of our plan right now. Going forward, we want to just have the entire website running all on Node.js, and we think that will be a really, really cool thing once we get there, but we got to go piece by piece, we don't want to just have one giant release and we switch over everything at one time, we want to go piece by piece, and kind of feel things out, and that's the one that I worked on.

So I wanted to kind of give you guys an overview of how actually runs on Node.js. We're not the typical start-up or website running on Node.js where everything is new, and we're running on Linux servers, and we're using Macs to develop it. Since we're formerly Microsoft, everything we have is Microsoft stack. We got Windows servers, we got IIS, all kinds of things that maybe you all don't even want to know about. It's crazy. How many here actually develop Node on windows?

Anybody? OK, not too bad. So, we're using IIS, which is your standard hosting environment for Windows server, and we're using a really cool module called iisnode, if you've ever heard of it, it's really cool. Windows Azure uses it. If you're hosting Node on Azure, you're on IIS, and you're on iisnode, really cool module, it works really well. I haven't had any issues yet, and right now, it plays really well with other .Net applications. And what I mean by playing well, is that you have it running in IIS, we have it alongside all kinds of other .Net applications we've had for years, and it just runs. There's no conflicts, no issues. Well some issues, but…

So there's lots
of things involved whenever you're dealing with a big production website. You've got CDNs, we got all kind of levels of caching, in IIS you've got

web configs, we've got rewrite rules, all kinds of stuff. So it was really tricky getting Node.js to work with all these legacy Windows Microsoft systems. So whenever we actually depolyed it, it took us four tries to launch our new mobile cover. Pretty much everything just went wrong. It was crazy.

Let's see what else we've got. But, we got it working, third times a charm, but it actually took four times. We thought it was going to be three times, I was feeling pretty good, I thought we were good to go, I thought we had all the issues ironed out, and then what do you know, another load balancer issue comes up, and we have to roll back, so that was all kinds of crazy things. It was pretty fun.

Whenever you're deploying something like Node.js in a Windows server environment, there's, as far as I experienced, there's lots of things you can't quite foresee, especially whenever you have lots of different layers, lots of different components, all kind of working together and talking to each other, so that was pretty fun, getting it working, and that was how I felt whenever it actually worked and it actually launched, and I actually did do that in our operations area.

So why did we choose Node.js? This is kind of fun. At NBC, since we left Microsoft, we—at first, we kind of still had the mindset that we use .Net, we still need to use .Net, we're always going to use .Net, and that's just the way it is. I remember whenever we first had a meeting, and we were going to actually start working on a brand new platform for, we were sitting down, and there was me and a few other developers on my team, and we had figured out everything we were going to do using our—creating a new solution in visual studio, calling it this, having this running and having that running, and all in .Net. And then in the meeting, I said, hey, guys wait a minute.

Why are we using .Net? Is there a reason why we're using .Net? And we kind of all sat for a second then we said, you know what? We don't have to use .Net. What else can we use? Why are we using the same technology that we always use? And the cool thing about a brand new green project is that you can explore new things.

You can try out new technologies, you can experiment, you can kind of play around, and you don't have to stick with what you've always done, and in this case, we've always done .net, but we want to see if we can do something else. And I said Node.js, we all thought it was cool, and we decided, OK, let's just try this out.

So I had to build a prototype, and I actually had to prove to our architects and to the operations guys that Node.js will work in a Windows server alongside our other legacy apps, and it'll run, and it'll work well. So if you're going to do that, just make sure it's going to work first. It was really fun, I had to do all the research, I had to do all the homework, I had to kind of build out a little app, and I got it working.

I was able to actually prove that using tools like iisnode, you can actually run this stuff in your existing systems. And the proof is in the pudding. I built something pretty simple that would let us load test and performance test this new app, running in production alongside our current applications, and make sure that this new app will actually perform efficiently. It won't have memory issues, it won't have CPU issues, and it will work.

So we had this app running in production for a few weeks, and basically what we did was anytime a user would hit or any other site on Today, we would send an HTTP call to this node app running in production and they would hit it with pretty much the exact same load that we're getting right now on all our current sites. So it was cool to see this app that you're running. Is getting the same traffic level, and it's each running awesome. It's using less memory, has a smaller footprint, and that was really, really cool to see.

Is there anybody here that's using Node.js at work right now? Lots of people, and anyone who wants to use it at work, but is not? Not as many, but a good number. It's always really cool to have a chance to actually use something you want to use like Node.js, in actual work, and you've just got to be able to find the right opportunity to use it, and just make yourself use it, try and convince your bosses, even build it on your own, get it working, get it running, and then show them, hey, look this works, it's already running, it's working, and why don't we use this? And that's basically what we did, we had to get it working, we had to get it running, and we had to prove it actually worked.

I think I'm going over my time a little bit, but I'm almost done. The future's so bright. So, going forward, we've got lots of cool plans with Node.js. One of them is our deployment. Right now we have the worst deployment story you probably ever heard, and you probably don't want to hear it, but basically, since we're on .Net, we're using TFS if anyone's ever heard of it, anyone's ever heard of TFS, you know what TFS is? OK, it's basically Microsoft source control plus whatever else you can do with their project management tests, but I'm basically running right now, I'm basically running a grunt build on my machine which compiles my JS and my CSS and anything else, and I'm just copying and pasting that into TFS, and then we're deploying that using our big deploy process which takes two hours. So we have a really, really cool new idea we're doing for deployments, right now for, they have a really cool deploy story since it's a single page application, they're only really deploying JS code and CSS front-end code. So they have a cool system where we have an operations panel, it's a panel of internal website, they just go there whenever a new build is ready to go, they push the build, they go to this new ops panel and they click a button, and that deploys the new code to all of our servers.

So for our new app on, we want to kind of some similar where we push our code to master and GitHub, and right now we're using codeship, if anyoen' ever heard of it? It's a pretty nice CI tool, online service, and anytime we push to master, codeship pulls all that code to their server, runs our unit tests, runs our acceptance tests, if everything passes, it deploys that to Heroku and we actually test our stuff on Heroku, make sure it looks good before we deploy to our production servers. And with our new deploy story, we want to have codeship push a new build, maybe a zip file to a new S3 storage server on Amazon, and then we go to our ops panel, we click a button, it pulls that zip file down and it pushes it to all of our servers that are running in production, and then the app is running. It takes minutes, so hopefully we'll get that running and it will be really awesome. I'm looking forward to that.

And we want to move basically the rest of our website for Today, over to Node. We only launched mobile, but pretty soon we're going to have the entire site running on Node, and it's going to be really awesome., internally there's some pretty serious talks right now about moving even the rendering from .Net to Node. It's just talks, but you never know, it could be kind of cool, and that is it.

That's the end.

Sign up now for Instant Cloud Access Get Started