Tux the Linux Penguin

I work with Linux quite a bit at CG and it comes up in some interesting places. Consistently we find that the largest and smallest systems we work with are running Linux. If it’s a large system, like a database or storage network, Linux can provide the stability, robustness, and uptime required to make the project a success. If it’s a very small system, like an embedded display or a custom purpose device, we choose Linux because of its flexibility.

Over the last year or so, we’ve had a need to spin up many more virtual computers than we used to. It’s necessary to create a handful of  computers for each project we work on and the number of projects that we are taking on at a time is growing. Tools like EC2 and VMWare have made it very easy for us to create new virtual computers, but what about managing them? For Linux there are some great solutions. In this blog post I would like to talk about how we’re using some of the tools to deal with the large number of machines we are managing in Amazon EC2 for ourselves and our clients.

Cloud Control

The first part of the system is Amazon’s Elastic Compute Cloud (or EC2 for short). I’ve written about EC2 before, but to be really brief about it: EC2 allows you to create computers on the Internet quickly and lets you pay by the hour for their use. The instances you create are virtual machines that run in a large Xen based infrastructure that Amazon provides. To customize the software and configuration of the instances, Amazon lets you create snapshots of your computers (called Amazon Machine Images or AMIs) that you can launch new instances from.

It’s hard to say that making an AMI is difficult, it’s just a few keystrokes and a coffee break while the image is prepared and stored for your future use. When you begin to manage dozens and dozens of different images this becomes a problem. It became clear to us that we needed to have one AMI to manage, and build every computer from image dynamically. To do this we needed a way to pass specific information to each instance when it started up and then have a tool customize the instance based on the specific information about it. Amazon provides a method to pass specific information to an instance (it’s called user-data) and the good folks at Alestic have made some great AMIs around it with some good documentation.

Enter the Puppet Master

We have been using a tool called Puppet for nearly two years now to manage a handful of computers. We selected that to manage the whole lot of Linux computers that we were dealing with. Puppet allows us to describe how a computer should work in a general way. We can make collections of configurations for standard things so it’s easy to reuse what we create over and over again. Our Puppet configuration is stored in a Git repository so many administrators and developers can collaborate on it and our server configurations are backed up and version controlled automatically.

Bringing it all together

Puppet decides what configuration to use for a computer based on its hostname. We use the EC2 user data to pass a new instance the address of a Puppet server and the hostname that the machine should assume. When it boots up it sets its hostname and checks in to receive the configuration that we’ve stored for it. All changes to the machine happen through Puppet so we don’t have to spend a lot of time SSHing in and customizing the machine. It’s also very easy for us to duplicate a machine for testing or whatever we need.

While we originally did this to make our lives easier as we manage more machines there turned out to be some really cool side effects:

• Excellent Security: We don’t want to store sensitive information in Puppet. This means no passwords or secret stuff. To resolve this we require all developers and administrators to use key-based authentication to get access to the computers via SSH. This is very handy and it eliminates the need to remember passwords or for an administrator to have to reset passwords for users. Someday I would love to take the next step and have all of our users and machines be part of the Monkeysphere.
• Accountability: All of our configuration is tracked in a Git repository so we can see the history about what has changed on certain hosts and who changed it. Change control occurs automatically with Puppet and it’s easy to examine and understand what has changed and why it changed.
• Repeatability: By storing all Linux computer configuration in a single place we can easily repeat what we did for one server on another. Puppet institutionalizes all of our Linux knowledge in one place and saves us time every time we have to create a new machine. If you want to see how someone has done something in the past there’s no more need to dig through emails or documentation, just look at the facts in the Puppet repository and even copy and paste it into your new configuration.
• Portability: We use Puppet to manage much more than just EC2 instances. Physical machines and virtual machines in our ESX installation are managed this way too. It gives us one tool to take care of any Linux machine we deal with. Puppet also supports other operating systems. We’re looking to expand our use of it to Mac OS X machines and maybe even Windows sometime soon.

There are certainly other ways to solve the problems we’re up against, but this is the way we chose. We did some extensive evaluation of Chef (which is a tool like Puppet) and we’ve used Rightscale before. This sort of thing is becoming very important as we manage more and more computers. I expect we’ll see a lot of exciting products, techniques, and services in this space as time goes on.

If you have any questions or comments about our setup, or would like to discuss implementing something like this for your business, leave a comment or get in touch with us. We’d love to help.

Entertainment = Consumption + Interaction:

Internet-enabled TV will bring about some radical changes.  Nielsen’s Three Screen Report indicates that consumption of the moving image is increasing across all three screens: traditional TV, internet, and mobile. The interactivity of this “data” will surely change. Entertainment now equals both watching and interaction: checking out additional content online, playing games, or participating in ads as entertainment. 3D and gesture-based interactions will also redefine this blurring of the line between passive and active viewing over the next few years.

Ditch the “dumb” set-top:

Content models that rely on “captive audience” set-top box capture of viewing habits are outmoded as delivery systems, and deliver poor analytics and reporting compared with the information we can glean from a data-only model. If content owners rely on this information, why does the old model remain unchecked? The set-top box is largely a passive unit that doesn’t include an interface, platform, or APIs for allowing advertisers to interact directly with their target audience in real-time like the Web does. We have seen some recent (and awesome) successes with new methods of delivery: Hulu, Netflix, MLB, and on-demand efforts. However we are in an infancy of thinking about the possibilities of satellite, cable, and their antiquated set-tops, and how broadcast can recapture some of the money that migrated to online advertising.

This is an opportunity for direct access to consumers in their living rooms. The delivery method is there, there is already a large internet-enabled box in many living rooms: tuner, DVR, AppleTV, Slingbox, etc. MSOs already have the access to provide value-add applications to the experience, but what is preventing them from radically changing the intersection of TV, Internet, and advertising? They actually have had a better chance than anyone.

Develop a platform:

Imagine tying content delivery to analytics and advertising – a platform that delivers both choice and guidance to direct viewers to shows they like, and then targets accompanying ads based on more granular information and feedback. Visible World does last-mile ad insertion for parent company, Comcast, who acquired one of the big three: NBC.  A small wrinkle in this still-disputed merger is Sen. Kohl, who is asking for a divestiture of NBC’s holdings in Hulu, arguing that it potentially violates anti-trust.

But this is an amazing opportunity for MSOs to radically change the model for consumers and advertisers by providing a platform for interaction. MSOs have a chance to drive people back to their subscriptions — otherwise we will see a switch to online viewing, where advertisers can get a lot more feedback for their digital dimes, and consumers can have more choice, and augment their experiences.

The TV could supersede the MSO-provided set-top box as a platform:

Internet-enabled TVs or TVs with companion internet-enabled boxes will supersede the current MSO interface and platform. Federated search across Internet/DVR/broadcast is far more useful and less clunky than a remote-driven interface.  In addition, the ability to bring in dynamic web content — ads, additional content, or related information — should quickly reduce the three screens to just two.

When MSOs realize they are missing out on key advertising opportunities, we will see a rush to market with subscription- and licensed-content across the three screens. You would have thought that Hulu would have put the fear in them, but wait until Google TV takes their bite.  Their recent partnerships with TV manufacturer, Sony, ensures that this will be widely distributed to Best Buy, Walmart, etc. beyond the market reach of Roku, Slingboxes, of the world.

Prepare to lose ad revenue:

NBC Universal’s (current) CEO Jeff Zucker and his oft-quoted ‘trading analog dollars for digital pennies’ was revised last year to ‘digital dimes’ from the man himself. That is good news, especially for the multitudes of investors that entered into internet video ventures last year. Out of that gold-rush of investment though, there hasn’t been significant pay-dirt for many. Clearly the answer lies in uniting broadcast to the Internet, bringing the interactivity the web provides, in addition to the type of analytics that are possible. I predict Google TV will make more of an impact than Apple TV did to unite those advertising schemes, although watch out, Apple’s clearly got some plans a brewin‘.

Google TV offers the opportunity to sell ad’s within the interface, and provide targeted advertising, on the TV. Show me what you got MSO’s.

In Conclusion:

It is a rapidly changing world, and people have clearly shown that the TV, cable-tuner, and DVR do not have all the features people want. The cable companies who own the infrastructure and delivery method of video and data are being left out of this equation. Google has side-stepped around cable companies and the licensing issues — wisely I might add — and added a layer with enhancements that will allow them get deep into people’s living rooms, to collect data and provide a smarter ad platform. Things are going to get really interesting….

Everyone seems to be touting HTML5 video as the “open” alternative to the proprietary Flash plugin required for .flv playback in the browser. But how open is H.264, the codec that powers HTML5 video, and the current pick for encoding video for online delivery? Using H.264 as the codec behind HTML5 video sours things a bit for me. H.264 is encumbered by software patents; to develop or distribute a player or encoder for H.264 you might have to pay a licensing fee to MPEG-LA. Even though MPEG LA announced last week (PDF) that H.264 will remain fee-less for free internet video through 2016, this is not the same as being free or open. MPEG-LA can still go after people that produce the software to encode or decode H.264. And MPEG-LA is not just one organization, it’s a collection of patent holders that have their own agendas.

All this is a bit of a slap in the face to the open standards that power the web. Imagine if you had to pay a half million dollars to create or display JPEGs, GIFs, or HTML… The only people that would be able to afford to make software for the web would be huge companies. But what are our alternatives? Beyond Ogg Theora and Matroska, the pickings are slim. These codecs are open and free, but not necessarily better than H.264. Plus it would be next to impossible to compete with the marketing machine of Apple behind H.264.

Open and free standards have been what has made the Internet successful since its inception. I think it’s important that users understand this so that the Internet of the future cannot be controlled by corporations with enough cash to cover licensing fees.

To me, this isn’t so much a debate about Flash/ActionScript 3 versus HTML5, but rather another win for HTML + Flash/AS3! It’s all about creativity as a developer. Bad coding leads to poor applications, proper coding leads to a proper experience. It is a common pitfall that most people think “HTML or Flash”. I see this as more power for the mixing of technologies, raising the cap on what can and can’t be done in a web browser.

Adobe’s technologies provide clear benefits to the end user, but also (and perhaps more importantly) the developer. As a developer, I can utilize the unique capabilities of the .flv format to protect content in some fashion or for metadata injection, all of which can be done on the fly and server-side, if implemented using Flash Media Server (FMS).  I’m also really excited about Flash Player 10.1 – it brings the ability to scale all the way from mobile to HD flavors, and will be available on smartphones and other Internet-connected mobile devices.

This is about more than just video delivery, it is the platform combined with the tools, and Adobe has been making tremendous strides in going open-source with them. Adobe is providing a cohesive environment that is deeply integrated with some of the best tools out there for content creation. HTML5 is just starting out, and the gap between the tools and technology is too immense to make it the competition. Flash has fantastic penetration and Adobe can rapidly evolve the technology. Remember, HTML5 still is not a standard – in fact we are looking at sometime in 2012 before we’ll see a final recommendation. These are cohesive technologies and they are here to stay for a very long time, which means plugins will be around for a long time too. Simply put,  plugin implementations have the potential to penetrate faster, and as history has shown they often do. Plugins forge the path, and the Web comes right behind them to standardize those paths.

If you want to better understand what the big picture is regarding Adobe and its technologies, I recommend reading a little bit about:

OpenScreenProject
Catalyst
SVG and FXG
Flex Data Services
RTMFP (Real Time Media Flow Protocol)
Binary Sockets

So it is with great excitement that we announce the SPIN Mobile iPhone application. It’s available today for free at the iTunes Store. (Update: check out this great video that our friends at SPIN put together!)

SPIN Mobile iPhone Application

For this initial release, we wanted to accomplish something really simple: get the writing, pictures, and music that makes SPIN one of the top music magazines in the world into the palm of its readers’ hands. We wanted to push the fresh daily content, and close the loop on the complete experience: from the music news, to the latest reviews, and photo galleries of shows we wish we had been at. We also wanted to add direct downloads from iTunes, letting users listen to and purchase the albums and songs that they were experiencing on their iPhones.

With things changing at the rate they are, there was a heavy emphasis on getting an app out there quickly, staking a claim in the App Store land rush, and building from that stake down the road with a killer app for the iPhone OS 3.0 launch.

We developed the core app in four weeks with a small group of our dev team members, meeting with Spin’s design team on a weekly basis. Adjusting plans mid-flight, in a truly agile way, we changed designs, integrated with their publishing workflow, their CMS (Drupal) and added a few new tweaks in the process. I won’t say it was all roses — developing an application in four weeks on a new platform, with a full page ad in the presses can put some pressure on the team — but we got it done, and SPIN was right there with us, positive and understanding the whole time.

It’s hard not to get a little sentimental and think about my first Walkman, a bulky yellow Sony thing that was supposed to be waterproof. I used to wait weeks to get the latest copy of SPIN and then run to The Garage in Harvard Square to find the latest tape from the band they were raving about, pop it in and walk around, knowing I was up on the latest thing. Here we are 20 years later with that complete experience condensed down to just few seconds, anytime, anywhere I want. That’s what mobile is all about.

With all the turmoil in the publishing world, its so refreshing to have a client who really ‘gets it’ — understands what its base wants, knows how to monetize without being offensive, and has a plan for the future. I’m glad we could be part of the team that got this out there, and I’m looking froward to all the cool stuff we have cooking for the 3.0 release this summer.