The automotive lineup for CES 2012 next week includes the debut of the Ford Evos concept car, a sensor-rich vehicle that is designed to connect to the cloud, all for the benefit of user experience. What a great approach! Why aren’t all occupiable spaces, including cars, designed like this? Let’s look at the car concept. What do the cloud and sensors bring to driving experience? In short, it could make driving as forward-thinking as an iPhone. The concept car features include connectivity with entertainment and content; energy management based on environment and locality; parental controls; and smartphone integration. Other auto companies are working on similar ”cloud connected” and sensor-rich concept cars, and it’s hard to knock any of them for taking another five years — the estimated time to get real vehicles into showrooms — to perfect the technology. I can’t wait!

That said, during this five-year span, countless buildings, schools, Starbucks, Walmarts, etc. will be built without the forward-thinking technology frameworks for really enhanced user experiences. There are sensors, sure – like temperature management, which can effect productivity by gigantic numbers. But why is it not pervasive and evident, in ways that people can really notice and appreciate? Is it cost? Is it the building process? Is it the profit model? Maybe it’s all of these, but if there is meaningful benefit to user experience, shouldn’t the architect be thinking about technology frameworks – the way they think about new exotic building materials that only they have access to? It should be noted that it isn’t only architects that can leverage these tools and frameworks – retail marketers, advertisers, product managers, and property owners can reap benefits.  But architects have more opportunity to thread infrastructural thinking into the program and design of the space.  (I have spent much of my career working in both architecture and technology, which is why I pick on architects.)

What could an architect do with a rich technology infrastructure like the Evos concept? I’m open to suggestions, but the answer is likely a combination of the internet of things (sensors and other hardware), my digital self (i.e. Facebook, Linkedin, and particularly smartphones, which are broadcasting all the time), a robust web-services community, and great user experience design.

Here are some things that it’s not: The Jetsons, where everything has a robotic arm. It’s also not exactly “responsive architecture,” at least the physical part where buildings deform to meet changing conditions or provide some interactivity. And it’s not a marketing element, like lobby displays, although the best ones, like the Cosmopolitan Hotel in Las Vegas, are beautiful and certainly benefit user experience.

If not robotic arms for better services, then what? We can use tech infrastructure to augment and improve the delivery of services. A great example is the gate redesign work that’s happening at some of the major airports. Why should the airport gate look like a bus station and serve terrible food anymore? OTG Management, an innovative airport food and beverage operator, has revolutionized the gate holding area by making major restaurant improvements (actually good food) and extending that enhanced experience throughout the terminal with food delivery service via iPad-based menus and ordering systems, along with charging stations and comfortable seating. Now travelers are able to relax and dine from any seat.

Also, instead of deforming a building physically, perhaps we can deform it digitally. Shopkick allows a retailer to install a small transmitter that can communicate with local phones. Shoppers can pull out their mobile devices and get rewards and offers based on their digital and physical histories. Invisible to those that don’t want it, but a new experience for hardcore shoppers.

Instead of “screensaver” signage for effect — again, some are beautiful — perhaps we can conform the content to the viewer base. Techstars startup Immersive Labs is developing an outdoor advertising technology that uses cameras (as sensors) and facial recognition to tailor content to viewers, as in Minority Report. Not everyone wants “billboards” at their school or hotel, but this same capability is a fantastic tool for user experience design.

These aren’t the end-all, be-all examples but useful data points. There are many. For instance, Walmart Labs is making lots of social and mobile aquisitions – Grapple and Small Society, and I haven’t seen the new master store…but then again, I don’t get to Walmart much living in Manhattan.

For most users, this sense of super connectivity could be disturbing. As mentioned above, it’s going to be up to the designer, particularly the user experience designer, to turn this potential nightmare into a pleasant dream. User trust is essential – see Toby’s article on 5 Practices for Securing User Confidence for more.

Share this:

Deploying code is a big part of our job and we’re always looking to increase our efficiency when deploying applications. Recently, we decided that our goal is to package every application as a single file archive that can be easily built and deployed. We want our continuous integration system to spit out a single file per project that can be used to deploy the everything. PHP offers a way to store PHP apps into one single file, a PHP Archive or “phar” file, so we began our experiments with phar archive deployment.

To test deployments of PHP apps in a phar archive, we generated a very basic Yii Framework-based web application for testing: a “yii/” directory with the Yii Framework files and a “webapp/” directory with the web application files (e.g. “index.php” and “protected/”). We also protected the “yii/” directory with an “.htaccess” file and deleted some runtime data to save up space in the phar archive we wanted to build.

We modified our configuration to serve phar files with the PHP module and whitelisted phar files in the Suhosin PHP extension configuration. We generated a testing “index.phar” archive and put it in the DocumentRoot along with a bootstrap “index.php” file with the following content:

<?php
include ‘phar://index.phar/webapp/index.php’;
__HALT_COMPILER();

An error occurred when the application loaded in the browser: realpath() was not able to determine the location of the “protected/runtime/” directory in the web application. This function seems to be having issues when used inside phar archives and there was no point in storing runtime or user data inside of it.  So we needed a real directory outside the phar file for that. We then overrode realpath() in the bootstrap file with the “runkit” PHP extension.

In the overridden function, we expunged the “phar://” and the “index.phar/webapp/” path components and returned the results when the Yii Framework was trying to determine its runtime directory. If a path was beginning with “phar://” we simply returned it, and if none of those conditions were met, we simply returned the value returned by the original realpath() we made a copy of in the bootstrap file. To correctly display css files stored in the phar archive, we also used “mod_rewrite” to redirect requests to “/index.phar/webapp/css/”. We created the “protected/runtime/” and “assets/” directories outside the phar archive in the DocumentRoot, and we protected the newly created “webapp/protected/” directory with an “.htaccess” file.

We also noticed that captcha images were not being displayed because a needed “ttf” font that ships with the Yii Framework was not found at runtime: dirname() was not able to return/determine the whereabouts of the directory inside the phar archive where that font was. We overrode dirname() to extract that file at runtime from the “index.phar” archive into a temporary location, if not already there; the overridden dirname() was coded to return this new path, or the value returned by the original dirname() function in all the other cases.

As you can see, there are a lot of overrides required just to make a simple application work. We’ve stopped our work on phar archive deployment because managing all of these overrides is unworkable. We also have no assurances that the overrides will be appropriate for a more complicated application.

We’re going to try some other experiments to get closer to our goal of a single file deployment for our applications. Our next experiments will be around automation the creation of tarballs with custom code to deploy them appropriately.

Is anyone else using phar archives to package their applications? We’d be curious to know if anyone else has had better luck. Any comments and ideas are welcome!

Share this:

Bought something at the Apple Store on West 14th Street yesterday and tried the new Apple Store app for self checkout. Launch the app and it recognizes you’re in a store (GPS? SSID? Geo-fencing?). A special interface appears within the app, you click the EasyPay button and take a pic of the barcode on whatever you’re purchasing. Pay with your Apple ID (same as iTunes account) by typing your password. Your receipt appears on screen so a sales rep can give you a “paid” sticker.

Pretty slick. Apple’s got something incredibly powerful with their Apple ID system tied to customer credit card info, and they’re one of the only big players in the space that has both physical and online stores. Amazon’s got this down (duh) but not so much for offline purchasing. Facebook and Google are trying to figure out how to monetize offline purchases too, but seems like they’re playing catch up here.

Share this:

If you replaced your runbook with a puppet recipe, spun up a dev environment for breakfast, moved your production infrastructure to AWS, and have a few Arduinos on your desk… we want to talk to you.

Who are we? Just some geeks building the next. next thing and having a blast along the way. We work on dozens of projects every year, using the latest tools and inventing them when they don’t exist yet. We’re super busy creating new infrastructures for our clients, supporting our developers, and working on our own R&D. Your networking, database, storage, cloud, and hardware hacking chops will be challenged and honed.  Since DevOps is an emerging discipline, we’re writing the playbook as we move along.  So we’re looking for someone who lives and breathes this stuff– not necessarily the person with the most experience.

If your interested in joining our team, send us your resume or LinkedIn profile. (GitHub account and OSS contributions will also get our attention!)

Share this:

I’ve deployed Windows in every version since 3.0, and I looked at those early Windows through a greyscale monitor on a 286. I’ve used Microsoft Office in every version up to 2010 and hooked an old Neanderthal smartphone up to a hosted Exchange server and tapped my way through my emails with my little stylus.  I’ve installed and configured Windows server back-ends from Windows NT up to 2008r2. I’ve worked in IT rooms that were so full of loud server iron, that it was like being in the engine room of a submarine. I’ve installed or worked on every version of Exchange from version 5.0 up to 2007. I spent thousands of dollars of my own money on Microsoft manuals and certification exams in order to stay up to speed on developments.

And then it all began to change, and for me, it began with Exchange. Exchange 2007 was the last version I deployed, and the last version my own email account was connected to. I’ve been a Google Apps user for two years now, and I’ve also helped transit some of our clients over to Google Apps also.  My personal transition away from a Microsoft-centric working experience started with email, and has continued on through the rest of what I do. I feel like I’m boarding the Google boat and there are just a few bags left on the MS dock.

Google Apps has been offering full-featured email and collaboration services for a while now. An Exchange server can be replaced in a cost-effective fashion, providing shared calendars, spam filtering, message archiving, chat services, access anywhere from a web browser, and almost no mailbox size limits– starting at $50 per mailbox per year. And while you can certainly use Outlook, Apple Mail and iCal to connect to Google, you can use nothing but your web browser if you want and have full functionality. About the only major feature of Exchange not replicated in Google Apps is Public Folders. But then again, Microsoft isn’t providing for that either going forward.

Replacing email servers is one thing, but replacing those desktop applications is another story altogether. Microsoft has held a monopoly on workstation software and business productivity applications for years. The fact is, Windows and Office work well enough for the majority of users out there. They’re easy to use and familiar. A real challenger has to offer a better way of doing something the average user is already doing. And Cloud computing’s Software As A Service  is finally maturing to that point.

Speaking as a heavy Microsoft user, I’ve personally been anchored to Windows and Office, primarily by Visio and Project.  Microsoft Word is still critical for final document drafts and printing, since the offerings on Google Apps just aren’t there yet in terms of refinement and features. But it’s just a matter of time. The recent addition of Smartsheet to our toolkit has now removed our reliance on Microsoft Project except for our very largest initiatives. And we are eagerly awaiting the evolution of Google Drawings to allow us to build the type of schematics we’re creating with Visio.

Microsoft of course has its own Cloud offering, Office 365, and naturally it’s tied into licensing of their existing products. They have a tiered licensing model that’s much more complex than Google’s. But it’s also a much deeper system than Google Apps.  It goes without saying that Google has no legacy revenue streams to drag into the 21st century. Even so, any company can make a migration decision based on what functionality works best for them, but it’s nice to be able to start fresh at a very aggressive price-point.

For further comparison, since Google Apps is platform agnostic, Mac users don’t draw the short straw yet again when it comes to software and collaboration with their Windows brethren. For remote users in a company that’s migrated to Google Apps, all they need is an Internet connection and a web browser, and they will have exactly the same experience as they have sitting in their office. While the individual feature-set of Google Apps isn’t as elegant or robust as Microsoft Office or Office 365, Google Apps is radically simpler in that you get full access to it’s features with just a web browser. This makes it a serious competitor for today’s geographically diverse, small to mid size business.

Regarding remote workers, smartphones deserve a mention. I used a Blackberry for years, but opted for a Droid recently. You probably knew this was coming, but Google Apps on the Droid takes only minutes to configure, and works smoothly. Google Apps functionality includes: Gmail, Calendar and Contact sync, Push support, Google Docs,  Enterprise Admin controls, and 2-way verification for extra security. This feature set is available for almost all platforms, including Windows:  http://www.google.com/apps/intl/en/business/mobile.html

By contrast, Office 365 applications for iPhone and Android are not coming anytime soon. Mobile access is limited unless you’re using Windows Phone, which let’s be honest, almost nobody is.

Handheld prevalence as per end-of-year 2010:

Android #1 – 33.3 milllion

Symbian #2 – 31 million

Apple #3 – 16.2 million

RIM #4 – 14.6 – million

MS #5 – 3.1 million

Source: http://on.mash.to/rG6bfd

As I write this, I have my email, shared documents, and a Project Plan open on my Windows workstation and on my Linux laptop. I use these applications every single day. My documents appear and function identically across these two computers. My email appears and functions identically. The Gannt chart appears and functions identically. On my Droid phone I have read and write access to my calendar, documents, and email. With the exception of Visio and Word, I can be anywhere with an Internet connection, on a borrowed computer running OSX, Linux or Windows and be fully functional. The last of my bags on the dock contain Word and Visio. I can’t leave them behind just yet, but I’m waiting. I want to get going.

Share this:

We love what we do.  So much so that we work on pet projects together outside of the normal course of business.

Share this:

I was in a meeting the other day discussing a problem that a client keeps running into. They need a platform to analyze trends in a rapidly growing data set, where the criteria is changing as fast as their business is changing, which as it turns out, is pretty fast. Right now they are storing the data in a relational database and writing complex SQL queries to mine information from it. The DBA told us that he would run a query and then go to lunch, hoping it would be done by the time he gets back. They need the results faster, and they know that their problem is just going to get worse as the data grows.

The kneejerk reaction to a problem like this is to get a bigger database server. Sure, this may help right now when the data is only a few hundred gigabytes, but what happens when we are dealing with a few hundred terabytes? A few hundred petabytes? This kind of solution just does not scale.

The real answer here is to step back, examine the problem, understand what the goal is, and then design a process that can achieve that goal. In this case, the problem is that a business needs to be able to understand patterns and trends in a rapidly growing data set. The goal is to be able to do this quickly and consistently even as the data grows. One process that can achieve this is by using something like Hadoop or Spark to build a cluster that can scale as the data scales.

There were concerns as soon as I brought this up; What about the schema? How do you write SQL for that? Why not just shard the database? Some of these concerns may be valid, but I feel we must evaluate this without emotion. Do people want to use the relational database because it is a better solution for the problem or because they feel comfortable with it?

I’m not sure it’s accurate to say that we are facing new problems these days, but the shape and size of our problems have changed. Now even the smallest company has something to gain from working with big data– anyone with a credit card can spin up a compute cluster. We should not be afraid to change our tools as our challenges change.

Technology is continuously evolving. This means our tools are continuously changing and so must our processes for tackling new challenges. I believe that the system we came up with in that meeting will be the one to solve our client’s problem. If someone gave us the same problem five years ago or five years from now we would probably have wildly different suggestions, but we would come to those suggestions in the same way: through deep understanding of both the problem and the technology available.

Share this:

I’m really excited about several new iOS development and deployment projects that we’ve been working on at CG. We’re working closely with Apple on a bunch of solutions: at the most basic level, we’re building solutions for security and management of employee iPad and iPhone use; at the other end of the spectrum, we’re helping to realize visions such as a kiosk-like platform of thousands of iPads deployed in retail environments around the country.

We’ve learned a ton about what is and isn’t possible as we strategize ways to scale to thousands of units. Here are some of the challenges we’ve come across:

• How do we deploy and support iPads – whether ten or ten thousand – in a secure, efficient, and centralized way?
• How can we architect kiosk-like application experiences on the iPad, enabling us to design and curate the customer experience, while also allowing a true iPad experience complete with app-switching, web browsing, Facebook-checking, game-playing, and movie-watching?
• What kind of network and server architecture is needed to support a platform of iOS devices across the globe? How do we enable caching and pushing of dynamic data to the devices – particularly large amounts of media content?

Centralized deployment and support of iOS devices

How do we deploy and support thousands of iPads or iPhones in a secure, efficient, and centralized way? Mobile Device Management (MDM) platforms like AirWatch, Casper, MobileIron – and soon, OS X Lion Server – allow us to push XML configuration profiles to iOS devices. This enables centralized inventory and basic management of the devices: from what version of iOS they have installed, to some security control over how/if users can install and delete apps. For many enterprise customers, these tools are useful for administering security policies on employee-owned iOS devices. But for custom platforms like kiosks and retail experiences, MDM is not ideal due to the need for end-user interaction. What we need is a way to easily restore iOS devices back to their “golden” state in a centrally managed way.

We’re excited about the potential of over-the-air restores and software updates coming in iOS 5, but as of today, iTunes is the only game in town for this. Working within this limitation, we’ve architected some innovative solutions that enable iOS devices to connect to iTunes virtually over USB to IP converters and a content distribution infrastructure. Until iOS 5, this is a good option to have, and I haven’t heard of anyone else embracing this approach.

Rearchitecting Apple’s iOS user experience

Put an iPad in front of someone and they’re going to tap, scroll, pinch, and squeeze the user interface. The user experience is still the leader in the tablet space – though we’ve been recently impressed by the BlackBerry PlayBook. For a project we’re working on now, we want to encourage this user experimentation and interaction, while locking down some important components of the UX. Things like App Store purchases, iTunes downloads, deleting apps, rearranging icons, and changing the home screen wallpaper will quickly affect the kiosk experience. MDM solutions can help disable some of these features, but the aforementioned need for user interaction just doesn’t work for specialized user environments.

One solution we’ve had success with is a combination of custom code to disable user customization of the Springboard, plus a WebKit-based Safari replacement for browsing that enables us to prevent user download of unauthorized content. Combine these with some configuration profile-based customization of iOS and we have a good solution for locking a customer experience down and reducing the frequency of unit restores or reimaging.

The CG approach to iOS projects

Part of what makes CG stand out as a solution provider is our deeply embedded collaboration between our application development team and our infrastructure team. As the Enterprise’s appetite for customized mobile platforms and experiences grows, we’re uniquely suited as a technology partner to build and innovate on our customers’ vision. iOS is at the core of this vision and I couldn’t be more excited to be working with these technologies today. Plus, iOS 5 is on its way and it’s shaping up to be a giant leap forward!

Share this:

We’ve done a lot of work recently to revamp the way we deploy computers in the cloud and I wanted to share a little bit about how we’re doing this at a pretty low level to give you an idea of how we are approaching this. Our software and processes are cloud agnostic, but we mostly work with Amazon Web Services because we feel that they offer the best solution for most of our clients at this time.

We maintain two base Linux images as part of our cloud toolkit. The only difference between the two images is their architecture. One is 64bit and the other is 32bit. The images are minimal– they have just enough software and configuration to get them off the ground and configured. We have copies of the images in each region in Amazon, but when it comes to maintenance and upgrades we really only deal with the two master images. All of the computers that we deploy in EC2 come from these two images.

The base image by itself is not very useful. When a computer is instantiated from one of the images, our toolkit combines it with our Puppet repository and some instance specific configuration. The Puppet repository contains the Puppet manifests for how we deploy software. The repository is where we store our collective knowledge around deploying successful software. The instance specific configuration is crafted by the developers and operations teams to pick and choose the appropriate things from the Puppet repository provide the very specific configuration about how to deploy the server and the application that will run on it. As the instance boots, it configures itself, installing the software and making the changes required to bring it into service.

This is all pretty low level, but it provides some capabilities that makes our solution very flexible:

• With only two images to maintain, keeping software up to date is simple. We anticipate that we will be releasing new images about once a quarter to capture any updates to the packages in the base system.
• Everything is version controlled. It is easy for us to see what a machine looked like on a specific date or understand the changes that have been made to how the software is configured on an instance.
• The instances are very self sufficient. There is no single point of failure that would prevent instances from starting correctly.
• This is all very portable. With just a little bit of work we can deploy things in a different region of Amazon. Also, our Puppet code and instance specific configurations can work in more places than just Amazon. With a little bit of work to recreate the base images in another platform we can consistently and predictable recreate infrastructure anywhere, giving our clients the ability to choose the right solution for them.

This last item is something that should be on everyone’s mind (especially considering the outage at Amazon last week). As Steve said last week, everything fails and you need to design your infrastructure and applications around that. A process for redeploying your infrastructure in another AWS region or a different cloud is an important part of building a very reliable service in the cloud. It is hard to say what the next kind of failure in the cloud will be like, but with a process like ours we can be ready to deal with outage when it happens.

Share this:

Control Group designs cloud-based solutions with the philosophy that every system fails at some point. Embrace this chaos and build for the rainy day. Today we are seeing some major outages on Amazon’s us-east-1 region. Reddit and Quora are two of the high profile victims, but this is affecting everyone in a very popular data center.

You can design around regional performance degradation though. Years ago, having global traffic management in place was an expensive pipe dream. Today you can easily turn up another EC2 region and use a service like Dynect or Akamai GTM to provide failover and/or load balancing. Even better, consider making your systems portable so you can have multiple cloud providers and maintain your machines and applications with Puppet.

3-5 years ago this would have taken a year of planning, purchasing and hands-on labor to implement two data centers. Earlier this year we were able to create two data centers with complex infrastructure on EC2 and active/active load balancing in under two months and for a fraction of the cost.

Share this: