While more than 20,000 VMworld 2013 attendees packed the conference halls in San Francisco to hear about enticing new technologies like virtual networking with NSX and storage provisioning in VMware Virtual SAN, a homebrewed concoction quietly percolated behind the scenes to deliver more than 10,000 online lab sessions during the event.
VMware Inc.'s Next-generation Education Environment, or Project NEE, is a cloud service the company began developing in April 2012 to provide an online learning environment such as the Hands-on Labs. Project NEE users can access the virtual machines and the accompanying lesson materials -- in the form of notes, video and audio -- with an Internet connection and a Web browser that fully supports HTML 5. The project uses VMware's vCloud Suite to provide the cloud that runs the virtual labs, so there's no need to install client software.
SearchVMware.com spoke with Curtis Pope, Mike DiPetrillo and Sean Borman of the Project NEE team to hear more about the work behind this learning platform and its technical underpinnings.
What is Project NEE?
The making of
The origins and developing the technical platform
Advanced features streamline the learning process
Project NEE team: Project NEE is a multi-tenant, cloud-based service that provides a rich online learning environment that integrates instructional materials and virtual labs that users can access from anywhere, using almost any device. Project NEE is extensively used by VMware to support online labs at major events like VMworld as well as for online product training for VMware's customers, partners and employees.
Project NEE lets you set up a virtual lab for each student based on a template that you define. These labs can be as simple as a single virtual machine, or as complex as a multi-tier application comprising many VMs. At the VMworld 2013 conferences, attendees could choose to take one of more than 30 labs, each of which provided the student with a complete, self-contained, enterprise IT environment. We had over 450 people taking these labs simultaneously.
What was the motivation to start Project NEE?
Project NEE team: Project NEE grew out of VMware's need for a robust infrastructure for delivering online labs for major events like VMworld. The NEE project leaders, Curtis Pope and Mike DiPetrillo, had been deeply involved with VMworld labs for many years and they saw firsthand the limitations of the solutions we'd been using.
VMworld had grown enormously and the systems we'd used successfully in the earlier years just couldn't handle the growing demand for VMworld labs.
Curtis and Mike also recognized that demand for a powerful online lab environment extended beyond VMworld. There were a number of groups within VMware that needed a tool that would allow them to make complete lab environments available to customers and partners online. Curtis and Mike evaluated a number of commercial offerings and couldn't find any that could address VMware's needs in terms of scale and features, so they resolved to build Project NEE from the ground up. … Their ultimate goal was to see NEE grow up to become a powerful general-purpose platform that anyone -- VMware or its customers or partners -- could use to provide online labs and training.
Is Project NEE built on a previous effort or something entirely new?
Project NEE team: The core of Project NEE is almost entirely new technology. This was necessary since our requirements -- especially those regarding scale -- were so challenging. The NEE architecture incorporates the hard-won knowledge that Curtis and Mike gained from their years of experience with earlier VMworld lab technologies.
NEE also relies on many pre-existing components. Probably the most obvious one being that NEE uses VMware's vCloud Suite to provide the cloud on which the lab VMs run.
What is the underlying technology for Project NEE?
Project NEE team: Project NEE is a multi-tier and multi-application solution. One of the guiding principles when the NEE team first started out was to leverage the best-of-breed VMware technology where appropriate, but the team also used open source components.
The vast majority of the core NEE platform is written in Groovy using the Grails framework. Project NEE uses RabbitMQ for messaging between components and uses two different databases (PostgreSQL or MongoDB) depending on the needs of the application components.
The application was architected and written so that it would be easy to scale out (or in) any component at any time to accommodate increased or decreased load. For example, if delivering the lab manuals comes under heavy load, we can simply add more instances of our content engine application to handle the requests, and the overall solution will adapt accordingly. We can do this on the fly without users noticing. This scale-up and -down architecture also helps us with local and remote disaster recovery scenarios.
NEE is designed so that it can deploy lab workloads to any VMware powered cloud -- that could be VMware's Cloud Hybrid Service (vCHS), a VMware Service Provider [Program] (VSPP) cloud or your own private VMware cloud. To make this work, the team wrote a cloud engine component. The theory is we can run a collection of cloud engines in a SaaS-based model or deploy a cloud engine to a remote site for a private cloud attachment. All communication to the cloud is over SSL, so not only is it encrypted, it also means that all that NEE requires to talk to a private cloud is to have the cloud listening on port 443 at the destination. Using this approach, VMware can host a single instance of NEE as a SaaS offering, yet still be able to work seamlessly with secure private clouds.
It's important to point out that none of the remote display traffic between the student taking a lab and the cloud where the lab VMs actually run goes through the core NEE service. NEE acts as the connection broker only; all display traffic goes point to point between the client and the cloud.