Systems administration inexperience a reality at Open Source Lab

A lab architect discusses the rewards and challenges of employing inexperienced systems administrators at a data center that hosts major open source projects.

The Oregon State University Open Source Lab's data center hosts some of the Linux community's heaviest hitting projects including the Linux Master Kernel and the Linux Foundation. It is also the primary location for the Apache Software Foundation and Drupal, open source content management software. The lab, aka OSUOSL, also hosted the core infrastructure for Mozilla's Firefox project, and currently host's six of Google's servers.

Despite housing such high-profile open source projects, the lab's systems administrators are actually OSU computer science students. In the following Q&A, Lance Albertson, an OSUOSL systems administrator and architect, talks about his job -- managing a data center with a staff of 18- to-21-year-olds. SearchDataCenter.com recently visited the OSUOSL data center on OSU's Corvallis, Oregon, campus and talked with Albertson about the rewards and challenges of his job.

OSUOSL's data center hosts and mirrors some of the most important open source community projects in the world. Undergraduate students run the systems administration -- and it works. Why do you let undergrads manage your data center?
Lance Albertson: I wasn't around for the original decision, but I can certainly tell you what I think the reason was. Foremost, hiring students is really low cost and as a nonprofit we need to ensure that we keep our costs to a minimum. However another huge factor is giving students "real world" responsibility and experience. Most students leave the OSL with employers chomping at the bits to hire them because of the experience they get here

Do you encounter challenges in having young, inexperienced systems administrators at the lab?
Albertson: Yes definitely. Most of our students have set up basic server(s) at home to play with, so they have some understanding of how things work. We try to mentor new students with other students and train them on new tasks.

We also encourage documentation of everything in our internal wiki. It generally takes around six months for a student to feel comfortable with our environment.

Additionally, the use of configuration management with git (or source control) helps them learn the process in action.

Another challenge is the short turnaround with students, as we usually only have them for two to three years before they graduate. This creates a constant issue to ensure our documentation and training is honed.

You typically have about five student admins. What skills do they typically learn?
Albertson: From a technical side, they're learning how to administrate a large installation of servers and all the aspects that entails (server racking, OS installation, app setup, security, updates, etc.). They also gain excellent troubleshooting skills, as we are always dealing with interesting problems.

From a nontechnical side they are learning how to interact with customers and deal with issues through a ticket system. We also try and keep a professional yet fun environment, so we try to keep them to that.

OSUOSL uses Cfengine for systems management but hopes to migrate onto Puppet. What are the primary advantages of Puppet?
Albertson: For one, Puppet has a real community behind it instead of a single person who gets to decide what to include for patches. Additionally, Puppet's primary goal is managing a large installation of servers instead of being a research project at a university (which Cfengine is). Also the primary contributors of Puppet just moved to Portland and have offered help to us if we switch, so that is an advantage.

From a technical point of view, Puppet offers more flexibility and an ability to actually use real code to deal with tasks. Cfengine has its own syntax language, but it's not really suited for complex tasks. Puppet also offers some inheritance in its rules, so we could write a module that just does Apache configuration and have a host inherit that code.
Then we can offer our module to the world without worrying about revealing security issues.

Will it be difficult to migrate to Puppet, and why?
Albertson: Most of the issue is time commitment for me and our staff. We are constantly dealing with upgrades, new projects and regular tickets, so it is hard to find time to work on this full time. It will take us some time to look over Puppet and write the modules and code to convert our infrastructure over to it.

Is one systems management tool more suitable than the other for OSL students applying for their first job?

Albertson: If a student gets experience in Cfengine or Puppet, they're better off than anyone else in their field. The concepts between the two are similar -- just executed in a different manner. I know of one student that graduated last year who got a job at Intel in Hillsboro and is on their worldwide Cfengine team.

Matt Stansberry is the Executive Editor of SearchDataCenter.com. Let us know what you think about this story; email Matt Stansberry.

Dig deeper on Data Center Jobs and Staffing and Professional Development

Pro+

Features

Enjoy the benefits of Pro+ membership, learn more and join.

0 comments

Oldest 

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

-ADS BY GOOGLE

SearchWindowsServer

SearchEnterpriseLinux

SearchServerVirtualization

SearchCloudComputing

Close