This is a project to pull together many diverse areas of my expertise to demonstrate my knowledge. It is also to learn some new skills, specifically cfengine.
Objectives:
- Use VMware to model a complex set of data centers hosting a site
- Set up Linux based firewall routers to protect a data center
- Set up Linux based management servers to provide DNS, LDAP, MySQL services for the data center
- Set up redundant Linux based web servers in each data center
- Use cfengine to centrally manage all of the Linux servers
- Use Nagios/Cacti to monitor all of the Linux servers
- Document the project
Overview
This project implements three virtual data centers using VMware Server. The three data centers are replicas of each other. Each have two redundant networks for fail over with each server being dual homed. Each has a management node and two redundant web servers. They host a simple loudspeaker inventory application I wrote for a class using PHP and MySQL. The project starts with a single data center to get the basic overall configuration correct and is replicated to the other two data centers.
Click on image for larger image or download PDF.
The hardware for this project is modest 2U server I built:
- AMD&tm; Athlon(tm) 64 X2 Dual Core Processor 5000+
- ??? Mother board
- 1 Gb DDR2-??? memory
- Two SATA disk drives
It is hosted at my colo facility, Layer 42, in Santa Clara which is a half hour drive away. This means that all of my configuration, administration, and trouble shooting will be done remotely over the network.
Documentation
These web pages are written using raw HTML. My HTML editor is vi. No fancy GUI which tend to spew out ugly HTML.
This section talks a bit about how I maintain my web pages and my skills writing html and cascading style sheets
Installing The VMware server
This section describes my VMware server 2.0 configuration
Installing The Virtual Hosts
The way I install vmware hosts is different. When I create a new vmhost I do not create a new virtual host via the GUI and then mount an ISO image of the install media or use a Kick Start server to install the OS. Instead I clone the vmhosts. I create the first vmhost the normal way and install the OS from an ISO image of the install CD. I then create a tar image of this vmhost, LinuxCentOS5.2.tar.gz, that I use to clone new vmhosts. I have a script that untars the generic VM host, renames the vmhost directory and files and munges the host.vmx file. Once the vmhost is booted, I then have a second script in the vmhost itself that configures the hostname, network interfaces and default gateway. I reboot the vmhost and now it is configured with the correct hostname and IP addresses. Once this is done, all configuration is done over the network with ssh.
In this section I also have a discussion about installing servers with a minimal OS and adding additional software as needed. I find that this creates servers that are more secure and require far fewer software patches.
Routing In The Virtual Network Configuration
So how do you handle routing in a collection of virtual networks? You could run a routing protocol like RIP's routed or the ICMP router discover protocol's rdisc daemon. But most data centers in a colo facility have fairly simple network topologies. Typically multiple Gb Ethernet LANs hanging of a single router or a pair of routers for redundancy. In this environment a routing protocol is overkill. A combination of default and static routes can be quite effecitive.
In this section I describe how I use variable length sub-net masks and static routes to implement routing from the VMware server to the data centers and between the peer data centers.
Using cfengine to centralize administration
Learning cfengine was one of the two main impetus for starting this project. The other impetus was to show case the range of my skills. This section is the starting point of my cfengine journey/odyssey. It describes the initial setup of cfengine on the server and the client.
Setting up a local yum repository
Because my VMware server is also my web/mail/dns/ftp/etc... server I have decided that I would configure VMware so that the VMware software network have no direct connectivity to the Internet. This allows me to keep IP forwarding off on the server. But a drawback is that the vmhosts can not reach the CentOS repositories and update servers. To get around this I have setup private yum os and updates repositories.
In this section I also discuss using a private yum repository as a controlled way to introduce OS patches into a production environment after the patches have been tested by development and QA.
Deploying additional base OS software packages with cfengine
This section describes how to use cfengine to install additional software packages in the vmhost servers. It was my first real work with cfengine.
Setting Up The Linux Routers With Cfengine
In this section I set the dual homed Linux hosts as basic routers by enabling IP forwarding and opening up the iptables firewall configuration. To make the routers managable the configurations will be pushed from the master to the servers using cfengine.