This is section from my web pages Musings/Experiments With A Virtual Data Center

When I first started installing the network I did not think much about network numbering. I did what I had always done with my demo networks and numbered them 10, 20, 30, 11, 21, 31, etc... But then I started thinking about routing and realized that since I was having Linux firewall routers protecting the various servers in the separate data centers, I would have to do something about routing. I could use a simple routing protocol like RIP, or I could use a more complex routing protocol. But then I thought about routing protocols in a data center.

Why run routing protocols internally to a data center?

Do you need to run a routing protocol internally to a data center? In my experience, generally no. When you look inside a data center most of the time the network configuration is a nearly flat network. You typically have a router connected to the Internet and then to a set of internal data center networks.

In its simplest form there are Three cases of routing:

• Local routing to another host on a network you are connected to
• Routing through the router to an internal network
• Routing from the data center to the Internet

All of these cases can be handled without any routing protocols.

For an internal server all it needs is a default route to the local gateway router. Just set the GATEWAY parameter in the /etc/sysconfig/network/ifcfg-eth# files to point to the local gateway. I will have to experiment with redundant dual homed hosts. I think this will set two default routes. News at 11.

News at 11: Two GATEWAY parameters in two ifcfg-eth# files do not work. See Routing in the data center for my work around.

For the data center router it will also need a default route. But this time the route will point to the bandwidth provider's router. The data center router will pickup direct routes to all of the internal networks it is connected to automatically. The default route will be used for all external Internet traffic.

For more complex network configurations such as a load balancer or content switch that has to sit between a DMZ network and the internal network or bonded networks then things are a bit more complicated. But static routes would probably still work. I may experiment later.

Routing to the virtual network cloud

Since the VMware server is at my colo facility and I am using host only virtual networks with private 172.16 network numbers, I will need to route to this network from my desktop/laptop. I have allocated a /19 net-block (netmask 255.255.224.0) starting at 172.16.32.0. I have a backbone network, 172.16.32.0/24 which all of the data center routers will connect to. I have sub-divided the 172.16.32.0/19 net block into /21 netblocks (netmask 255.255.248.0) one for each data center. So my network assignments look like this:

Network Number:	Use:
172.16.32.0/21	Backbone
172.16.40.0/21	Palo Alto
172.16.48.0/21	San Francisco
172.16.54.0/21	New York

I have a VPN connection from my desktop/laptop to the VMware server and will use a static route to the VMware for the 172.16.32.0/19 net block over the VPN tunnel. The VMware server will then have 3 static routes to the three /21 netblocks at each of the three data centers. This will forward traffic for any of the hosts at a data center to the data center’s router.

The routes are added to /etc/rc.d/rc.local with:

route add -net 172.16.40.0 netmask 255.255.248.0 172.16.32.40
route add -net 172.16.48.0 netmask 255.255.248.0 172.16.32.48
route add -net 172.16.56.0 netmask 255.255.248.0 172.16.32.56

Routing in the data center

Each data center will have a router connecting to two local sub-nets. Each server in the data center will be dual homed to both sub-nets.

The data center router will have a default route pointing to VMware server’s network interface on the virtual backbone network. The data center router will pick up direct routes to the two sub-nets it is connected to automatically.

To reach the two other data centers, each data center router will have two static routes to the /21 netblocks at each of the other two data centers.

The routes are added to /etc/rc.d/rc.local with:

route add -net 172.16.X.0 netmask 255.255.248.0 172.16.32.X
route add -net 172.16.Y.0 netmask 255.255.248.0 172.16.32.Y

Where X and Y are:

	X	Y
Palo Alto	48	56
San Francisco	40	56
New York	40	48

Routing in the data center

The servers in the data center will use a pair of default routes to the data center router for redundancy.

Since the default Linux start up scripts do not allow two default routes, I will add them in the /etc/rc.d/rc.local with:

route add default gw 172.16.net1.1
route add default gw 172.16.net2.1

Where net1 and net2 are the two subnets in the data center:

	net1	net2
Palo Alto	41	42
San Francisco	49	50
New York	57	58