If you are responsible for the IT operations of a successful organization there will come a day when the closet in the back of your corporate office will no longer suffice as your data center. You will be tasked to evaluate and select a data center to serve as a business continuity or disaster recovery site for mission critical services that power your business. The intent of this article is to provide IT professionals with the rudimentary knowledge necessary to make a well informed decision about where and how their environments need to be hosted.
The first order of business is to determine the budget you will have for your data center expenses. This is a multifaceted objective that needs to be reviewed carefully to determine how much redundancy you can afford to build into your solution. This will not cover the costs of server or networking equipment as that is outside the scope of this article. Besides the expenses outside of our scope (server hardware, networking hardware, cables, mounting hardware, labor) you will encounter two types of data center expenses. A one time expense (also called setup fee) and recurring expenses. The more redundant the environment the more expensive it will be. Your setup fee will cover the labor and parts required by electricians and data center staff to prepare your space. Recurring expenses typically include the cost of space (for our purposes consider this an aggregate expense of space, power, cooling, and infrastructure maintenance), the cost of bandwidth, cross connect fees to ISPs or NSPs, and the cost of data center staffing. The most important portion of this objective is to determine how much it will cost your organization if your environment is down. The cost of downtime will allow you to build your business case when presenting the costs of data center service to your executive leadership team.
Now that you have a working budget we can begin diving into the data center details, lets first look at physical security. There are numerous controls that need to be in place to secure a data center. Some primary security topics include the site location, site perimeter, computer room access, facilities, outsiders, users, and disaster recovery. The topic of physical security has been reviewed in detail by the SANS Institute. They provide a checklist which can be found at www.sans.org.
Based on the area of business you operate in you may be held accountable to adhere to certain compliances such as HIPPA or Sarbanes-Oxley. You will want to choose a provider who has a SAS70 certification that will ensure your compliance objectives are met. In some cases your auditors may need to access the facility to verify your controls are in place. Prior to choosing a provider make sure they are willing to allow your auditors escorted access should they require it. Security is of paramount concern and you should review the SANS checklist when touring any prospective facility.
The next topic is your physical footprint at the facility. You need to determine how much space and power you will need to accommodate your computer and network infrastructure and also what type of power it will require. We will discuss this further in the power section of the article but you will want to note the size in rack units of each device, the power type it requires (alternating or direct current), the voltage it operates on, and the amperage it will draw. Data center providers normally offer space in 1/3 rack, ½ rack, full racks, and cage space to accommodate multiple racks or racks that do not have enclosures. Racks range in size but are typically around 44 units (a 1 unit server takes up 1 unit). Each provider will have a different amount of power they will provide per rack (based on cooling capacity) so you will need to ask that when determining your space requirements. It is also important to ask if you can have adjacent cabinets, free cross connects between cabinets, and a first right of refusal on empty cabinets for future growth.
Continuing on to power, there are several items to consider when choosing your data center provider. The majority of the decisions you will make concerning power will depend on your availability (uptime) and system requirements (A/C, D/C and voltage). Data centers are classified into tier levels from one to four, with one being the least available to four being the most available. When deriving a tier classification for a data center many aspects are taken into account but at the end of the day the classification is a way of expressing the average expected uptime percentage based on redundant systems in place and fault tolerance of the site. Different tier classifications tie into the power discussion based on how many power grids the site sits on, the alternate energy sources (diesel or turbine generator), automatic and/or static transfer switches, transformers, and remote distribution centers (RPPs) available at a particular classification level. You will often see data center providers use the n+1 terminology which simply means they offer what is needed plus one backup or alternate. The term is often applied to power and cooling.
Power is going to be your largest recurring cost by far so it is important to perform a cost analysis to determine if redundant power is viable for your application. In order to properly design a solution you will need to determine the type of power your equipment utilizes and if you will be delivering redundant power to each device.
This can be accomplished if the data center can provide you a diverse path of power fed as such; Utility/Generator → UPS A → Transformer A → RPP A → Customer PDU A → Customer Server Power Supply A along with a path on the B infrastructure.
Utilizing the aforementioned design, the critical load can suffer a loss of an entire path of power and still remain on line. This will also allow your provider to perform maintenance on the power infrastructure without the need for you to power down your equipment. Most providers will offer single phase 120 volt and 208 volt and three phase 208 volt power. Some providers will offer 48 volt D/C power. Standard A/C breakers are 80 percent rated at 20 or 30 amps. This means that they can safely operate at 80 percent of their rating 100 percent of the time. If you plan to utilize redundant circuits you will need to drop that percentage to 40 percent as a single circuit may need to pick up the other 40 percent should its redundant counterpart fail. You should ask your prospective data center provider if they provide the PDU (power strip in the rack) or fuse panel (for D/C applications) or if they expect you to. If they provide it, you need to ask if it includes remote reboot and SNMP capabilities and if you will be given administrative level control of it. If you are expected to provide it, ensure that the specifications of the PDU you choose are approved by the data center and that the receptacle will match the power cable on the PDU. It will also be important to determine the cost (setup fee) and availability of future power should your needs grow over time. With the recent increases in server density and power consumption for blade servers, many sites are unable to deliver additional power until costly upgrades to the infrastructure are implemented.
Heat is a byproduct of the work servers do and this creates a large problem for data centers. The optimal climate will increase the lifespan of your equipment. You can again refer to the data center tier classification system to determine the amount of redundancy a prospective data center has to ensure that it fits your uptime requirements. If the cooling fails in a data center it will not take long for the temperate to reach critical levels where equipment will fail or be shut off. You will want to contact your equipment vendors to verify what temperature and humidity range your equipment was designed to operate in. Data center operators have traditionally kept temperatures at or below seventy degrees Fahrenheit but research has shown that traditional mindset is doing nothing more than wasting money and energy.
This cost savings will trickle down to you as the customer if the energy costs of the data center are reduced.
Connectivity and service is the next topic of discussion. You will need to explore what your organizational needs are for bandwidth to the Internet, to satellite locations, and other data centers. The key to connectivity is selecting a carrier neutral data center. If the facility is on net with many ISPs or NSPs you will be in a very advantageous situation for acquiring cost effective solutions for Internet access, MPLS networks, point to point T1s, and PRIs for PSTN termination. You will want to evaluate which carriers are on Net and how much it will cost to cross connect you to each carrier. This is another area where you can perform a cost analysis to determine if implementing redundant paths to the Internet fits your needs. Internet routing protocols and the hardware necessary for implementation is outside the scope of this article however those familiar will know that the equipment necessary to receive full routes, the requirements of ARIN for acquiring IP space, and the know how of staff to configure and support that technology are not cheap. It may be wise to determine if the data center provides a solution that may meet your needs halfway. You may find that they offer NSP style services and can provide you a redundant pair of copper or fiber optic drops which allows you to utilize their egress point to redundant ISPs.
Lastly, we must discuss support. You will need to establish that the facility offers 24x7x365 phone support (manned by trained technicians) and physical access in case you need to repair something off hours. Some data centers offer hands on service that is top notch while others will only offer reboots. You should determine what if any services the data center offers, the skill level and certifications their staff has, and the cost per hour should you utilize them. This becomes critical if you are considering a site that is not geographically near your operational area.
In summary, this article should provide you with base for considering a data center that meets your needs. Each one of the topics covered in this article can be explored at a much more granular level and I encourage you to do so before signing any contracts. You should take a tour of any facility you are considering and be sure to bring a checklist with you to determine where your needs are or aren’t met. Speak with the key personnel at the site as well. The staff should be able to answer any of your questions and provide references for you to speak with should you be inclined to do so. You should ask questions about routine maintenance and ask to see service contracts with vendors and priority refueling agreements in case of extended utility power loss. Be aware that just about every piece of equipment will fail at some point. You want to make sure that facility you choose understands that and has plans in place to recover quickly and hopefully without any impact to your organization.
Joe Cosmano is the director of engineer at Atlantic.Net. He manages a 25,000 sqft data center powering the mission critical operations of thousands of clients. He has been with Atlantic.Net since 2005 and currently holds both Red Hat and Cisco Technical Certifications.
