High-performance computing (HPC) has traditionally been associated with the enterprise-level supercomputers and mainframes performing intensive scientific, financial, and engineering research. By aggregating the computing power of several machines into clusters and using specially designed software that ties them together, HPC systems make it possible to run tens of thousands of parallel processing tasks as if those machines were one computer.
Recent trends in cloud computing and the miniaturization of processing technology, however, have changed the way organizations utilize HPC resources. The growth of cloud and colocation data centers have played a major role in these trends.
Although long exclusive to large companies with the resources to build these powerful machines, HPC has become appealing to many other industries due to its ability to process massive amounts of unstructured data to identify actionable business insights. With so many computing processes moving to cloud platforms and software systems becoming more standardized, companies are looking at ways to incorporate HPC into their data operations.
Jim Ganthier, VP and GM of Engineered Solutions HPC and Cloud at Dell, elaborates on this trend:
As the commoditization of IT expanded over the years, it has had a tremendous impact on HPC. Companies used to build everything from the ground up for their supercomputers, which created complex, proprietary, and expensive systems. Now, most of the underlying technologies are standardized from Intel and other component providers, making it easier to create HPC solutions fast and affordably.
Finding a data center partner will be crucial for many of these companies, even if they’re primarily turning to cloud services for their HPC needs. The continued growth of the Internet of Things (IoT) market will only increase the amount of data they have to manage, and much of that data will remain in a cloud environment throughout its entire lifecycle. “The rapidly increasing volume of data presents an opportunity for enterprises to adopt cloud for HPC workloads so they can efficiently and securely analyze, share and generate new business insights,” notes Keri Olson, director of HPC offerings at IBM Cloud.
The biggest challenge associated with data center HPC is power requirements. While HPC systems can perform a wide array of data-intensive operations, all that processing power comes at a major energy cost. Running such a high power density deployment generates significant heat as well, requiring cooling systems to accommodate them. For most small to medium-sized businesses, the capital resource necessary to build this sort of computing infrastructure is cost-prohibitive.
Colocation data centers may be able to deploy HPC services for some of these companies. While not all facilities are set up to provide this specialized form of computing, Aegis Data CEO Greg McCulloch explains how they might do so:
High Performance Computers simply aggregate computing power in a way not typically associated with standard server infrastructure. It requires denser banks of computer resource to minimize latency and increase capacity, while minimizing floorspace. In a data center, optimizing for this will happen by redesigning server racks, for example removing the need to have cooling slots between processing units. This is, of course, provided that the data center in question can provide the power and alternate cooling systems needed to support higher contiguous rack stacking, while limiting the cost for the end user.
Assessing infrastructure matters when evaluating the HPC potential of a data center. Quocirca’s Clive Longbottom cautions companies to be careful about locking themselves into a facility’s hardware and deployment limitations:
A dense HPC platform will typically draw around 12kW per rack and in some cases 30kW or more. Can the colocation facility provide that extra power now – not just promise it for the future? Will it charge a premium price for routing more power to your system? Furthermore, do the multi-cabled power aggregation systems required include sufficient power redundancy? Careful consideration must therefore be given to future-proofing when it comes to power availability to avoid the potential for unplanned downtime, or the disruption and cost involved in the event of migration/de-installation should the facility become power-strapped.
While HPC scalability may present problems for many private data centers, colocation facilities with sufficient power density and connectivity options may have the flexibility to overcome these challenges. Data collected through edge computing architectures by way of IoT devices can be incorporated into a hybrid network that allows companies to easily manage resources in secure, private environments while retaining the scalable computing power of the public cloud.
Jean-Luc Assor, Worldwide Manager of Hybrid HPC/HPC Cloud at Hewlett Packard Enterprise, sees clear benefits to combining edge computing with hybrid cloud architectures for HPC workloads:
HPC growth often can’t be served by the space, energy, and environmental footprint of constrained legacy data center operations, which is why more companies are looking for ways to hand off their HPC operations, either through classic outsourcing or by hosting operations in a private or public cloud. A parallel HPC evolution is the move to the network edge. This dual movement of HPC workloads to off-premises and to the edge requires integrated hybrid architecture from the edge to the data center to the cloud.
Dell’s Ganthier agrees that hybrid cloud deployments present an ideal solution: “The hybrid cloud approach is best for most companies, providing flexibility for additional resources as needed and guaranteed use of your own systems for mission-critical operations.”
With more and more companies looking for ways to manage their unstructured data effectively, HPC solutions will surely become more appealing. Whether they provision those services through cloud providers or utilize the computing infrastructure of a colocation data center optimized for HPC, companies will need to work closely with data centers to build the solutions that are best suited to their unique business requirements.