DCs total Power distribution from 1 to 100 MW
Data centre infrastructure is growing rapidly. In the past years, total capacity has increased from less than 1 gigawatt in the 90s to several dozen gigawatts now, demonstrating significant growth. This expansion includes both the number and power of DCs, reflecting a global trend toward heightened infrastructure and capacity, particularly in response to emerging technologies like artificial intelligence.
A research report by Spherical Insights & Consulting shows that the Global Data Center Infrastructure Market Size will increase from USD 50 billion in 2021 to USD 120 billion by 2030, with a Compound Annual Growth Rate (CAGR) of 12% expected during the forecast period.
Power per Rack
When it comes to power allocation, there has been a significant shift towards higher power allocations per rack as well. Historically, servers and processors were low-power, allowing for the accommodation of many servers within a rack with a power capacity of 5 or 4 kW. However, as servers and processors became more powerful, the existing power allocations proved insufficient, necessitating a reassessment of power needs and allocations.
During the mid-00s, the introduction of Blade servers marked a turning point, leading to a substantial increase in density within racks.
This innovation also brought about challenges, particularly in terms of cooling, as the densely packed Blade servers struggled with adequate air flow, leading to overheating issues. Consequently, the industry saw a decline in the use of Blade servers and a corresponding reduction in power allocations per rack.
Following this period, there was a resurgence in the use of conventional servers, leading to a decrease in power allocations per rack, as the industry pivoted towards more convenient server arrangements with sufficient cooling. This trend towards lower power allocations, however, was short-lived, as the demand for more powerful servers and GPUs required higher power allocations once again.
Currently, the industry has settled on an optimal power allocation of approximately 12-15 kW per rack unit, with some operators already implementing 30 kW infill for GPU solutions and anticipating a future demand for 50 kW. This trajectory indicates a continuous increase in power requirements, driven by the escalating demand for infrastructure, computing power, and GPUs to support artificial intelligence applications.
HV, MV, and LV Power Distribution Approaches
Voltage types are traditionally classified into the following:
Voltage Type | Range |
Low Voltage | Below 1000 volts |
Medium Voltage | Between 1000 volts and 35 kilovolts |
High Voltage | Exceeding 35 kilovolts |
In data centres, high or medium voltage is preferred over low voltage for efficient energy transfer. When being delivered to a DC, voltage is transformed step by step: if delivered at high voltage, it is first transformed to medium voltage and then to low voltage. This approach helps in cost-saving on conductors, as using high-power strength would require more expensive conductors.
Corporate DCs vs Commercial DCs
Initially, the data centre industry primarily consisted of commercial DCs, with no large-scale corporate facilities built until the late 90s. However, the trend has shifted over time, with corporate data centres experiencing growth, primarily driven by hyperscalers like Google, Facebook, and Amazon.
These hyperscalers, being IT giants, build massive infrastructure to support their services, contributing significantly to the growth of corporate data centres. Large companies like Vantage also invest in building their own infrastructure. This growth trend is reflected in server equipment, where power continues to increase according to microelectronics laws.
Server and Network equipment
The same situation we can witness with server equipment, initially they were low-powered, but with advancements in microelectronics, power has increased and will continue to do so. This growth is evident in Total Distribution of Power (TDP) graphs for Intel processors and NVIDIA GPU chips, demonstrating the growth over years.
Similarly, network equipment has seen a rise in data transfer speeds, progressing from 1 megabit to potentially 1 terabit, representing a million-fold increase. Consequently, modern network switches consume significantly more energy, around 1.5 kilowatts compared to 100 watts for older models, due to increased speed and power consumption.
DC Consumers
The general consumers of computing power installed in data centres worldwide primarily fall into two categories. Firstly, there are individuals who use the services of hyperscalers such as Facebook, Instagram, and iPhone features like Siri. The electricity used to power these services is consumed by users for various activities like sharing photos on Instagram, reflecting a growing trend of mass adoption of hyperscale services and algorithms for navigation and decision-making.
The second group comprises corporations that integrate IT equipment into their business processes, including production, sales, and service operations. These companies rely heavily on IT infrastructure to conduct their core business activities. The demand for computing resources among such corporate entities is increasing rapidly, with IT services becoming critical for businesses across various sectors, from small bakeries to large corporations. As a result, more companies are embracing IT infrastructure to streamline their operations, whether through purchasing their own servers for co-location or opting for cloud solutions provided by platforms like KG-600. Cloud platforms offer a convenient and hassle-free option for businesses to access computing resources without the need to invest in physical hardware, as the platform handles all server management and maintenance, providing a ready-made solution for customers.
Load Profiles
Currently, data processing centres primarily operate under three key load profiles: compute, storage, and GPU, each serving distinct functions. Compute resources are tasked with executing calculations and processing data, while storage resources are dedicated to storing vast amounts of data. GPU resources, on the other hand, are designed for intensive artificial intelligence computations, particularly evident in the recent surge in demand for tasks like training large linguistic models such as chat-GPT.
This surge in GPU demand has resulted in a notable market shortage, with production capacities struggling to meet the escalating needs. Efforts to scale up production face challenges, with new facilities taking years to construct. Moreover, the shortage is expected to persist as the demand for GPU and machine learning resources inevitably grows.
In anticipation of this growing demand, data centres must also consider the accompanying need for expanded storage capacity. As GPU and machine learning resources require substantial data storage for processing, there will be a corresponding increase in the requirement for storage infrastructure. Looking ahead, the data centre landscape is poised to witness the emergence of extensive GPU clusters alongside robust data storage facilities, with electricity consumption from these facilities projected to constitute a significant portion of overall energy usage within DCs.
How IT trends affect the DC industry?
The most obvious trend observed over the past 5 years is fueled by the explosive growth, popularity, and hype surrounding models like GPT and linguistic models in general. This surge in interest is driven by both giant corporations and smaller companies alike, all seeking to use or develop these models to enhance their operations. The focus isn't solely on linguistic models but also extends to other applications like text-to-video, text-to-image, and deepfake technology, which collectively serve as a major growth driver.
ML Technologies Growth
To support the demands of machine learning applications, neural networks, and similar technologies, computing devices such as GPUs are becoming in greater demand.
As most computational models are implemented on large clusters of GPUs data centre management has to deal with their high power consumption and heat generation. These chips, though physically small, can consume significant amounts of electricity, ranging from 100 to 800 watts per microchip. Consequently, they produce considerable heat, with operating temperatures reaching up to 120 degrees Celsius. This level of heat generation is akin to that of a boiler, capable of boiling water without issue.
Power per Rack and Power per Device Increase
To address the challenges posed by GPU-intensive workloads, data centres have adopted several strategies. Firstly, they have upgraded their electrical infrastructure to support the increased power demands of GPU clusters. This includes installing more robust electrical wiring and power distribution systems, with facilities accommodating power densities of up to 100 kW per rack.
Need for Advanced Cooling Solutions
Furthermore, data centres have implemented advanced cooling solutions to manage the heightened heat output from GPU clusters effectively. Traditional air cooling methods are often insufficient, prompting the adoption of alternative cooling techniques such as water cooling or immersion cooling. These methods involve circulating coolants around or directly through the hardware to dissipate heat more efficiently.
Additionally, data centres are optimising their physical layouts to accommodate larger and
more effective cooling systems. This involves allocating more space for the installation of powerful radiators and air handling units, ensuring adequate airflow to cool the components effectively.
Internet of Things
While the focus on machine learning remains acute, the Internet of Things (IoT) has not been forgotten either. Despite its diminished media presence, IoT devices continue to proliferate in both household and corporate environments, contributing to the exponential growth of network traffic.
The global number of individuals using the Internet has grown from 1 billion in 2005 to 5.5 billion in the mid-twenties
Image source: Research Gate
This growth necessitates continuous upgrades to network infrastructure, with communication channels requiring greater throughput to accommodate the increasing volume of data exchange.
In the late 90s and early 2000s, the standard port speeds started at 10 megabits per second (Mbps) and gradually increased to 100 megabits per second. These standards persisted for some time, with 100 megabits per second being commonly used. As technology progressed, faster speeds became necessary, leading to the adoption of 1 gigabit per second (Gbps), followed by 10 gigabits per second.
In the current era, the standard for server equipment has shifted to 25 gigabits per second per port, while for switches, it has reached 100 gigabits per second per port. Additionally, newer standards such as 200 gigabits per second and 400 gigabits per second per port have begun to emerge in the market, indicating a continuous push towards higher data transfer rates.
Looking ahead, the expectation is for network equipment manufacturers to release terabit-per-second port standards in the coming years, further advancing the capabilities of networking infrastructure.
Spine-Leaf and CLOS Architecture
As the demand for data transmission continues to grow, traditional network architectures face limitations in handling the increasing traffic volume. The transition from 10 Mbps to 1 Tbps over the past decades reflects the exponential growth in network capacity, accompanied by significant energy consumption by modern network equipment.
Structural Cable Systems and spine-leaf architectures address these challenges by introducing hierarchical structures that distribute network traffic efficiently. SCS, resembling a star-shaped cable network, has become obsolete due to its inability to accommodate the massive traffic demands. Instead, the spine-leaf architecture, characterised by its multi-level, hierarchical design, has emerged as the de facto standard in the industry.
In contrast to the centralised structure of SCS, the spine-leaf architecture features interconnected layers, allowing for scalable expansion and enhanced throughput. By distributing the traffic load across multiple levels, this architecture mitigates the strain on individual components and facilitates seamless communication between network elements.
The spine-leaf architecture's scalability and efficiency make it well-suited for modern data centres and network infrastructures. However, its implementation requires careful consideration of factors such as equipment compatibility, power consumption, and cable management.
Content growth
The surge in content consumption is driving a consequent increase in outgoing traffic across the internet. This growth is not limited to just the Internet of Things; rather, it includes a wide range of digital content formats, with a notable shift towards high-definition video content such as 4K and even 8K resolution. The demand for superior-quality visuals extends beyond videos to include images, which are now expected to be of the highest resolution and quality.
As a result, the overall size of content files has grown, leading to a corresponding rise in internet traffic volume. This trend is evident across various industries and companies, including our own, where the proportion of video content within overall content has reached 100%. Consequently, there is a pressing need for network infrastructures to accommodate this increase in traffic while maintaining optimal performance.
This exponential growth in content quality and size is fueled by advancements in technology, particularly in mobile devices and cameras, which continually push the boundaries of resolution and image quality. As mobile phones evolve into high-resolution screens and cameras improve their capabilities, the demand for higher-quality content becomes self-perpetuating, driving further innovation and expansion in digital content creation and consumption.
Equipment Installation Speed
To expedite the installation of network equipment, another shift has occurred in the industry. Previously, companies would spend considerable time and resources on logistics and installation processes. Servers would be delivered, unpacked, and meticulously set up over several days or even weeks. This traditional method involved multiple steps, from unboxing and installation to configuration and testing, all of which contributed to lengthy deployment times.
However, the advent of pre-configured and tested equipment has revolutionised the installation process. Manufacturers now pre-configure servers and other network components at the factory, ensuring they are ready for immediate deployment upon arrival. These pre-assembled solutions are delivered in self-contained racks, ready to be rolled out and connected with minimal effort. As a result, what once took weeks or months can now be accomplished in a matter of hours or even minutes.
This approach to equipment installation has become standard practice for hyperscalers and large companies alike. By reducing the time and resources required for deployment, organisations can rapidly scale their network infrastructure to meet growing demands. This efficiency improves operational agility and enhances productivity and customer satisfaction.
Display of future DC
As we have previously discussed, in the future landscape of corporate and commercial data centres, a significant role will be played by MLGPUs in resource allocation. Currently comprising around 10% of resources, MLGPUs are projected to dominate up to 50%, 60%, or even 70% in the near future. Consequently, there will be a shift towards specialised cooling systems, such as large GPU clusters with water cooling, to accommodate this trend.
Additionally, the share of data storage units is expected to increase alongside the growth of MLGPUs, indicating a parallel demand for enhanced network capacity and accelerated equipment installation. This reflects the need for the industry to adapt quickly to escalating demands, requiring adjustments in logistical processes, cooling systems, and overall facility design to facilitate rapid expansion.
Moreover, the conservative nature of the industry is being challenged as the focus moves towards rapid scalability and increased energy consumption. Companies are now planning with a substantial margin, aiming to build an infrastructure capable of accommodating triple the forecasted demand. This shift marks a departure from previous concerns about overbuilding, with leaders now prioritising robust infrastructure to meet evolving demands.
In summary, the future of data centres will be characterised by swift adaptation, significant growth, and a proactive approach to infrastructure development.