From high-speed interconnects and resource disaggregation to persistent memory and evolving software stacks, the landscape of data center computing is rapidly changing.
Many modern applications are powered by data center platforms and services. These data center-based services are distributed systems.
The End of Moore's Law and the Rise of Specialization
For decades, Moore's Law—the observation that transistor density doubles approximately every 18 months—has been a driving force in computing. As this trend is slowing down, the industry has shifted towards specialization to continue improving performance.
This specialization comes in various forms:
- Many-core systems and multi-threading: A direct response to the slowdown in single-core performance improvements.
- Specialized components: GPUs and TPUs for AI workloads, FPGAs for programmable circuits in network elements.
- New memory and storage technologies: Addressing data access and processing requirements.
High-Speed Interconnects and RDMA
One significant trend is the adoption of high-speed interconnect networks, such as InfiniBand, with Remote Direct Memory Access (RDMA) capabilities. RDMA allows for direct memory access from the memory of one computer into that of another without involving either computer's operating system. This technology enables:
- Higher bandwidth and lower latency compared to traditional Ethernet networks
- Reduced CPU load for network operations
- New possibilities for shared memory access across distinct physical nodes
Resource Heterogeneity and Disaggregation
Modern data centers are becoming increasingly heterogeneous, with a mix of different types of compute, memory, and storage resources. This heterogeneity, combined with changing workload requirements, has led to a trend towards resource disaggregation.
Disaggregation allows different types of resources (compute, memory, storage) to be independently added and scaled. This approach offers several benefits:
- Greater flexibility in resource allocation
- Improved utilization of hardware resources
- Ability to scale specific resource types independently
Persistent Memory: Bridging the Gap Between Memory and Storage
The emergence of persistent memory technologies, such as Intel's Optane, is blurring the traditional boundaries between memory and storage. These technologies offer:
- Byte-addressable access like DRAM
- Persistence like storage devices
- Performance closer to DRAM than traditional storage
- Larger capacity than typical DRAM configurations
This development has implications for system design, particularly in areas like data persistence and recovery.
The Evolution of Software Stacks
The software stack in data centers is also evolving. There is a shift from virtual machines to containers and microservices. This change affects how we package, distribute, and manage software components in cluster systems and data centers.
Key Concepts
Moore's Law The observation made by Gordon Moore, co-founder of Intel, that the number of transistors on a microchip doubles about every two years, while the cost halves. This trend has been a driving force in technological advances in the digital age.
Specialization In computing, specialization refers to the design of hardware or software components to perform specific tasks more efficiently than general-purpose components.
GPU (Graphics Processing Unit) A specialized processor originally designed to accelerate graphics rendering. Modern GPUs are used for a wide range of parallel processing tasks, particularly in machine learning and AI.
TPU (Tensor Processing Unit) A specialized AI accelerator application-specific integrated circuit (ASIC) developed by Google specifically for neural network machine learning.
FPGA (Field-Programmable Gate Array) An integrated circuit designed to be configured by a customer or a designer after manufacturing. FPGAs contain an array of programmable logic blocks and a hierarchy of reconfigurable interconnects.
RDMA (Remote Direct Memory Access) A direct memory access from the memory of one computer into that of another without involving either computer's operating system. RDMA allows high-throughput, low-latency networking.
InfiniBand A computer networking communications standard used in high-performance computing that features very high throughput and very low latency.
Resource Heterogeneity In data centers, this refers to the presence of diverse types of computing resources (e.g., CPUs, GPUs, FPGAs) and memory/storage technologies within the same system.
Resource Disaggregation An approach to data center architecture where different types of resources (compute, memory, storage) are separated into distinct pools that can be independently scaled and allocated as needed.
Persistent Memory A type of memory that combines the performance characteristics of DRAM with the persistence of storage devices. It retains its contents even when electrical power is removed.
Intel Optane A brand of persistent memory and solid-state drive products developed by Intel using 3D XPoint technology.
Virtual Machine A software emulation of a computer system, providing functionality of a physical computer. It allows multiple OS environments to co-exist on the same physical hardware.
Container A standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another.
Microservices An architectural style that structures an application as a collection of loosely coupled, independently deployable services.
Distributed System A system whose components are located on different networked computers, which communicate and coordinate their actions by passing messages to one another.
Data Center A facility composed of networked computers and storage that businesses or other organizations use to organize, process, store and disseminate large amounts of data.