What are the stages of enterprising infrastructure?
- Monolithic app, with minimal network demands and preparatory protocols.
- Client server, with high network demand inside the enterprise, applications walled within the enterprise, TCP/IP plus proprietary protocols.
- Web applications, with ubiquitous TCP/IP, access from anywhere, servers are broken into multiple units
- Microservices, infrastructure moved to cloud providers, servers broken into Micro services, increase of server to server traffic.
Regarding networking, what is the consequence of the increase in the performance of servers?
The performance of servers increase overtime, the demand for inter-server bandwidth naturally increases as well
Why networking has no straightforward, horizontal scaling solution
Because although doubling the leaf bandwith is easy, if you assume that every server needs to talk to every other server, we need to deal with bisection bandwidth
What is the bisection bandwidth?
It is the bandwith across the narrow line that equally divide the cluster into two parts
It characterizes network capacity since randomly communicating processors must send data across the middle of the network
We assume that every server needs to talk to every other server, we need to double not just the leaf bandwith, but section bandwidth
What are the principles of designing the data center network?
Very scalable in order to support a very large number of servers
Minimum cost in terms of basic building blocks
Modular to reuse simple, basic modules
Reliable and resilient
Data center network can be classified into three main categories
Switch-centric architecture. Uses switches to perform packet forwarding.
Server-centric architecture, uses servers with multiple network interface cards (NIC) to act as switches in addition to performing other computational functions
Hybrid architectures. Combine switches and servers for packet forwarding.
What are the possible traffics flows inside a data Center network
- North-South Traffic:
• Definition: This is the traffic that flows between external clients/users and the data center. It moves in and out of the data center.
• Examples:
• Client requests to a web server.
• Data uploads or downloads from external users.
• Path: External network (e.g., internet) → data center firewall → load balancers → application servers (and vice versa). - East-West Traffic:
• Definition: This is the traffic that flows within the data center, between servers and other infrastructure components.
• Examples:
• Communication between application servers and database servers.
• Data replication between storage systems.
• Server-to-server communications for distributed computing tasks.
• Path: Server-to-server traffic within the same data center, often involving ToR, EoR, or MoR switches.
What is the East-West traffic generally used for
Storage replication
VM Migration
Network function virtualization (NFV)
What are the layers in a classical network
- Core Layer:
• Function: The primary role of the core layer is to provide high-speed, reliable connectivity between different parts of the network, often connecting multiple aggregation/distribution layers. - Aggregation (or Distribution) Layer:
• Function: The aggregation layer serves as an intermediary between the access and core layers, aggregating traffic from multiple access layer switches before forwarding it to the core layer. It often handles policy enforcement, routing, and load balancing. - Access Layer:
• Function: The access layer is where end devices, such as servers, storage devices, and other endpoints, connect to the network. It is the first point of entry for data into the network infrastructure.
What are ToR architectures?
Top of Rack (ToR) architecture is a common design approach in data center networks where a network switch is placed at the top (or sometimes the middle) of each server rack. This architecture simplifies cabling and improves network management and performance.
How is the designing layout of ToR architecture
- Design and Layout:
• Switch Placement: A network switch is installed at the top or middle of each server rack.
• Server Connectivity: Servers within the rack connect directly to the ToR switch via short Ethernet cables.
• Uplink Connections: The ToR switch connects to aggregation or core switches via uplinks, which are usually high-speed connections (e.g., 10Gbps, 40Gbps, or higher).
What are the advantages and disadvantages of ToR architectures
Since the number of cables is limited, it offers simpler cabling. The number of ports, therefore, per switch is also limited.
Disadvantages: higher complexity, off switch management
What are EoR architecture?
End of row architecture is the one that positions aggregation switches at the end of a line rack of each corridor. Servers in the rack are connected directly to the aggregation switch in another rack.
What are the advantages and disadvantages of EoR architectures
Disadvantages: Since aggregation switches must have a large number of parts, it offers more complex, cabling, longer cables are required also
Advantages: simpler switch management
How can we increase bandwidth in a tree tier network?
Bandwidth can be increased by increasing the switches at the core and aggregation layers, and by using routing protocol, such as equal cost multiple path (ECMP) the equally shares the traffic among different routes
What are the advantages and disadvantages of increasing the bandwidth in a three tier network?
Although it is a very simple solution, it can be very expensive in larged data centers
How is a DCN Clos Topology devided?
- Leaf Layer:
• Position: The leaf layer is the first level of switches (ToR switches) that connect directly to servers and other endpoints. - Spine Layer:
• Position: The spine layer consists of high-performance switches that form the core of the network. Dedicated switches (aggregation switches)
Where does the Clos topology comes from? And what is its main logic?
Spine – leaf topology are borrowed from the telephone world
Given M middle stage switches, and in input and output, if M is greater or equal than N, there is always a way to rearrange communications to free a path between any pair of idle input/output. Also, if M is greater or equal than the double off N minus one there is always a free path between any pair of idle input/output.
What is the function of the leaf?
Function: These switches handle traffic within the rack or row and connect to the spine switches for inter-rack communication.
What is the function of the spine?
Function: Spine switches interconnect all leaf switches, providing a high-speed backbone for the network.
What are the advantages of Clos topology
Use of homogenous equipment
Simple routing
The number of hops is the same for any pair of nodes
Small blast radius
What is a point of delivery?
A POD or point of delivery is a module or group of network, compute, storage, and application components that work together to deliver a network service
What are some of the advantages of POD?
It increases the modularity, scalability, and manageability of data
What is a camcube server centric architecture?
CamCube is a novel server-centric architecture designed to optimize data center network (DCN) performance, scalability, and efficiency. Developed by researchers at Microsoft Research, CamCube rethinks traditional data center designs by integrating network functionality directly into servers. Here are the key features and concepts of CamCube server-centric architecture:
Key Concepts
- Server-Centric Design:
• Unlike traditional DCN architectures that rely on dedicated network switches and routers, CamCube integrates network functionality into servers themselves. Each server has multiple network interfaces, and servers directly connect to each other
