Access a website on the Internet

Let’s walk through an example where we need to access a website on the Internet, say “distributedsystemsblog.com” from browser or curl command on terminal. We have discussed what happens when a users perform any action in Linux in this article, this post will focus more on networking concepts.

DHCP

When our computer or system just boots up, it needs a way to get the IP address - a numerical label that serves as the host’s identification, this is done with the help of the DHCP protocol.

A typical DHCP process involves 4 steps:

1. Discover step: our client system broadcasts a very first request message (DHCPDISCOVER) to server’s port 67 Using UDP with the following information in the link-layer Ethernet frame:
   • Broadcast IP address: 255.255.255.255:
   • Broadcast MAC addresses: FF:FF:FF:FF:FF:FF
   • Source IP address: 0.0.0.0
   • Source MAC address: The computer’s MAC address
2. Offer step: the server(s) broadcast DHCPOFFER to users’ port 68. The client can only receive offer(s) from 1 or many servers. The message contains: the client’s IP address, subnet mask, default gateway IP address, DNS IP address, IP lease time and DHCP server’s IP address.
3. Request step: the client, broadcasts DHCPREQUEST message to tell every servers its choice by putting the IP address of the selected DHCP server in the DHCP Server Identifier (option 54) field of the DHCPREQUEST message.
4. Acknowledge step: the chosen server broadcasts DHCDPACK with the same information as Offer step and hopefully the clients can receive the message.

Step 3 and 4 are needed because there might be more than 1 DHCP server in the network. We notice that every packets sent are via broadcast with UDP protocol since the client has NOT been allocated IP. Moreover, TCP does not support broadcast and also requires unique IP address.

The DHCP server can also be configured to perform DDNS to update A and PTR record. For example, in Discover step, the client can also include its’ hostname in the messages. When the DHCP server responds with an IP address, it usually provides a domain name. The combined host name (from the client) and domain name (from the server) form an FQDN (fully qualified domain name). The DHCP server then updates the DNS server with the FQDN (in PTR record). On the other hand, if the DHCP client does not send a hostname, the DHCP server provides a lease but does not update DNS. We will discuss more about DNS later.

Our computer finally get its own address. However, in order to communicate to the rest of the world, it needs to know 2 pieces of info:

1. The IP address of destination, i.e. distributedsystemsblog.com in our case and
2. The MAC and IP addresses of the gateway or router (gateway and router can be used interchangeable in this context)

For Info 2: We know the IP address of router from DHCP process, but we do not know the router’s MAC address. Every system has an ARP table in its memory with expired time, which contains mappings of IP-to-MAC. If our computer does not have the gateway’s MAC in ARP table, it needs to retrieve the address with ARP protocol.

• The Internet is divided into different networks/subnets and many networks are also internal within organizations. Routers exist in different subnet to facilitate the communication between devices of different subnets. Data from our system first need to go through the gateway/router before being routed to the destination address. If the address of distributedsystemsblog.com is in the same network of our computer, no gateway is required. However, it is rarely the case, and hence computer needs to know address of its gateway.
• Mac address is 6-bytes link-layer address and typically expressed in hexadecimal, i.e. 00:00:5e:00:53:af. When a company wants to manufacture network interfaces, it purchases a chunk of the address space consisting of 2^24 addresses for a nominal fee. The IEEE (the authority that managed the allocation of Mac addresses) allocates the chunk of 2^24 addresses by fixing the first 24 bits of a MAC address and letting the company create unique combinations of the last 24 bits for each adapter.

ARP

A general process of ARP looks like this:

1. The client broadcasts an ARP packet into the subnet, placing ARP message into the link-layer Ethernet frame together the following information:
   • Destination IP address: The gateway’s IP address
   • Destination MAC addresses: FF:FF:FF:FF:FF:FF
   • Source IP address: The computer’s IP address
   • Source MAC address: The computer’s MAC address
2. The ARP query is received by all the other hosts on the subnet. The ARP modules of these hosts checks to see if its IP address matches the destination IP address in the ARP packet. The one with a match sends ARPreply message to indicate its MAC address corresponds to IP address. The query ARP message is sent within a broadcast frame, whereas the response ARP message is sent within a standard frame
3. The client then update its ARP table and from now, it can encapsulate the gateway’s MAC as destination MAC in a link-layer frame.

The next part is to get the IP address of distributedsystemsblog.com. We will now introduce The Domain Name System (DNS) - a hierarchical and decentralized naming system for computers that helps to translate domain names into IP address.

DNS

Let’s first introduce the DNS. DNS is organized in a tree-like hierarchy. The top of that hierarchy is the root domain whose label is an emty string in the DNS hierarchy. There are 13 logical root name servers that implement the root name space domain (although there are more physical servers distributed around the world for redundancy).

Like all name servers, root name servers store information provide name resolution services for, all the nodes within the “root zone”. This includes top-level domains (TLD), i.e. .COM, .EDU, .GOV, .MIL, .NET, etc. and subdomains. The root name servers are used to obtain the names and addresses of the authoritative servers for each of these TLDs, i.e. if we want to resolve the name “ www.distributedsystemsblog.com”, the root name servers are where a resolver would find the identity of the name server that is responsible for “com”.

When a request reaches the root DNS servers, they answers requests for records in the root zone or have to delegate to an authoritative server for the appropriate top-level domain (TLD) because it would be so much overhead for root to handle so many requests.

DNS Requests are sent with UDP to server’s port 53, however, DNS also uses TCP, especially for zone transfer - a process where primary DNS servers notifies and sends updated zone data to its secondary servers. Moreover, even DNS query can also be used with TCP, i.e. UDP over DNS is limited to 512 bytes (somewhat outdated), the query cant use TCP for larger payload.

Some common records in DNS header:

• ‘A’ record: address record – mapping between address and domain name (In IPv6, it is “AAAA”
• ‘NS’ record: name server: lists the name server authoritative for the delegated zone
• ‘PTR’ record: Pointer - provide a pointer to another location in namespace
• ‘CNAME’ record: canonical name - provide mapping between alias and real name
• ‘MX’ record: Mail Exchange - specified device name that handles email sent to domain
• ‘SOA’ record: start of authority - used to mark the start of DNS zone and provide info on it, i.e name of zone, master server name, etc.
• ‘TXT’ record - Text String: allow additional text associated with domain to be stored

Every computer has a piece of software called local DNS resolver that can lookup the domain name for any IP address. When the user types www.distributedsystemsblog.com, the resolver checks its cache and local host table file, i.e. /etc/hosts. Otherwise, it needs to perform DNS Name Resolution using either of 2 methods:

• Iterative query: The client queries the DNS server, the server responds back with the next level server and let the client to query itself.
• Recursive query: The client queries the DNS server, the server either responds with IP or sends request to other server on behalf of the client.

Example of DNS resolution process for “distributedsystemsblog.com”:

1. The resolver will first send a recursive query to the local DNS server - DNS servers that set up by ISP or organization. The local DNS server check its cache and returns the info if it found, probably with “non-authoritative answer” because the local DNS servers are not the authoritative nameservers of “distributedsystemsblog.com”.

2. Let’s say the local DNS server does not have the info, it sends an iterative request to root name server. Root server returns the name and address of the TLD server for the “.COM” domain. The local DNS server then generates an iterative request and sends it to “.COM” server.

3. Usually the TLD servers know the authoritative DNS server, it usually stores and serves a special the glue record that maps the IP address of authoritative DNS server for the delegated zone.

A glue record is served by a DNS server that’s not authoritative for the zone, to avoid circular dependency. For example, let’s say an authoritative nameserver ns1.example.com of “example.com” is inside the domain itself. If a computer tries to resolve “example.com”. It might need to resolve “ns1.example.com” which requires resolving “example.com” first, which presents a circular dependency. To break the dependency, the TLD server for “.COM” includes glue along with the delegation for “example.com”. The glue records are address records that provide IP addresses for “ns1.example.com”. The resolver then uses the IP to query the authoritative servers.

1. After a few round of iterative request, local server receives IP, caches it and returns to local resolver. Local resolver caches the info and give info to the browser. Browser sends HTTP request to the given IP address. If domain name enters was an alias, indicated by a CNAME record, it adds another step: we resolve alias to the canonical name and then resolve the canonical name

The Routing Process for DNS query

Recall that our computer knows the IP address of DNS server from DHCP. When the resolver sends DNS Request message, the string www.distributedsystemsblog.com is put in the DNS Question Section of DNS message with the following information in the Ethernet frame:

• Destination IP address: The DNS server’s IP address
• Destination MAC addresses: The gateway’s MAC address
• Source IP address: The computer’s IP address
• Source MAC address: The computer’s MAC address

Suppose the gateway receives the message from our computer, it will forward the data to a series of routers before it reaches the DNS server. When each router receives the packet message:

1. The layer-2 adapter of router will check to see whether the destination MAC address in the frame matches its own MAC address. If there is a match, the adapter extracts the enclosed datagram and passes the datagram up the protocol stack.
2. The router examines the IP header to determine destination. It uses the forwarding table (In layer 3 , it is also called routing table) to determine the next interface for the packet. It Also modifies the IP header, i.e. reduce TTL and send packet to output interface.
3. This interface then passes the datagram to its adapter, which encapsulates the datagram in a new frame and sends the frame into next subnet. It replaces the source MAC with its own MAC and replace destination MAC to the next router/gateway MAC address.

When the message finally reaches the DNS server, the server forms a DNS reply message containing the hostname-to-IP mapping, and places the DNS reply message in a UDP segment before sending back to the source. Our computer extracts the IP address of the server www.distributedsystemsblog.com from the DNS message and ready to contact www.distributedsystemsblog.com server.

HTTP

Our computer now can create the TCP socket and send the HTTP GET message to www.distributedsystemsblog.com. The HTTP server at www.distributedsystemsblog.com reads the HTTP GET message from the TCP socket, creates an HTTP response message, places the requested Web page content in the body of the HTTP response message, and sends the message also with the TCP socket.

Unfortunately, TCP and HTTP are too complicated to include in this post, instead we will discuss them in future posts.