10.2 A Closer Look
In the previous example, you manually interacted with a web server on the network with telnet, using the HTTP application layer protocol. Although you’d normally use a web browser to make this sort of connection, let’s take just one step up from telnet and use a command-line program that knows how to speak to the HTTP application layer. We’ll use the curl utility with a special option to record details about its communication:
$ curl --trace-ascii trace_file http://www.example.org/
Your distribution may not have the curl package preinstalled, but you should have no trouble installing it if necessary.
You’ll get a lot of HTML output. Ignore it (or redirect it to /dev/null) and instead look at the newly created file trace_file. If the connection was successful, the first part of the file should look something like the following, at the point where curl attempts to establish the TCP connection to the server:
== Info: Trying 93.184.216.34...
== Info: TCP_NODELAY set
== Info: Connected to www.example.org (93.184.216.34) port 80 (#0)
Everything you’ve seen so far happens in the transport layer or below. However, if this connection succeeds, curl then tries to send the request (the “header”); this is where the application layer starts:
❶ => Send header, 79 bytes (0x4f)
❷ 0000: GET / HTTP/1.1
0010: Host: www.example.org
0027: User-Agent: curl/7.58.0
0040: Accept: */*
004d:
Line ❶ is curl debugging output telling you what it will do next. The remaining lines show what curl sends to the server. The text in bold is what goes to the server; the hexadecimal numbers at the beginning are just debugging offsets curl adds to help you keep track of how much data was sent or received.
At ❷, you can see that curl starts by issuing a GET command to the server (as you did with telnet), followed by some extra information for the server and an empty line. Next, the server sends a reply, first with its own header, shown here in bold:
<= Recv header, 17 bytes (0x11)
0000: HTTP/1.1 200 OK
<= Recv header, 22 bytes (0x16)
0000: Accept-Ranges: bytes
<= Recv header, 12 bytes (0xc)
0000: Age: 17629
--snip--
Much like the previous output, the <= lines are debugging output, and 0000: precedes the lines of output to tell you the offsets (in curl, the header won’t count toward the offset; that’s why all of these lines begin with 0).
The header in the server’s reply can be fairly long, but at some point the server transitions from transmitting headers to sending the actual requested document, like this:
<= Recv header, 22 bytes (0x16)
0000: Content-Length: 1256
<= Recv header, 2 bytes (0x2)
❶ 0000:
<= Recv data, 1256 bytes (0x4e8)
0000: <!doctype html>.<html>.<head>. <title>Example Domain</title>.
0040: . <meta charset="utf-8" />. <meta http-equiv="Content-type
--snip--
This output also illustrates an important property of the application layer. Even though the debugging output says Recv header and Recv data, implying that those are two different kinds of messages from the server, there’s no difference in how curl talked to the operating system to retrieve the two messages, in how the operating system handled them, or in how the network handled the packets underneath. The difference is entirely within the user-space curl application. curl knew that until this point it had been getting headers, but when it received a blank line ❶, which signifies the end of headers in HTTP, it knew to interpret anything that followed as the requested document.
The same is true of the server sending this data. When sending the reply, the server’s operating system didn’t differentiate between header and document data; the distinctions happen inside the user-space server program.