HTTP uses 8-bit ISO Latin 1 (ASCII with Western Europe Extensions). This means the data we will be transmitting and receiving will be ASCII text and perhaps some 8-bit binary data. HTTP uses TCP on port 80 for servers. The basic sequence for HTTP v1.0 is the following:
SP = Space
CR/LF = Carriage Return/Line Feed
request-line = request SP request-URI SP HTTP-version CR/LF
request = GET or HEAD or POST
Example: The following is a valid request-line to retrieve the /my_dir/home_page.html file from a server:
GET /my_dir/home_page.html HTTP/1.0
status-line = HTTP-version SP response-code SP response-phrase CR/LF
response-code = 3 digit number
response-phrase = human readable response for response code
The xx codes below indicate the categories of responses. The actual digits represent defined responses.
| Resonse Code | Description |
|---|---|
| 1xx | Informational (not used) |
| 2xx | Success |
| 200 | Success (OK) |
| 204 | Success (No Content to return) |
| 3xx | Redirection |
| 301 | Redirect (New permanent URL) |
| 302 | Redirect (New temporary URL) |
| 304 | Document not modified |
| 4xx | Client Error |
| 400 | Bad Request |
| 404 | Not Found |
| 5xx | Server Error |
| 501 | Not Implemented |
Headers appear in both requests and responses. Some headers are used for logging information. Some describe the body of the data. And some modify processing. The table below lists the header names and whether they are used in requests and/or responses and if they describe the body or not.
| Header Name | Comments |
|---|---|
| Allow | Describes Body |
| Authorization | Requests |
| Content-Encoding | Describes Body |
| Content-Length | Describes Body |
| Content-Type | Describes Body |
| Date | Requests and Repsonses |
| Expires | Describes Body |
| From | Requests |
| If-Modified-Since | Requests |
| Last-Modified | Describes Body |
| Location | Responses |
| MIME-Version | Requests and Responses |
| Pragma | Requests and Responses |
| Referer | Requests |
| Server | Responses |
| User-Agent | Requests |
| WWW-Authenticate | Responses |
request-line = GET SP request-URI CR/LF
A HTTP 1.0 or 1.1 server is required to understand this older request style and to generate a simple response. A simple response has no status line or any headers. It is just a body.
The practice of keeping older protocol compatibility in new protocols is very common with application protocols. However, for lower level protocols it becomes more rare.
This presents the overview and basics of the HTTP protocol. A more thorough description is present in the HTTP protocol specification itself.
/-----------\ +----------+ +--------+
| User at a |<-->| Terminal |<-->| Telnet |
| Terminal | | Driver | | Client |
\-----------/ +----------+ +--------+
| TCP/IP |<--+
The Terminal Driver, TCP/IP, +--------+ |
and Psuedo Terminal Driver are |
normally part of the OS. +--------+ |
| TCP/IP |<--+
+-----------+ +----------+ +--------+
| Login |<-->| Psuedo |<-->| Telnet |
| Shell | | Terminal | | Server |
+-----------+ | Driver | +--------+
+----------+
| FTP Client | FTP Server |
|---|---|
| Performs passive open on ephemeral port | |
| Sends port to FTP server | Performs active open to FTP clients ephemeral port. Always uses port 20 for the local port number. |
As you can see, for the data connection, the roles of client and server seem to reverse themselves.
/-----------\ +-------+ + - - - - - - - + +----------------+
| User at a | -->| User | -->| Queue of mail | -->| Message |
| Terminal | | Agent | | to be sent | | Trasnfer Agent |
\-----------/ +-------+ + - - - - - - - + +----------------+
Mail Sender | TCP/IP |<--+
+----------------+ |
|
+----------------+ |
Mail Receiver | TCP/IP |<--+
/-----------\ +-------+ + - - - - - - - + +----------------+ Port 25
| User at a |<-- | User |<-- |User mailboxes |<-- | Message |
| Terminal | | Agent | | (maildrops) | | Trasnfer Agent |
\-----------/ +-------+ + - - - - - - - + +----------------+
POP v3 is specified in RFC 1939. It is a simple command line protocol that uses 7-bit ASCII on top of TCP, port 110. POP is primarily designed to dynamically access a maildrop on a server host in a useful fashion. The dynamic nature is important so that periodic connectivity or resource limits do not impair the usage of the system. POP is designed for simple manipulation of email (normally download mail and delete). More complex manipulations, such as mail folders, etc. is the domain of IMAP (Internet Message Access Protocol) (RFC 2060). POP and IMAP do not cover the process of sending mail.
A POP connection is persistent until the QUIT command is sent by the client.
Each command is case insensitive, terminated by a CRLF, and each command
generates a response.
The response begins with one of two status indicators +OK to
indicate success or -ERR to indicate an error. Multi-line
responses are terminated by a CRLF.CRLF combination. If a response
line starts with ".", another "." is pre-pended before being sent by the
server. (This is also done in NNTP).
A POP server maintains some state information for each connection. This is primarily necessary for access control to maildrops and for some level of fault tolerance. The three states are:
A list of the POP commands and a short description follow. Each command
contains the command name (in capitals) and its arguments. Commands
with *'s in the desciptions are optional commands and may not be
available on some implementations. An argument in [] indicates an
optional argument.
| POP Command | Description |
|---|---|
| USER name | Specifies the name of the maildrop to access |
| PASS string | Specifies the password used for the maildrop |
| APOP name digest | (* optional) Replaces both USER and PASS with a single command for identification and authentication. It uses MD5 (RFC 1321) for generating a 128-bit integer value that is cryptographically secure from the password. |
| QUIT | Close maildrop, go into UPDATE state, close connection, and perform deletions. |
| STAT [msg#] | List status of maildrop or specific message |
| LIST [msg#] | List messages in maildrop (1 per line) |
| RETR msg# | Send the contents of the specified message |
| DELE msg# | Mark a specific message for deletion after QUIT |
| NOOP | Do nothing, but return a response |
| RSET | Unmark all messages marked for deletion |
| TOP msg# #lines | (* optional) Return mail header and some number of lines of the body of the specified message. |
| UIDL [msg#] | (* optional) Generate unique identifiers for all messages (or a specific message) in the maildrop and return the identifier. The identifier can be used in place of msg# for any call. |
NNTP uses commands much like POP and SMTP. Each command is terminated by CRLF. Multi-line responses and commands are terminated by the CRLF.CRLF as described with POP. NNTP responses begin with a response code value (which is formatted much like HTTPs is). Most news servers use the IP address of the connecting agent to determine two things: (1) who may connect, and (2) which news service to use, either a news server if another news server connects or a more simplified news reader server if a news client connects.
NNTP supports a number of commands. However, several servers support more commands that are not standard. A list of the more common commands follows. This is only a small subset of the commands available.
| NNTP Command | Description |
|---|---|
| HELP | List the set of supported commands |
| LIST | List newsgroups available |
| GROUP name | Set the active newsgroup that will be used for subsequent commands and display the numebr of articles in group and starting and ending article numbers. |
| HEAD art# | Display headers for article number in the active group |
| BODY art# | Display body of the article |
| QUIT | Disconnect |
| POST | Post an article |
Some of the more common extended commands are XHDR and XOVER which provide condensed information about the headers of articles.
In X terminology, client and server are a bit differently defined. An X server is the entity that controls the keyboard, mouse, and display on the local machine a user is physically on. An X client is an application running on some host (remote or local) that displays on the local hosts display. This may appear backwards unless you think of a server as providing a service and then it should be somewhat clearer.
The protocol that X uses is 8-bit data that has a format that varies based on the operation being performed (such as drawing a line or turing individual pixels on or off). There are about 150 or more message formats for X. So, the protocol is fairly complex.
X was primarily designed for UNIX, and local X clients to an X server use
UNIX domain sockets for communication. The path is based on the display
number of the X server and is usually /tmp/.X11-unix/Xn.
Where n is the display number. (A machine may have multiple displays).
For remote X clients, the X server uses port 6000+n. Where n is the
display number.
X client applications do a variety of jobs. One important X client runs locally to the X server and is the window manager. The window managers task is to manage window layout.
The lookup operation of this mapping begins on the host side. Each
host has a "resolver". This resolver may use multiple name services, such
as DNS, local file mappings, NIS, NIS+, etc. The resolver picks the right
mechanism and may even try multiple mechanisms. In C, the interface
to the resolver is the gethostbyname() and
gethostbyaddr() functions.
The DNS name space provides the abstractions needed for architecture and control of the database. The name space is a hierarchical tree like the UNIX file system. The root is unnamed. Its direct children are the Top Level Domains (TLDs). Each node of the tree has a label of up to 63 characters in length. They are case insensitive. The TLDs are the domains like: arpa, com, edu, gov, mil, us, uk, as, etc.
A domain name is either a Fully Qualified Domain Name (FQDN) or not. An FQDN ends in a ".". e.g. csee.wvu.edu. is a FQDN. FQDNs are also called absolute domain names. Without a FQDN, it might need to be completed by the resolver. This is usually done by appending a domain to the name before it is sent to the DNS. For example, "naur" is not a FQDN, so the resolver might attach a domain to it that it is configured with, such as "csee.wvu.edu". So, the FQDN it asks DNS for might be "naur.csee.wvu.edu.".
This covers the name-to-address mapping, but what about the address-to-name mapping. The address-to-name (the process of taking an IP address and getting a mapping to a hostname) is done by making a special zone and using labels to indicate the mapping. For an IP address of 157.182.194.28 to be looked up, it must first be converted to a hostname under the TLD arpa. This is done in the reverse order of the IP address. So, for the address we just mentioned, we get 28.194.182.157.in-addr.arpa. Looking this name up, we get the hostname and entry.
No single entity manages every label in DNS. The InterNIC maintains the TLDs and delegates to others for specific zones (subtrees). A zone is required to maintain a primary name server and at least one secondary name server. A primary name server keeps its info on local disk storage. While, a secondary name server gets its info from the primary name server periodically (usually every 3 hrs or so). The task of obtaining info from the primary is called a zone transfer.
A resolver contacts a local DNS server (primary or secondary). If the name server contains info for the mapping, it returns it. If not, it must contact another name server. Usually, it contacts the root name servers maintained by InterNIC and asks what the servers are that it needs for the TLD in the mapping. Then it contacts that server and asks what the server is for the next zone in the mapping. This happens until it fails or it finds a server which has the information.
At each step of a lookup above, the DNS server caches the results. So, that if it needs to perform the search later, it has some of the information on which server to go to next in the search and does not have to contact the root servers or other higher level servers as often. Cached entries flush out after a period of time and cause the DNS server to get new information from some servers when new requests for lookups come in.
The messages that DNS servers and resolvers exchange are either UDP or TCP. Both use port 53. The data is 8-bit and formatted. UDP is used for most operations. However, if a response is over 512 bytes or a zone transfer is performed, TCP is used.
RPC hides all the network code in client "stubs" and server "skeletons" that are generated automatically by specifying twhat their interfaces should be.
RPC raises a lot of issues. Some are harder to work out than others. One of these issues is parameter passing and data representation. Both sides of a communication must agree on how data is respresented. In RPC, this is done by XDR (External Data Represetnation - RFC 1014). Once both sides agree on the respresentation of data, it must work out how to pass parameters. Passing parameters by value is easy. Passing parameters by reference is much harder. (Imagine how a 10 MB database would be passed to another machine). Another issue in RPC is binding a server. A naming service must exist so that you don't have to hardcode IP address or hostnames and port numbers into procedure calls. This is done using a "portmapper" application that identifies services by program name and program version. Under UNIX, this is usually the rpcbind daemon.
RPC uses a choice of transport protocols. These are either connectionless (like UDP) or connection-oriented (like TCP). Connectionless protocols must provide reliable delivery in the stub code, but they tend to offer higher performance if the procedure is a simple operation. Connection-oriented protocols provide reliable delivery, but at a price of increased overhead thus reducing performance.
Suns RPC v2 is specified in RFC 1057.
NFS v2 is specified in RFC 1094. Typically, NFS uses UDP. However, more recent versions can also use TCP. TCP provides better performance over WANs than UDP in this case.
An NFS server provides a set of procedures. These are operations such as getting/setting file attributes, retrieving status of a file system, looking up a file, reading part of a file, writing part of file, creating a file, removing a file, renaming a file, making directories, removing directories, etc.
An NFS server is stateless. It does not keep track of which client accesses which files. This information may be tracked, but it is NOT required for correct NFS operation. This simplifies server crash receovery. In that case, a client keeps requesting the operation until the server responds. Each request will be independent and no sort of rollback needs to be performed to resynchronize. The use of RPC also explains why most NFS clients totally hang up when a server crashes. This is because the client usually can't continue operation until the procedure completes and a response is sent by the server.
TFTP uses UDP on top of IP and is pecified in RFC 1350. TFTP messages come in five varieties. These types are given below along with their formats.
General Format:
Bytes: 2
+------+----------------------------+
|opcode| contents depend on opcode |
+------+----------------------------+
Opcode of Read ReQuest (RRQ) is 1 and Write ReQuest (WRQ) is 2:
Bytes: 2 Var 1 Var 1
+------+----------+-+------+-+
|opcode| filename |0| mode |0|
+------+----------+-+------+-+
Opcode of Data is 3:
Bytes: 2 2 0-512
+------+---------+-------+
|opcode| block # | Data |
+------+---------+-------+
Opcode of ACK is 4:
Bytes: 2 2
+------+---------+
|opcode| block # |
+------+---------+
Opcode of Error is 5:
Bytes: 2 2 Var 1
+------+---------+---------------+-+
|opcode| error # | error message |0|
+------+---------+---------------+-+
The defined error numbers are: