This document follows along with Chapter 12, the concurrent programming chapter, of Computer Systems: A Programmers Perspective.
It’s basically a literate implementation of some of the examples, along with the notes I found valuable as someone new to C.
This first example is actually from Chapter 11: Network Programming. It’s a non-concurrent server that we’ll use throughout Chapter 12 by using different methods of making it concurrent.
The server will use accept and when a connection is made it will pass the newly connected client socket file descriptor to an “echo” function.
The echo function uses CS:APP’s Robust I/O utilities to wrap the file descriptor in a struct that provides a buffered reading.
It then repeatedly reads a line from the connected socket file descripter and writes that line back to the file descriptor, until it reads an interrupt or EOF.
#include "csapp.h"
void echo(int connfd);
void main(int argc, char** argv)
{
@{handle invalid args}
int listenfd, connfd;
socklen_t clientlen;
@{declare a struct to hold the client address}
char client_hostname[MAXLINE], client_port[MAXLINE];
@{listen on port passed as argv[1]}
@{indefinitely loop, accepting connections and echoing}
exit(0);
}
We could use struct sockaddr_in.
struct sockaddr_in {
sa_family_t sin_family; /* address family: AF_INET */
in_port_t sin_port; /* port in network byte order */
struct in_addr sin_addr; /* internet address */
};
But see Beej’s Guide to Network Programming or the man 7 socket page for reasons to use sockaddr_storage instead.
Beej:
Last but not least, here is another simple structure, struct sockaddr_storage that is designed to be
large enough to hold both IPv4 and IPv6 structures. See, for some calls, sometimes you don’t know in
advance if it’s going to fill out your struct sockaddr with an IPv4 or IPv6 address. So you pass in this
parallel structure, very similar to struct sockaddr except larger, and then cast it to the type you need:
```
struct sockaddr_storage {
sa_family_t ss_family; // address family
// all this is padding, implementation specific, ignore it:
char __ss_pad1[_SS_PAD1SIZE];
int64_t __ss_align;
char __ss_pad2[_SS_PAD2SIZE];
};
```
What’s important is that you can see the address family in the ss_family field—check this to see if it’s
AF_INET or AF_INET6 (for IPv4 or IPv6). Then you can cast it to a struct sockaddr_in or struct
sockaddr_in6 if you wanna.
Socket manpage:
Socket address structures
Each socket domain has its own format for socket addresses, with a
domain-specific address structure. Each of these structures begins with
an integer "family" field (typed as sa_family_t) that indicates the type
of the address structure. This allows the various system calls (e.g.,
connect(2), bind(2), accept(2), get‐ sockname(2), getpeername(2)), which
are generic to all socket domains, to determine the domain of a
particular socket address.
To allow any type of socket address to be passed to interfaces in the
sockets API, the type struct sockaddr is defined. The purpose of this
type is purely to allow casting of domain-specific socket address types
to a "generic" type, so as to avoid compiler warnings about type
mismatches in calls to the sockets API.
In addition, the sockets API provides the data type struct
sockaddr_storage. This type is suitable to accommodate all supported
domain-specific socket address structures; it is large enough and is
aligned properly. (In particular, it is large enough to hold IPv6 socket
addresses.) The structure includes the following field, which can be
used to identify the type of socket address actually stored in the
structure:
sa_family_t ss_family;
The sockaddr_storage structure is useful in programs that must handle
socket addresses in a generic way (e.g., programs that must deal with
both IPv4 and IPv6 socket addresses).
See csapp.c for implementation.
listenfd = Open_listenfd(argv[1]);
printf("fd: %d\n", (int)listenfd);
while (1) {
@{accept client connections}
@{print connecting client's hostname and port}
@{present an echo service to the client}
}
Used by 1
Accept is just a tiny wrapper around the accept system call.
It awaits a connection on listenfd.
When a connection arrives, it opens a new socket to communicate with it, sets clientaddr to the address of the connecting peer and clientlen to the address’s actual length, and returns the new socket’s descriptor or -1 for errors.
SA is just a shortcut to struct sockaddr.
clientlen = sizeof(struct sockaddr_storage);
connfd = Accept(listenfd, (SA*)&clientaddr, &clientlen);
Used by 1
NAME
getnameinfo - address-to-name translation in protocol-independent manner
Getnameinfo((SA*)&clientaddr, clientlen, client_hostname, MAXLINE, client_port, MAXLINE, 0);
printf("Connected to (%s, %s)\n", client_hostname, client_port);
Used by 1
Close is just a wrapper around the close system call.
The CS:APP has a habit of making tiny wrappers around system calls. The wrappers basically just condition on errors to log and exit.
The Rio_ functions in echo.c come from CS:APP’s “Robust I/O” package from section 10.4.
From the book:
In some situations, `read` and `write` transfer fewer bytes than the application requests. Such _short counts_ do _not_ indicate an error. They occur for a number of reasons.
Those reasons include:
Of all the errors that read can return, the implementation explicitly handles EINTR by retrying.
That seems notablej.
#include "csapp.h"
void echo(int connfd)
{
size_t n;
char buf[MAXLINE];
rio_t rio;
Rio_readinitb(&rio, connfd);
while((n = Rio_readlineb(&rio, buf, MAXLINE)) != 0) {
printf("server received %d bytes\n", (int)n);
Rio_writen(connfd, buf, n);
}
}
➜ ./echo 9999
Connected to (localhost, 44788)
server received 7 bytes
➜ socat - TCP:localhost:9999
foobar
foobar
if (argc != 2) {
fprintf(stderr, "usage: %s <port>\n", argv[0]);
exit(0);
}
#include "csapp.h"
void echo(int connfd);
@{signal handler for child stopped/terminated}
int main(int argc, char** argv)
{
@{handle invalid args}
@{setup signal handler}
@{declare listen and connection file descriptors}
@{declare variable to hold sockaddr for client}
@{listen on port passed as argv[1]}
@{repeatedly accept connections and serve echo utility}
}
This is one of the things that differs from the non-concurrent echo server.
The handler itself is defined in Handling child stopped or terminated signals.
Not all Unix systems handle signals the same way.
Some systems restore the action for signal k to its default after signal k has been caught by a handler.
In those systems, it’s the handler’s responsibility to reinstall itself.
sigaction is a POSIX function that unifies this above behavior.
CS:APP provides the Signal function which is a wrapper around sigaction.
Signal function.System calls can be interrupted.
System calls such as read, wait, and accept that can potentially block the process for a long period of time are called slow system calls. On some older versions of Unix, slow system calls that are interrupted when a handler catches a signal do not resume when the signal handler returns but instead return immediately to the user with an error condition and errno set to EINTR. On tehse systems, programmers must include code that manually restarts interrupted system calls.
sigaction and SA_RESTARTman 2 sigaction
DESCRIPTION
The sigaction() system call is used to change the action taken by a process on receipt of a specific signal. (See signal(7) for an overview of signals.)
...
The sigaction structure is defined as something like:
struct sigaction {
void (*sa_handler)(int);
void (*sa_sigaction)(int, siginfo_t *, void *);
sigset_t sa_mask;
int sa_flags;
void (*sa_restorer)(void);
};
...
SA_RESTART This flag affects the behavior of interruptible functions;
that is, those specified to fail with errno set to [EINTR]. If
set, and a function specified as interruptible is
interrupted by this signal, the function shall restart and
shall not fail with [EINTR] unless otherwise specified. If an
interruptible function which uses a timeout is restarted, the
duration of the timeout following the restart is set to an
unspecified value that does not exceed the original timeout
value. If the flag is not set, interruptible functions
interrupted by this signal shall fail with errno set to
[EINTR].
while (1) {
clientlen = sizeof(struct sockaddr_storage);
connfd = Accept(listenfd, (SA *) &clientaddr, &clientlen);
if (Fork() == 0) {
Close(listenfd);
echo(connfd);
Close(connfd);
exit(0);
}
close(connfd);
}
Used by 1
socklen_t clientlen;
struct sockaddr_storage clientaddr;
Used by 1
void sigchld_handler(int sig)
{
while (waitpid(-1, 0, WNOHANG) > 0)
;
return;
}
Used by 1
We’re using waitpid here rather than wait because the waitpid function accepts an options argument as a 3rd parameter.
By passing -1 as the first argument (the PID) it is equivalent to wait in that the calling thread blocks until information generated by any child process termination is made available to the thread.
From man 7 signal:
SIGCHLD P1990 Ign Child stopped or terminated
The WNOHANG option is described in this StackOverflow thread.
If you pass -1 and WNOHANG, waitpid() will check if any zombie-children exist. If yes, one of them is reaped and its exit status returned. If not, either 0 is returned (if unterminated children exist) or -1 is returned (if not) and ERRNO is set to ECHILD (No child processes). This is useful if you want to find out if any of your children recently died without having to wait for one of them to die. It's pretty useful in this regard.
And the man 3 wait manpage:
WNOHANG The waitpid() function shall not suspend execution of the calling thread if status is not immediately available for one of the child processes speci‐
fied by pid.
Suppose you are asked to write an echo server that can also respond to interactive commands that the user types to standard input.
For example:
./echoserver 8888
"Now listening for client connections on 8888."
"Repl commands: help, status, stats, quit"
"> status"
"Idle: listening on port 8888"
"> stats"
"17 client connections processed"
So, the server has to respond to two independent I/O events: (1) a network client making a connection request, and (2) a user typing a command line at a keyboard. Which event do we wait for first? Neither option is ideal.
I/O multiplexing is one solution to this.
It uses the select function to ask the kernel to suspend the process, returning control to the application only after one or more I/O events have ocurred, as in the following examples:
See man 2 select for more details.
The “set” mentioned above is just a bit vector of file descriptors.
#include "csapp.h"
void echo(int connfd);
void command(void);
int main(int argc, char **argv)
{
@{handle invalid args}
int listenfd, connfd;
socklen_t clientlen;
struct sockaddr_storage clientaddr;
fd_set read_set, ready_set;
listenfd = Open_listenfd(argv[1]);
@{initialize the read_set for the `select` call}
@{repeatedly select from the ready_set and process input}
}
I found the definition of fd_set in /usr/include/sys.
/* fd_set for select and pselect. */
typedef struct
{
/* XPG4.2 requires this member name. Otherwise avoid the name
from the global namespace. */
#ifdef __USE_XOPEN
__fd_mask fds_bits[__FD_SETSIZE / __NFDBITS];
# define __FDS_BITS(set) ((set)->fds_bits)
#else
__fd_mask __fds_bits[__FD_SETSIZE / __NFDBITS];
# define __FDS_BITS(set) ((set)->__fds_bits)
#endif
} fd_set;
And __FD_SETSIZE is defined as:
/* Number of descriptors that can fit in an `fd_set'. */
#define __FD_SETSIZE 1024
FD_ZERO(&read_set);
FD_SET(STDIN_FILENO, &read_set);
FD_SET(listenfd, &read_set);
Used by 1
The first argument to Select is nfds, described as:
nfds This argument should be set to the highest-numbered file descriptor in any of the three sets, plus 1. The indicated file descriptors in each set are
checked, up to this limit (but see BUGS).