CS 537 Class Notes

Lecture Notes for Client-Server Interfaces

19 February 2004 - Concurrency and Processor Performance

The Objectives

See how concurrency affects architecture.
See how concurrency affects performance.

The Environment

Server host.
- Sparc Ultra 10, 333 MHz, 128 Mbytes, Solaris 2.7.
Client hosts.
- Sparc Ultra 10, 270 MHz, 64 Mbytes, Solaris 2.8.
Misc.
- g++ ver. 2.95.2.

The Client

static void tcp_echo(
  char * host, char * service, 
  unsigned size, unsigned iters)

  char * str = new char[size]
  char * bfr = new char[size]

  for (unsigned i = 0; i < iters; i++)

    gettimeofday(&start, NULL)

    int s = tcp_connect(host, service)

    write(s, str, size)
    tcp_read(s, bfr, size)

    shutdown(s, 1)
    close(s)

    gettimeofday(&stop, NULL)

An Iterative Server

const unsigned buffer_size = 1024*16;
static char buffer[buffer_size];

static void echo(int s)

  int cc

  do
    cc = read(s, buffer, sizeof(buffer))
    write(s, buffer, cc)
  while cc > 0


int main(int argc, char * argv[]) 

  const int ms = passive_tcp(port, tcp_qlen)

  while true
    const int s = accept_tcp(ms)
    echo(s)
    close(s)

  return EXIT_SUCCESS

The Results

One client.

cs14 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 12 req/sec, 500.4 kbytes/sec.

Two clients.

cs14 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 6 req/sec, 267.6 kbytes/sec.

cs16 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 6 req/sec, 267.2 kbytes/sec.

Three clients.

cs14 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 4 req/sec, 181.1 kbytes/sec.

cs16 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 4 req/sec, 178.2 kbytes/sec.

cs20 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 4 req/sec, 178.7 kbytes/sec.

The Results Continued

Four clients.

cs14 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 3 req/sec, 137.7 kbytes/sec.

cs16 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 3 req/sec, 136.8 kbytes/sec.

cs20 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 3 req/sec, 134.5 kbytes/sec.

cs18 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 3 req/sec, 136.8 kbytes/sec.

Comments

The network is psychotic.
- The variance renders all data meaningless.
Just using shutdown() doesn't work.
- shutdown() doesn't recover resources; close() does.

A Forking Server

const unsigned buffer_size = 1024*16;
static char buffer[buffer_size];

static void echo(int s)
  int cc
  while cc = read(s, buffer, sizeof(buffer)) > 0
    write(s, buffer, cc)
  close(s)


int main(int argc, char * argv[])

  const int ms = passive_tcp(port, tcp_qlen)

  if signal(SIGCHLD, SIG_IGN) == SIG_ERR
    errexit("signal(sigchld) failed:")

  while true
    int ss = accept_tcp(ms)
    switch fork()
      case 0:  close(ms)
               echo(ss)
               return EXIT_SUCCESS
      default: close(ss)
               break

  return EXIT_SUCCESS

The Results

One client.

cs14 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 11 req/sec, 461.5 kbytes/sec.

Two clients.

cs14 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 6 req/sec, 253.2 kbytes/sec.

cs16 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 6 req/sec, 257.2 kbytes/sec.

Three clients.

cs14 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 4 req/sec, 170.5 kbytes/sec.

cs16 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 4 req/sec, 165.8 kbytes/sec.

cs20 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 4 req/sec, 166.8 kbytes/sec.

The Results Continued

Four clients.

cs14 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 3 req/sec, 123.4 kbytes/sec.

cs16 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 3 req/sec, 123.0 kbytes/sec.

cs20 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 3 req/sec, 123.6 kbytes/sec.

cs17 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 3 req/sec, 124.5 kbytes/sec.

Comments

No zombies.

if (signal(SIGCHLD, SIG_IGN) == SIG_ERR)
  errexit("signal(sigchld) failed:");

Not POSIX compliant.

Don't use signal(), use sigset().
1. signal() has unpredictable behavior.

A Threaded Server

static void * echo(void * arg)

  const int s = (int) arg
  const unsigned buffer_size = 1024*16
  char * buffer = new char[buffer_size]
  int cc

  while cc = read(s, buffer, buffer_size) > 0
    write(s, buffer, cc) < 0)

  close(s)
  delete [] buffer

  return 0

#define create_pthread(_t, _a, _f, _v)\
  do { const int e = pthread_create(&(_t), &(_a),\
                     reinterpret_cast(_f),\
                     reinterpret_cast(_v))\
       if (e) errexit("%d = pthread_create(%s) failed:", e, #_f)\
     } while (false)

int main(int argc, char * argv[])

  const int ms = passive_tcp(opts['s'], tcp_qlen)

  pthread_t th
  pthread_attr_t ta
  (void) pthread_attr_init(&ta)
  (void) pthread_attr_setdetachstate(&ta, PTHREAD_CREATE_DETACHED)

  while true
    create_pthread(th, ta, echo, accept_tcp(ms))

  return EXIT_SUCCESS

The Results

One client.

cs14 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 12 req/sec, 493.1 kbytes/sec.

Two clients.

cs14 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 6 req/sec, 269.1 kbytes/sec.

cs16 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 6 req/sec, 269.2 kbytes/sec.

Three clients.

cs14 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 4 req/sec, 171.1 kbytes/sec.

cs16 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 4 req/sec, 169.1 kbytes/sec.

cs20 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 4 req/sec, 171.5 kbytes/sec.

The Results Continued

Four clients.

cs14 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 3 req/sec, 122.7 kbytes/sec.

cs16 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 3 req/sec, 123.2 kbytes/sec.

cs20 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 3 req/sec, 122.7 kbytes/sec.

cs17 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 3 req/sec, 122.8 kbytes/sec.

Comments

Oops
- The buffer should not be a global resource.
  1. Unmediated access to a shared resource.
- Put the buffer on a thread's stack carefully.
  1. Too big a buffer may cause thread stack overflow.

An Event-Driven Server

const unsigned buffer_size = 1024*16
static char buffer[buffer_size]

static int echo(int s)

  const int cc = read(s, buffer, sizeof(buffer))
  write(s, buffer, cc)

  return cc


int main(int argc, char * argv[])

  const int ms = passive_tcp(opts['s'], tcp_qlen)

  const int nfds = getdtablesize()
  fd_set rfds, afds

  while true
    memcpy(&rfds, &afds, sizeof(rfds))

    const int e =
      select(nfds, &rfds, NULL, NULL, (struct timeval *) 0)
    if (e < 0) errexit("select() error:")

    if FD_ISSET(ms, &rfds)
      int s = accept_tcp(ms)
      FD_SET(s, &afds)

    for int fd = 0; fd < nfds; fd++
      if (fd != ms) and (FD_ISSET(fd, &rfds))
        if !echo(fd)
          (void) close(fd)
          FD_CLR(fd, &afds)

  return EXIT_SUCCESS;

The Results

One client.

cs14 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 12 req/sec, 518.9 kbytes/sec.

Two clients.

cs14 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 7 req/sec, 281.9 kbytes/sec.

cs16 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 6 req/sec, 260.4 kbytes/sec.

Three clients.

cs14 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 4 req/sec, 176.6 kbytes/sec.

cs16 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 4 req/sec, 172.2 kbytes/sec.

cs20 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 4 req/sec, 172.1 kbytes/sec.

The Results Continued

Four clients.

cs14 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 3 req/sec, 125.4 kbytes/sec.

cs16 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 3 req/sec, 124.0 kbytes/sec.

cs20 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 3 req/sec, 123.9 kbytes/sec.

cs17 ./c -h clayton -s 1956 -b 40000 -i 300
300 reqs, 40000 size, 3 req/sec, 124.2 kbytes/sec.

Comments

Oops.

if (FD_ISSET(ms, &rfds))
  FD_SET(accept_tcp(ms), &afds);

This page last modified on 18 February 2004.