Customizable Devices for Concurrency and Communication

Raman Kannan

kannan@moncol.monmouth.edu

DNA Laboratory

Monmouth University

Abstract

Programming with threads -- an abstraction for concurrency -- and sockets -- an abstraction for inter-process communication -- is anything but simple. Both conceptually { timing errors, race conditions, synchronization } and programmatically { thread attributes, socket options, and notational nuances } programming with these devices is highly distractive in nature for application developers. Combining these two abstractions is even more complex. More often than not engineers trade performance for simplicity and choose not to use these high performance devices solely due to the complexity involved. It is in this context that frameworks become highly relevant. As an illustration, in this article we present the PAcceptor and SConnector frameworks -- which encapsulate these system level abstractions, hiding many of the difficult conceptual aspects-- such that these high performance devices may be leveraged sans their associated conceptual barriers.

Parallel Computing Halted

"Parallel Computing Still Not Ready for the Mainstream", so laments Domenico Talia in a recent Technical Opinion column in CACM [1]. As well articulated in the same article is that the reason is not due to the lack of parallel computing infrastructures. Perhaps parallelism and concurrency are foreign to us? To the contrary parallelism and concurrency are quite innate and we thrive based on our ability to solve problems concurrently. No one ever prepares a dinner --cooking entrees sequentially--. We do not come across traffic jams around Chinese buffet tables. In real life we naturally devise concurrent solutions while adeptly managing synchronization and asynchronous interrupts. What then is the reason for the lack of use of these powerful technologies? Perhaps it is the curse of von Neumann architecture but that is a topic of discussion of its own merit for yet another forum. In the interim, we posit that it is the combinatorial explosion of the conceptual difficulty involving complex application scenarios and these abstract albeit powerful devices. Simplicity and maintainability is chosen over overwhelming complexity and sigh --better performance--.

In this article we illustrate step by step -- from conceptualization to use -- how frameworks can be effectively used to abstract the complexities associated with threads and sockets and promote the use of these high performance devices.

Frameworks to the Rescue

In simple terms a framework captures the structure of things to come. Be it software artifacts or artifacts in other domain. There are numerous definitions of framework in the software domain. In [2] Wolfgang defines frameworks as "reusable semifinished architectures for various application domains" and architectures to be "several single components and the appropriate glue between them." Rogers[3] defines a framework as a partially completed software application that is intended to be customized to completion. "A framework is a class library that captures patterns of interaction between objects. A framework consists of a suite of concrete and abstract classes, explicitly designed to be used together. Applications are developed from a framework by competing the abstract classes". This definition of framework [a partial application that is completed by the developer or integrator] is also referred to as "application frameworks" [4]. As opposed to "integration frameworks" which refers to an infrastructure that is independent of application functionality [4]. Rogers differentiates a pattern from a framework in that patterns are prescriptions to a particular problem (need not include software) whereas frameworks are partial implementations to a problem. In other words frameworks are implementations in a particular language and may employ one or more patterns whereas patterns are problem specific but language independent prescriptions which may include illustrative implementations. Regardless frameworks are anything but new. For example the "sort" utility in Unix is a framework wherein users can sort any kind of data by merely providing an appropriate predicate that the Unix sorting engine uses to compare members in the set of data elements to be sorted. Window Managers and RPC programming environments are other popular frameworks that have been in use for many years. Now we introduce the problem domain and then the framework based solution encapsulating these services.

Acceptors, Connectors, Couriers and Threads

Rarely does one ever has to program threads and sockets directly anymore. Many useful reusable class libraries are available both in the public domain (ACE) and in the commercial market (OSPACE). There are several others and in our framework the system services for inter-process communication using sockets is achieved using three different classes. Acceptors are abstractions to listen to incoming connection requests. Connector objects initiate connect requests to distant Acceptor objects. Couriers objects are created --on both the Acceptor and the Connector side-- as a result of connection establishment. Couriers may then be used to transport arbitrary data objects. Details of implementation and rationale for this particular division of labor are presented in [5][6]. Threads and Mutex objects encapsulate the posix pthread library to create, manage and synchronize threads. These are essentially object oriented wrappers or convenient interfaces to the underlying system services.

Accepting in Parallel (PAcceptor)

As per the object decomposition above the sole responsibility of an Acceptor is to accept incoming connection request and create a Courier upon successful connection. Inherent in this scheme is that the socket abstraction facilitates two different streams which are independent. As a consequence when needed the two independent functions -- (1) transporting data stream (Courier) and (2) accepting incoming connection request (Acceptor) -- may be carried out concurrently.

Concurrency, TaskPile and ThreadPool

Venerable internet services such as ftp, telnet and several long duration services do indeed exploit this concurrency utilizing the Unix process creational service fork [7]. However, forks are expensive to create. Furthermore, sharing data between siblings requires other system services. Threads have been introduced to satisfy these two requirements, which are light weight with their own protection domain but within a process. Thus, threads within a process can share without much ado when needed using simple mutual exclusion primitives such as mutex (semaphores) and condition variables. A TaskPile is a shared work queue which allows clients to queue a job and dequeue jobs [8]. The implementation strategy is shown in Figure 1. A ThreadPool is an abstraction of a pool of threads to execute a common function [9]. We present ThreadPool implementation in Figure 2.

Transporting in Parallel (SConnector)

The function of a Courier is to transport data objects and the potential for concurrency exists under two different scenarios: (1) Consider a pipeline scenario where multiple worker threads in an upstream stage pass data to a single downstream stage processor using a Courier. The Courier then has to be shared and the upstream threads cannot write to the Courier simultaneously; and (2) Consider a Publisher/Subscriber Scenario [6] where data can be simultaneously published to all the subscribers.

Customizing PAcceptor and SConnector

PAcceptor and the SConnector implement a facade pattern so that clients do not have to contend directly with the interface of three different abstractions: Connector/Acceptor (communication device) or the ThreadPool/TaskPile (concurrency device). In Figure 3 we present how the Process member function is implemented. Essential services offered by these lower level abstractions are available to the client as shown in Fig. 4.

In Figure 5 we present some typical implementation of the applications using the PAcceptor/SConnector framework. Most importantly note that the application developer is oblivious to the intricacies of the communication or the concurrency devices. In Figure 6 we present an interaction diagram for processing incoming messages. In Figure 7 we present a Booch diagram for the classes participating within the frameworks.

Summary

In this article we have presented step by step the rationale and the implementation of a framework for realizing highly customizable devices for concurrency and communication. A family of data feed handler systems [10] have been realized using the framework presented here. PAcceptor/SConnector is a classic example of a calling framework in which applications register a function and the framework invokes the function when needed. We have demonstrated: (1) that frameworks are quite powerful tools to encapsulate and abstract the complexities of a well understood domain; and (2) that frameworks enable even naive application developers to exploit complex services without ever having to develop mastery over the encapsulated domains. Coupled with object orientation and using object oriented features (patterns, class hierarchies and delegation) frameworks can further lessen the cognitive effort required to design, implement and maintain complex real world applications and perhaps expedite the use of promising but complex technologies such as parallel computing with threads.

Acknowledgments

This chapter is a report of an ongoing project at Asset Enterprise Technologies Corporation, New Jersey since 1994 in contract to major multinational financial powerhouses. The PAcceptor and PConnector are in use at these financial corporations. Without the time, support and sharp technical and business acumen of the following dedicated professionals none of this would have been possible: William Sweeney, Byron Nicas, Vadim Dosych, Mitch Haviv, Angelo Susi, Dan Marcellus, Nachum Greenspan, Robert Sachs, Virendra Agarwal, Joel Rosner, Peter Delucia, Karen Minasia, Flourian Miciu, Henry Ming, Paul Kennery, John Sabini, Marge Kaelis, Kris Stala, Kathy Delligatti, Joseph Costantini and the many contributors of ACE Mailing List which is a think-tank in this domain with instant answers to most questions. This work is not funded by federal, state or other public agencies. The author is grateful to his family, Concurrent Engineering Research Center, West Virginia University, Software Engineering Chair Professor Jorge Diaz, Director of Center for Technology Transfer and Development Dr. Larry Dworkin and many other colleagues at Monmouth University for their encouragement and support.

Reference

[1] "Parallel Computing Still Not Ready for the Mainstream", Domenico Talia, Communications of the ACM, Vol. 40, No. 7, Pages 98-99, July 1997.

[2] Design Patterns for OO Software Development, Wolfgang Pree, Addison Wesley.

[3] Framework-Based Software Development in C++, Gregory Rogers, Prentice Hall, 1997, ISBN 0-13-533365-2.

[4] Even Wallace, Paul Clements and Kurt Wallnau, "Discovering a System Modernization Decision Framework; A case study in Migrating to Distributed Object Technology" in Component Based Software Engineering, Selected Papers from the Software Engineering Institute, Edited by Alan Brown, pages 113-123, IEEE Computer Society Press, 1996, ISBN 0-8186-7718-X.

[5] Douglas C. Schmidt, ACE: The Adaptive Communications Environment, see http://siesta.cs.wustl.edu/~schmidt/ACE.html.

[6] Managing Continuous Data Feed with Subscriber/Publisher Pattern. (TR-SE-DNA-95-003). OOPSLA 95 Workshop on Patterns (Austin, Texas, October 15, 1995).

[7] Unix Network Programming, Richard Stevens, Prentice Hall, 1990.

[8] The TaskPile is a customization of a shared workpile library written to accompany the book by "ThreadTime" by Scott J. Norton and Mark D. Dipasquale, Hewlett Packard Professional Books, Published by Prentice Hall, 1996.

[9] The ThreadPool implementation is a customization of a library written by Robert Sproull to accompany the book by "Programming with Threads," by Steve Kleiman, Devang Shah, and Bart Smaalders (SunSoft Press, 1995).

[10] Raman Kannan," Scalable Architecture for Reliable, High Volume Data Feed Handlers", TR-SE-DNA-97-002, Software Engineering Department, Monmouth University, NJ 07764.