XSAR and the TPOCC Display Subsystem

May 21, 1992

The Problem
A Solution
Tcl
Programmable Widgets and XSAR
Other Concerns
Summary
References

Since we will be creating new display widgets for our project (e.g, the form and list widgets), it might be worthwhile to consider some alternative approaches in the design of the XSAR widgets. I had some thoughts on the subject that might prove useful. (Jeff: "That's a change - on both counts!") This memo suggests that we investigate the possibility of making our widgets user-programmable, a capability that would have the following benefits:

• The domain of tasks to which a particular generic widget could be applied would be greatly expanded, thus lessening the number of specialized widgets we'd need to write.

• Significant, mission-specific customizations of the display system could be accommodated without changes to the widget source code.

The Problem

The TPOCC display subsystem is flexible in many ways and rigid in others. Ironically, the very mechanisms that give the TPOCC display system its flexibility - the resource database and UIL page definitions - are also the source of the display system's inflexibilities. With resource and UIL files, you can define the look of a display page, but you can't define its behavior beyond what is currently hard-wired in the widget's C code. This (i) limits the usefulness of a given widget with respect to the range of applications it supports and (ii) increases maintenance costs, since enhancing or correcting the functionality of a widget necessitates digging into the internals of the widget's code. The limited versatility of a widget with hard-coded behavior could pose a problem for XSAR. In particular, can we design a generic form widget that will support all of the forms planned for the XSAR operator interface, with the layout and function of each form specified solely through resource database entries and UIL page definitions? Or, must we design a whole slew of specialized widgets (or programs)?

With everyone and his/her brother working on the XSAR display system, maintaining hard-coded widgets would not seem to be a problem, at least not for manpower reasons. The problems that could occur in general might best be illustrated by example.

GSMS is a spacecraft monitoring system that is not part of a mission's control center; it is a separate entity, running on a separate computer, and connected to the POCC via the network. GSMS needs to display and access system variables of two types: its own GSMS-specific system variables (local to the GSMS computer) and the target mission's system variables (found on the POCC's computers). Consequently, GSMS must talk to two data servers: the local GSMS data server and the remote mission's data server, each of which has a different mission name and, thus, a different server name.

TPOCC does not currently support GSMS's multi-mission mode of operation. A proposal to add mission-level identification of system variables was rejected, probably because of the changes that would have been required in a number of TPOCC subsystems. For the display system, the declaration and handling of a new mission resource would have to be added to most of the TPOCC widgets, the TPOCC WML definitions and UIL compiler would have to be updated to reflect the new resource, and existing resource files and (possibly) UIL files would have to be changed.

  mnemonic#process@host[offset:count]

As Mike mentioned in one of our meetings, this problem with simultaneously processing and displaying data from multiple missions affects the XSAR design. Our requirement to support the decommutation and display of real-time and playback data at the same time is effectively a matter of handling two missions: real-time and playback.

We uncovered another problem at one of our recent TS/CS meetings: a UIL-based display system doesn't respond easily to database and configuration updates. The old TPOCC page description language (TDL) allowed resource values to be retrieved - at run-time - from the Unify database or from environment variables, a capability that XSAR would find useful. For example, when you bring up a page, a generic, UIL-specified page could be read in and particular resource values retrieved from our "database": the ADT and the GDT.

In the old TDL, a resource value could be expressed as a literal string, as "%dbms (table, key, field)", or as "%env (variable)". Adding a similar capability to the current TPOCC display program would require changing all the resources of interest to strings and modifying the widget creation functions to check for and resolve dynamic references in these resource values. (Perhaps you could just specify the resources as strings in UIL and let the resource converters perform the value resolution.) It might be argued that doing this subverts the validity checks of the UIL compiler, but these checks are solely syntactic in nature; the semantic checks have always been left to the data server.

A Solution

As briefly discussed in the introduction, user-programmable widgets could benefit the XSAR project in the following ways:

• Development costs would be reduced because a generic widget could be "user-programmed" to fulfill a range of roles that would normally be performed by multiple, "hard-coded" widgets.

• Test, integration, and maintenance costs would likewise be reduced because the standard test-edit-compile-link-test cycle could, in many instances, be replaced by a simple test-edit-test cycle.

Making a widget user-programmable requires that an extensible, command language interpreter (or two or more) be embedded in the display system. The benefits of an extensible command language are evident in TPOCC's TSTOL interpreter. TSTOL implements a generic set of STOL directives; mission-specific directives are defined and programmed using TSTOL itself. TSTOL's programmability allows users to extend the lexical structure, syntax, and semantics of the language - without changing the executable program.

The TAE Command Language (TCL) is similar to TSTOL. One feature of TCL is that it gives the user access, at the TCL level, to TAE's Wpt functions for creating and controlling display widgets. This allows you, for instance, to test new or updated widgets without having to write special test programs. Neither TSTOL nor TCL are embeddable languages and the requirements that guided their design make them unlikely candidates for that role.

Embedded command languages have proven their utility in a number of interactive applications. The Emacs editor uses a LISP-like internal language; the VAX/VMS EVE editor is programmed using the Pascal-like TPU language. Cadre's teamwork CASE tool allows users to define their own menus using the Teamwork Extensibility Language (TEL - my abbreviation). An arbitrarily-complex TEL expression can be assigned to a menu item; when that item is selected, its action expression is evaluated. With TEL, you can pop up a dialog box to query the user for more information, iteratively apply an action to multiple selections, store and retrieve variables, execute shell (DCL) commands, and so on. Although he found the language itself awkward, Tom Nicinski used TEL to couple two stand-alone applications: at any level and location in a dataflow diagram (built by Teamwork), one could switch to editing the relevant architecture-flow diagram (built by xfig).

Programmers developing X Windows-based, display applications have not been left out in the cold; they have several languages to choose from. One such language is WINTERP (Widget INTERPreter), developed by Niels Mayer at HP's Human-Computer Interaction Laboratory. WINTERP is built on top of XLISP, a public domain, object-oriented version of LISP. WINTERP is not really an embedded language; the WINTERP interpreter runs as a network server process, separate from its client application(s). WINTERP's advantages: it's free (as part of the MIT X11 distribution) and it knows about Motif. WINTERP's disadvantages: it's programmed in XLISP (a problem for those not familiar with LISP), it runs in a separate process, and it looks like it would be difficult to port to VMS.

                       network I/O
              X Windows <-------> Wafe <--------> Application
                                      standard I/O

The Widget Creation Library (Wcl) is another language that was praised in a USENET posting. Called by one user "a poor man's UIL", Wcl allows you "to specify all the static properties ... and some of the dynamics" of a display interface via the X resource database. Wcl is a library layered over the X Toolkit. Its ability to control the behavior of an application is limited to specifying a string to be passed to callback routines when they are invoked. Wcl's strong points are that it has been ported to VMS and that it allows you to write display programs that are, to a large extent, independent of the widget set being used (e.g., Motif or Open Look).

Tcl

A number of other embedded command languages are available, most of which are based on some variant of LISP. One language which is not is the Tool Command Language (Tcl). Tcl (no relation to TAE's TCL) can be characterized as follows:

• It's small (60K bytes of object code on a DECstation 3100).

• It's fast (an average of 160 microseconds per command on a DECstation 3100).

• It supports strings, numbers, lists, looping and conditional statements, file I/O, pattern matching, and variables.

• It's extensible. New commands can be defined at run-time to call and pass arguments either to Tcl procedures or to C functions that have been linked with the application. Tcl's C language interface provides a mechanism for accessing arbitrary operating and windowing system functions (e.g., Motif calls) from the Tcl command level.

• Multiple Tcl interpreters, each with their own execution context, can be active within an application.

• It's been ported to VMS. As long as you only use operating system functions that are available in both UNIX and VMS, Tcl commands and procedures will be portable across platforms.

• It's free. Despite being free, Tcl is no idle hack: the source code is meticulously commented and formatted.

The power of Tcl is evident from the fact that a complete X11 toolkit, Tk, was written based on Tcl. Tk provides most of the capabilities of the standard X11 toolkit, Xt. Although we won't be using the Tk toolkit, its use of Tcl illustrates how Tcl could be used in a programmable GUI:

• A Tcl command is associated with each widget type:

                   object_type pathname [attributes]
  

  Executing that Tcl command creates a new instance of the widget, identified by its pathname (in standard Xt dot notation).
  
  
• When a widget is instantiated, a new Tcl command, "pathname", is defined. This command allows Tcl-level access to the object's methods. The format of the command reproduces the classic message passing notation used in some object-oriented languages:

                      object method arg1 arg2 ...
  

  Some typical methods include configure (set or change a widget's resources), activate/deactivate, flash ("blink" the widget a few times), select/deselect, toggle, and invoke (execute the Tcl command assigned to this object).
  
  
• A Tcl command can be associated with a widget and will be invoked when certain events happen. For instance, depressing a button widget causes the execution of the Tcl command bound to that object. The command can be a simple Tcl command, a procedure invocation, or a call to a C function. Some widget types append additional information to the command before execution. Scrollbar widgets, for example, add a line number to the command string.
  
  
• Tcl commands can be bound to keyboard keys. For example, hitting the F11 key might send a ".allow invoke" command to the /ALLOW button on the critical command verification panel; the F12 key could send a ".cancel invoke" command to the /CANCEL button. (The buttons would, in turn, execute their commands to send the appropriate directives to TSTOL.)
  
  
• A send command is defined which allows an application to easily exchange information with or control other applications. Since send is implemented using X11 window properties, the participating applications must all be X11-based and have embedded Tcl interpreters.

Tk does not attempt to implement the inheritance mechanisms favored by other toolkits; instead, composition is the preferred method for reusing code in new widget types. As the authors of the C++-based, InterViews toolkit say,

"A rich set of primitive and composition objects promotes flexibility, while composition itself represents a powerful way to specify sophisticated and diverse interfaces."

In a sense, composition is a form of delegation, considered by some to be a superior object-oriented programming paradigm. Rather than inheriting capabilities from a parent class, an object delegates responsibilities it can't fulfill to other objects.

Currently, the TPOCC widgets comprise a forest of widget type hierarchies, because different widgets are descended from different Motif "seeds":

               XmLabel         XmPrimitive       XmScrolledWindow
                  |                 |                   |
            XtpRealTimeLabel    XtpRTPrim        XtpScrollingText
                  |                 |                   |
               XtpAscii,        XtpLGraph    XtpRealTimeScrollingText
              XtpNumeric,           |                   |
                 ...            XtpXTGraph,          XtpCpage,
                                XtpXYGraph        XtpEventWindow

By combining inheritance with delegation, a single inheritance hierarchy could conceivably be created for the TPOCC widgets (delegation is indicated by horizontal branches):

                             TPOCC Core Widget Class
                                      /   \
               Motif Class(es) -- Form     List -- Motif Class(es)

Structuring the widget classes in this fashion would have a number of advantages. First, common characteristics of the widget classes could be collected in abstract classes, a technique commonly used in languages that support multiple inheritance. For example, the data communications aspects (those dealing with the data server) of the widgets could be isolated in a separate class. Then, a real-time widget that needs to request data from the data server could delegate that function to a data communication object.

Mike recently discovered another advantage of having a single inheritance hierarchy. I'm not familiar with all the details; I wasn't following the discussion closely and, once Fred got involved, I couldn't hear the forest for the trees! From what I understand, Mike wanted to add a flag to control the update of a display object, but, since the object was a Motif widget, that was out of the question. Wrapping a TPOCC "core" widget around the Motif widget gives us a hook upon which to hang new resources like the update flag. The flag could be stored in the callback code or, if Tcl commands can be assigned to the TPOCC core widget, in a Tcl variable.

Programmable Widgets and XSAR

The remainder of this memo discusses how we might use an embedded command language like Tcl in XSAR's form and list widgets. Before continuing, it should be pointed out that using an embedded command language does not preclude the use of the X resource database or of UIL page definitions. The Tcl commands associated with a widget would be defined as resources of the widgets, specified in the user's resource file or in the display's UIL files.

A Generic List Widget

A generic list widget is particularly interesting because it could be used in many different ways on XSAR: to view reports stored in files, to monitor scrolling real-time data (e.g., events), to display tables of information, etc. List widgets could be used both to select and display report files. For example, John Ousterhout, in his paper on the Tk toolkit, presents a directory/file browser implemented by a 16-line Tcl script:

• 3 lines - create and couple a list object and a scrollbar object.
• 3 lines - load the list object with a directory listing.
• 8 lines - define an examine procedure.
• 1 line - binds the examine procedure to a function key.
• 1 line - binds the quit function to a key.

Short and sweet! Seen in action, the browse script brings up a window containing a list of the files in your current directory. Scrolling up and down the list, you can select the file you want to examine. When the defined function key is hit, the examine procedure is automatically called. If the selected file is a directory, examine recursively invokes the browse script to bring up the subdirectory's listing in another window. If the selected file is an actual file, examine spawns a process to edit the file.

The events/alarms subsystem (whether MPOS or TPOCC, I'm not sure) will be the data source for many of XSAR's displays; e.g., the Telemetry Validation "Table". The list widget is a natural for any events-based display page. Assume the generic XSAR list widget has the following methods (callbacks), each with an associated Tcl command:

initialize - initialize the list object.

select - process the selected list item.

popup - popup another display object.

insert/append - insert or add an item to the list.

update - process a real-time data update.

A method, when it is called, executes the Tcl command assigned to it, as well as any internal processing that needs to be done. Some commands will be provided with additional information; for example, update needs to pass the incoming data value to its Tcl command.

A general-purpose, XSAR events/alarms page could then be implemented using the generic list widget and the following Tcl command assignments:

initialize - execute a Tcl script to:

1. Create an unmanaged class selection popup.
2. Bind the popup to the right mouse button.
3. Issue a command to the events/alarms server to initiate a stream of event and alarm messages.

select - null.

popup - pop up the class selection "menu".

insert/append - add a line of text to the list.

update - call append to add the incoming event/alarm message to the scrolling list of messages.

To change the class of events and alarms being viewed, the operator would click on the right mouse button and a class selection menu would pop up on the screen. This popup could also be implemented as a list, with the following Tcl definitions:

initialize - execute a Tcl script to:

1. Construct an initial list of legal classes.
2. Add any UIL- or Xrm-specified classes to the list.
3. Bind the left mouse button to the select function.

select - highlight the current item and call popup. (A real implementation would support the selection of multiple items.)

popup - display an APPLY/CANCEL dialog window. APPLY effects the class change; CANCEL leaves everything as is.

insert/append - add a line of text to the list.

update - null.

Actually changing the event/alarm classes could be accomplished in either of two ways:

1. Send a class change command to the events/alarms server.
   
   
2. Store the new class in a Tcl global variable and let the update method's Tcl command filter the incoming messages.

Note that the latter approach could be used to give any dynamic, scrolling list display an input filtering capability - without changing the generic list widget's code. This might come in handy if, for example, we are forced to read messages from the MPOS Events/Alarms server and it doesn't support message filtering on its end.

Programming widgets in Tcl doesn't eliminate the need to write C code, but it does provide a clear separation between the display aspects of the widget and the functional aspects. New Tcl commands (to initiate data and events streams, for instance) must be implemented in C and "registered" with the Tcl interpreter. With some forethought, these C routines could be general purpose functions usable by other, non-display subsystems.

The XSAR display system will include a number of tabular displays, which, not surprisingly, could be handled by a generic list widget that allows an optional, non-scrolling header at the top of its list. The header could be specified as a single, text-string resource with embedded new-lines (if necessary); the widget would break the string into separate lines and determine how many non-scrolling lines (0 to N) to allocate for the header.

Despite the fact that a list is a list, a list object need not be limited to sequential updates. For example, TPOCC's command page (CPAGE) is essentially a random-access list of the current contents of the command buffer; each CPAGE update received by the display program specifies the associated command's position in the displayed list. If XSAR's generic list widget employed a similar scheme, it could even be used to implement screens such as the Telemetry Archive Log Display, a 4-line list of the 4 different kinds of major frame log. (I don't recommend this approach for this particular display, since it moves some of the display functionality [the formatting of the lines] into the Telemetry History Manager.)

Aside: As of Release 6, TPOCC did not have a generic list widget. Consequently, people had to resort to using the CPAGE for non-command, list-oriented displays. The XSAR generic list widget could be used for these displays, as well as for the command and events pages. The somewhat redundant CPAGE and EVENT data objects received from the data and events servers could be replaced by a single "list string" containing the following items of information:

• position information (e.g., CPAGE line numbers),
• display attributes (e.g., to provide for displaying events of different severities in different colors), and
• the text to be displayed.

The position information and display attributes should be encapsulated so as to allow for future additions without affecting existing software. With some effort, control sequences embedded in the string text could be recognized and used to change display attributes in mid-string.

A Generic Form Widget?

After giving the matter some thought, I'm not really sure what a generic form widget would look like, or even if one is needed. Presumably, it would be some kind of container widget, a subclass of the Motif XmManager metaclass (itself a subclass of the Xt Composite metaclass). The generic form widget might support the following methods, each of which has a Tcl command resource:

initialize - initialize the form.

process - process the current contents of the form.

terminate - destroy the form.

The initialize method is responsible for laying out and configuring the display and input objects of which the form is composed. The layout of the form could be specified in UIL or programmed in a Tcl script invoked by the method's Tcl command. Regardless of how the form is layed out, initialize's Tcl procedure could also perform such non-display functions as requesting command controller status for command processing panels.

The terminate method's Tcl command is executed when the operator exits the form. This command performs any wrap-up activities required for the form; e.g., relinquishing command controller status or saving current field values (to be recalled when the form is displayed next).

The process method is responsible for collecting and processing the values entered on the form. This can all be done by a Tcl procedure assigned to this method. If the form is built from TPOCC/XSAR input objects, individual field values could be stored by the Tcl commands bound to those objects; the form's process method would only be called when the operator has completed the form (e.g., has hit the SEND button on an XSAR command processing panel). Motif input objects have no Tcl command execution capability, so the activation callback defined for such an object could pass the field name and value to the form's process method. The process method would then have to remember the different fields' values until the form is complete, at which time the processing of the form as a whole could take place. (Or, now that I think about it, the Motif object's callback could execute a Tcl command; it's just that the Motif widgets have no explicit Tcl command resources.)

It is interesting and, perhaps, instructive to see how a form is created - entirely from a Tcl script - using the Tk X Windows toolkit. For example, the XSAR "Change Telemetry Replay Parameters" form consists of 7 input fields arranged vertically down the page and a row of 6 buttons along the bottom of the form. The Tcl script to generate and process this form would work as follows:

1. Create a Tk toplevel container object (similar to the Xt TopLevelShell widget, except that the Tk widget can have more than one child).
   
   
2. Create a Tk frame container object (similar to Motif's XmForm widget) to hold all of the operator input fields. Call this the input region frame.
   
   
3. For each of the 7 input fields:
   
   
   1. Create a field frame object for the field's label and input objects.
   2. Create a Tk label object for the field's label.
   3. Create a Tk entry object (a single-line, XmText-like input widget) for the operator's input.
   4. Define the entry object's keyboard bindings (e.g., for editing). Since these bindings will be common to all of the input fields, a Tcl procedure can be called to assign the bindings to a specified entry object.
   5. Position the label and entry objects side-by-side in the field's frame using the Tk-extended Tcl command, pack.
   
   
4. Pack the 7 field frames vertically in the input region frame created in step 2.
   
   
5. Create a button frame to hold the buttons.
   
   
6. Create a Tk button object for each of the 6 buttons.
   
   
7. Pack the 6 buttons horizontally in the button frame.
   
   
8. Finally, pack the input region and button frames, one above the other, in the top-level window.

(This example is based on a demo script included in the Tk distribution.) The Tk/Tcl commands that create a widget also specify its resources; e.g, button labels, Tcl command bindings, fonts, default text, etc. In all, about 50 or 60 lines of Tk-extended Tcl would suffice to build the XSAR "Change Telemetry Replay Parameters" form. Compare this to the number of lines of UIL required to layout the page (many more, I assume and hope!). Also, compare the ease with which the Tcl script-based form can be changed. Modifying the look or behavior of existing fields and buttons - or adding entirely new ones - is a simple matter of editing the script file and bringing up the form again. There is no need to recompile a UIL file (if a change was made in the UIL file) or to refresh the Xrm database (if the change was effected in a defaults file).

Other Concerns

Two questions that must be asked when considering design changes to any system are how the changes affect the existing system and how the changes will affect performance. Embedding Tcl in the display program would have virtually no impact on the current TPOCC display software. The main display routine would need to create the Tcl command interpreter (by calling Tcl_CreateInterp()) and register the XSAR command extensions (by calling, for each new command, Tcl_CreateCommand()), but the other display code would remain as-is. Tcl capabilities could be retrofitted into the existing TPOCC widgets by adding new Tcl command resources to the widget structures and by adding a Tcl_Eval() call (to evaluate the commands) in the appropriate callbacks.

Within the context of an interactive application, the performance of an embedded command language like Tcl is almost a non-issue. Countless users of programmable editors (Emacs, EVE, Brief, etc.), on computers ranging from PCs to VAXes to Crays, voice no complaints about the speed of their systems. John Ousterhout says that "the Tcl interpreter", averaging 160 microseconds per command on a DECstation 3100, "is fast enough to execute many hundreds of Tcl commands within a human response time". Inverting the 160-microsecond figure puts things into a little more perspective: 6250 commands per second. And the XSAR telemetry replay form described earlier was created by 50-60 lines of Tcl.

Executing Tcl commands as part of a real-time widget's update callback might be of concern if large volumes of data are being displayed or if data is coming in at a high rate. The solution to this problem, of course, is to specify an empty Tcl command for the update callback in the page definition.

Summary

In summary, user-programmable widgets would benefit the XSAR project in the following ways:

• Lower development costs, since a user-programmable, generic widget could play the roles normally assigned to multiple, specialized widgets.
  
  
• Lower testing and maintenance costs, since corrections can be made and enhancements added without having to rebuild the software.

User-programmable widgets also hold out the potential of more fully isolating the display system kernel from mission-specific functionality. TPOCC programs like TSTOL and Display achieve their mission-independence by attempting to be prepared for every eventuality. A better approach might be one similar to that used by the state manager and telemetry decommutator: link the core display system with a mission-specific library of Tcl-accessible, C functions, including an initialization function that would register the mission's new Tcl commands with the Tcl interpreter. (Storing mission-specific code in a dynamically-linked, shared library would enable the creation of a truly generic display program, decommutator, or whatever. Loading and linking object modules at run-time is supported under UNIX, VMS, and VxWorks.)

While decoupling an application from its user interface is usually considered a desirable goal, some people have pointed out that a user interface that doesn't know anything about its underlying application is liable to be a user-unfriendly interface. User-programmable widgets let you have your cake and eat it too: languages like Tcl make it possible to embed mission-specific knowledge in the user interface while, at the same time, keeping it separate from the hard-coded heart of the display system.

References

Century Computing, Inc., TAE Command Language (TCL) Programmer's Manual, Version 5.1, April 1991.

Daniel Connolly, USENET message on "Programmable Window-Based Interfaces", March 1992.

Mark Linton, John M. Vlissides, and Paul R. Calder, "Composing User Interfaces with InterViews" (PDF), in IEEE Computer, February 1989.

Alex Measday, TSTOL Reference Manual, January 1992.

Gustaf Neumann and Stefan Nusser, Wafe Manual, 1992. (1993 article: "Wafe — An X Toolkit Based Frontend for Application Programs in Various Programming Languages")

Tom Nicinski, "Extending Teamwork for Architecture Diagrams", in IEEE Software, May 1992.

John K. Ousterhout, "Tcl: An Embeddable Command Language", in 1990 Winter USENIX Conference Proceedings.

John K. Ousterhout, "An X11 Toolkit Based on the Tcl Language", in 1991 Winter USENIX Conference Proceedings.

Rob Pike, "8½, the Plan 9 Window System", in 1991 Summer USENIX Conference Proceedings.

David E. Smyth, "Wcl - Widget Creation Library: A Thin Veneer Over Xrm", January 1991.

Douglas A. Young, The X Window System: Programming and Applications with Xt, OSF/Motif Edition, 1990.