THEATRE DESIGN & TECHNOLOGY MAGAZINE

1988 FALL ISSUE - SOUND COLUMN

A SOUND FUTURE (PART 2)

By Charlie Richmond 

In the first part I talked about how computers tell designers
what they are doing and we saw (hopefully) how sometimes the most
intuitive way of doing this may not necessarily be the best or
most efficient.  Of course, the reason for this is most likely
the physical limitations of the presentation medium (the video
screen size and resolution) or possibly just the physical
limitations of the space in which the designer has to view the
data. Similar limitations are imposed by non-computerized sound
systems which require a full complement of physical hardware in
order to display system status.  The advantage that computer
presentations have is that enormous amounts of information can be
viewed in a small space, but not concurrently.

Just as displayed information can be selectively viewed, enlarged
and focused upon for greater detail, the computer controlled
sound system can provide the designer or operator with a vast
variety of control input capabilities using a few precision input
devices which may be assigned to control any desired system
parameters.  The same input device may even be the one to both
select the parameter to control and then to control it.  The most
common device of this type these days is the mouse. 

The major imperfection with the mouse (according to audio mixers)
is that it is not capable of controlling several 'analog'
parameters simultaneously and cannot be made to simulate
operating several different straight-line audio faders in
distinctly different ways using a number of fingers
simultaneously.  If this is required we usually have to provide a
number of conventional faders which may be either permanently
assigned to specific audio functions or may be assignable within
the system's programming capabilities. 

But I'm getting ahead of myself...  Let's go back a bit and look
at the task we are attempting to accomplish:  how a human can
tell a computer what they want it to do.  In some respects this
is an easier thing to do than the reverse, i. e. when the
computer is communicating back to the user, it must present
virtually all information in which the user might conceivably be
interested and it must be presented in a logical,
priority-related manner providing quick access to all desired
data.  As a matter of fact, telling the computer which data you
wish to see is a fundamental subset of the other side of the
coin. 

This other side can be defined from the user's perspective in a
very simplistic way if you wish:  "What I want it to do is very
simple; I know exactly what it is, it's only one thing and I want
it to do it now with a minimum of effort on my part."  Many
computer users cannot understand that this statement ignores the
fact that the user's brain has already filtered every single bit
of information they have learned since they were born and is
currently ignoring every sensory input impinging upon them except
for the glowing screen in front.  The task of providing an
adequately responsive environment for persons with these demands
is the software and hardware developers' greatest problem. 

This would be simpler than presenting all data simultaneously if
computers could read people's minds.  Have no fear, this will
come, but for now we have to use other techniques.  Humans have
developed elaborate ways of communicating the wishes of their
brains to their environment.  If the environment understands
speech, the voice could be used. 

Voice pattern and speech/word recognition are actually being used
widely in industrial computer applications.  In the theatre there
are two primary difficulties with this:  (1) response time and
error potential is still too great for the speed and precision we
require - the audio operator must be confident the correct
command will be given at precisely the right time; and (2) vocal
commands given by the audio operator in the house during a live
performance may not be appreciated fully by the audience (or the
director).

Delving into non-verbal communication brings us to body language. 
As we are all being currently taught, these movements and
expressions can be extremely telling and accurate.  Whether or
not a computer operator could maintain precise enough control
over them to run a show has yet to be shown.  Currently under
study for the purpose of furthering of our knowledge in this
area are complex experiments based on head and eye movements.  In
psychological and physiological testing, cameras track the focal
point of the subject's eyes to accurately determine specific
areas of visual interest as they scan typical scenes and images
in the performance of average human tasks.  Computer analysis of
this data is so complex and time-consuming it must be done frame
by frame with powerful interpretive software running on expensive
mainframe computers.  Maybe someday it will be in our ball park.

On a more practical level, the movement of a jet fighter pilot's
head is translated directly into gun aiming information so that
the plane's cannon are always facing the same direction as the
pilot.  Obviously this is done very quickly, albeit with some
rather expensive computer equipment, and has already been
translated to film use in the design of servo-controlled robots
which precisely follow the movements of a 'wired' actor.  What we
have so far not been able to imagine is what normal movements of
a sound operator would precisely translate into which standard
sound system responses, but we're definitely open to suggestions.

Moving into the more mundane levels of computer control, we have
two devices widely used for pre-programming, but which have not
gained much favour for live control:  the digitizing tablet and
the light pen.  The digitizing tablet is extremely precise and
well liked for drafting and architectural drawing.  Its major
disadvantage is that one is constantly looking back and forth
between the tablet (where the input is done) and the screen
(where the results are viewed).  This distracts, slows input
speed and creates slightly more margin for error.  Current
opinion is that new software for mouse-based environments will
make the digitizing tablet obsolete.

The light pen avoids the digitizing tablet's biggest problem, the
perceptual gap between input and output, but creates new ones: 
it has much lower resolution (in fact considerably less than its
corresponding video display because of physical interface
limitations) and it can be tricky to use without making mistakes. 
The light pen is still popular in some areas but has basically
been superceded by the mouse. 

Now if all this sounds like I'm going to say the mouse is great,
you're almost right.  The mouse has solved some of the more
obvious disadvantages of its predecessors.  But, just like the
search for the holy grail, the closer we get to totally intuitive
computers, the further we have to go.  The biggest encumbrance at
the moment, believe it or not, is that the mouse is too mobile. 
It never seems to be just in the right place; always just a
little too far to the left or right, or perhaps even lost
momentarily because someone pushed it aside to make room for the
script.  A petty complaint maybe, but certainly not one which is
normally levelled at the keyboard. 

To be fair, an 'inside-out' mouse called the trackball is
available which solves this problem yet again creates others. 
The trackball has a personality all its own which is surprisingly
different from the mouse:  in its various forms it tends to be
heavy, slow, awkward and/or somewhat inaccurate.  As well, it has
not yet incorporated other controls such as buttons or switches
nearly as successfully as the mouse.

Actually, the positioning problems with the mouse could be solved
to a large extent by changing the way the screen pointer responds
to the mouse.  By using a phantom 'absolute mouse position'
pointer that continues beyond the screen's boundaries, the mouse
would not always find itself 'running out of room' as it
currently does.  This would naturally require a new approach to
the way all current mouse-based operating systems are written
plus create a whole new set of operating difficulties.  One new
function required would be a quick way of 'finding the mouse and
putting it back on the screen' when it wanders too far away (this
brings to mind the difficulties sometimes experienced in finding
the beam on a badly misadjusted oscilloscope!)

There's also a 'new kid on the block,' the touch screen.  It's
been around for a long time in expensive interactive video
systems, but relatively new in PC environments.  There are a
number of new developments in this area allowing the addition of
touch screens to existing video displays as well as completely
integrated touch screen terminals with extremely sophisticated
capabilities.  Immediate reaction to the touch screen is always
positive:  here finally is a highly interactive device capable of
receiving input and displaying response at exactly the same
physical location, requiring only a press of the operator's
finger to respond to an on-screen request.

But, of course negatives exist in this world as well.  Even
though the touch screen is capable of very high display
resolution, its sensitivity to finger touch is very coarse,
somewhat touchy (intentionally) and necessarily slow (to
eliminate false triggering from any but firm responses.)  Still,
the best current approach for precise, intuitive, fast and
trouble-free computer control of audio systems is probably a well
developed combination of the best kind of mouse control
integrated properly with good touch screen technology. 

The mouse would control devices requiring precise, smooth and
wide range response such as faders, masters and other types of
'analog' controls, while the touch screen functions could be
easily applied to many other functions such as on-screen button
selection, menu selection and a multitude of programmable
capabilities in which the choices are explained in higher
resolution text and the method of choosing is to simply point at
the desired choice.  Additional physical devices such as
straight-line faders operating in a manner familiar to the
operator would be required for complete live control of multiple
audio channels.  These controls could be normal audio devices or
could actually communicate directly with the computer to provide
flexible software-based control of assignable parameters.

Of course, we have been making the assumption so far that the
kind of control desired is analogous to turning knobs, pushing
buttons and sliding faders on a phantom 'console' which now would
only be perceived as such on a video screen.  We must also
include by default a standard keyboard to facilitate the labeling
of controls and cues as well as providing a few important direct
system 'functions,' but live communication via typewritten
commands should be eliminated.

Conveniently, we will soon have an ideal opportunity to test this
combination of user controls since a new system with these
features is currently being installed at Det Norske Teatret in
Oslo, Norway.  This system uses a portable mouse- and
keyboard-controlled programmer's work station for designing and
auditioning designs in the theatre.  This computer actually
comprises the entire sound system control centre and controls all
system hardware which is permanently mounted backstage.  The
operator at this station has complete control of all editing and
live system control functions. 

At the same time, a remote touch screen work station and
conventional physical fader panel is located at the stage
manager's desk.  These controls provide complete live control
over the volume of 16 audio channels (chosen in a programmable
fashion) plus a fast and simple touch screen menu of 64
programmed system presets for instant selection and execution. 
All main system 'function' buttons plus start and stop controls
for eight sound sources such as tape machines are presented on
the touch screen.  This requirement exists because there is no
sound operator during performances.  The stage manager operates
the sound system entirely through these facilities and requires a
simply presented yet comprehensive range of controls.

So finally we are beginning to deliver ourselves from the dilemma
of having to use systems which are not fundamentally intuitive by
using special configurations of powerful and affordable
technology to create a system which performs as a transparent
extension of our intuitive desires; a tool needing no
interpretation between wish and result.

----------------------------------------

Copyright ©1988 Charlie Richmond