Evaluating Interpreters at Work — or Trying Not to Feel Superfluous

David Roberts
(Terminology Update, Volume 23, Number 2, 1990, page 13)

From March 1985 to March 1987 I worked as a senior interpreter for what was the Interpretation Directorate of the Translation Bureau. One of my responsibilities was to evaluate staff colleagues working for the Conference and Committee Sections. Observing the work of colleagues enabled me to appreciate more fully both the needs of customers and also the requirements of interpreters wishing to meet those needs.

In preparing for these assignments, I quickly realised that very little has been written on methods of assessing interpreters while they are at work. There are studies on the performance of student interpreters and candidates in interpretation examinations, but the circumstances in both cases are quite different from normal working conditions. Feeling on the one hand buoyed by a sense of freedom and yet on the other apprehensive about not having any frame of reference to fall back on, I decided to start by asking two seemingly simple questions.

• 1. Can the object be assessed accurately?
• 2. Does the assessment serve any purpose for the person being assessed?

1. Can the object be assessed accurately?

   In answering this question, we might consider three points:

   (a) conditions of work

   (b) understanding of the message

   (c) transmission of the message

   Obviously, the first point will influence the second, and the second will determine the third. The basic questions asked under (b) and (c) would not vary much whether asked in examination or evaluation conditions. They would include such considerations as: did the interpreter appear to understand the concepts involved or does the listener have to guess at the sense of the interpretation? Did the interpreter seem to understand all the nuances of the source language and the cultural references behind that language (allusions to sports, literature, politics, etc)? Under (b) we might consider, for example, whether the presentation was staccato or smooth, whether there was excessive self-correction, whether the interpreter was able to change pace when necessary or seemed unduly hesitant.

   These are just a few of the basic questions asked in tests or exams for interpreters, and a fair examiner would choose exercises consistent with the level of proficiency expected of candidates. In exam conditions the same quality of sound would be heard by all candidates, a transcript would be available of the original and a recording made of the interpretation. Regrettably, these ideal conditions are poles apart from the actual conditions in which professional evaluations are conducted. In fact, an evaluator rarely has a transcript of the proceedings or a recording available to check the accuracy of the interpretation. Also, the quality of the sound heard by the interpreter will vary according to the use participants make of microphones: although they are able to hear one another around the table, they do not always realize that they are not intelligible to the interpreters unless they speak into a microphone which has been switched on. These points seem to be too obvious to mention but they are in fact the very type of constraints which make it difficult to assess interpreters in normal working conditions. We might have to ask: did the interpreter misunderstand the question or did he simply not hear it because the microphone was not switched on? I would note in passing that this type of problem is compounded by some of the facilities in which interpretation is required. For example, in the main location for press conferences in Ottawa, the microphones hang from the ceiling and it is very difficult to hear, let alone interpret, the questions of journalists even when they stand to ask them. If they remain seated, they are inaudible to the interpreter and only someone endowed with remarkable gifts of ESP can fathom the question. Trying to interpret an answer without having heard the question is not normally considered conducive to a relaxed or confident delivery.

   These are examples of the physical conditions to be taken into account in conducting evaluations. There are also others regarding documentation and briefings which I shall refer to when considering the interpreter’s understanding of the message. Together, they constitute some of the factors an evaluator must take into account when assessing an interpreter’s performance.

   Notwithstanding such constraints, the reality is that professional evaluations are an administrative requirement. So, how should they be conducted? I found that the fairest approach in the circumstances was to put myself in the position of the person requiring the interpretation service. I might have decided to sit in the booth, thereby ensuring I received exactly the same sound as the interpreter, but this seemed artificial, likely to add tension to a job which is already stressful enough, and unpleasantly reminiscent of those Sunday afternoons when unwelcome relatives unexpectedly arrive for tea and there are never enough chairs to go around. So I listened in the meeting room itself using the earpieces provided. There were two disadvantages to this approach: first, it was sometimes difficult to hear the original; second, interpretation already requires very fast processing of information, and trying unconsciously to interpret the original (as I think all evaluators inevitably do so as to assess the difficulty of the original, while at the same time trying not to be too subjective) while also listening to the interpretation and evaluate its accuracy and presentation quickly proved to be a very tiring exercise. However, the advantage of this approach was that I received exactly the same interpretation as the people actually using the service, and this led me to appreciate more fully the importance of such factors as pace, structure, and particularly control as elements in good interpretation. But more on this later.

   The evaluator will try to assess whether the interpretation has fulfilled its purpose, which is to convey orally ideas expressed in another language. The ideas must first be understood clearly, and the words chosen to transmit them must be arranged to achieve that purpose efficiently. Obviously an interpreter who does not understand that may either parrot words or even resort to what is known loosely as "interprétation par glissement phonétique", working primarily with his ears rather than his brain and hanging on to words in the hope that some meaning can eventually be coaxed out of a disorderly stream of sound. Surprisingly perhaps, some may get away with that for a short while, but it is very hard on the listener who has to struggle with confusion expressed through an earpiece and without the support of body language suggesting at least a vague itinerary to the words.

   A good interpreter is intelligible and informative. But to be so and ensure the message is not submerged by a deluge of sound, the interpreter must go beyond words and have some knowledge of the matter being discussed. This usually implies being adequately briefed and receiving documentation to be studied before the meeting. Yet regrettably, many conference organizers still think interpreters can simply play it by ear, adapting to any subject with equal ease. This is very flattering but hardly realistic. Unlike the Pope, who is said to be infallible though not necessarily well informed, interpreters are very fallible if not given adequate information.

   There are sometimes surprising responses to the seemingly simple question: "Did the interpreter understand the message?" The question is relevant only when the conditions of work are taken into account. Some interpreters, as a result of their experience and conscientiousness in keeping glossaries and updating information, are able to give an efficient performance in almost any situation. This is to their credit, and it should be indicated in the evaluation. If other interpreters find a subject particularly difficult to grasp, the evaluator might consider whether adequate documentation in both official languages was provided before the conference to allow the interpreters to familiarize themselves with the subject.

   When a conference consists of readings of presentations, the interpreter who has received the documents only shortly beforehand may be able to produce something vaguely intelligible by hanging onto the text as best he can and giving linguistic equivalences of the technical terms involved. Although very unsatisfying from a professional viewpoint, this is often the approach taken. It works to some extent until (a) the speaker strays from his text to provide additional information; and (b) there is a question period following the presentation. It is then that the lack of briefing or documentation can create serious problems for the interpreter, who has been obliged to cope by dealing only with linguistic equivalents without understanding fully the concepts underlying them. The interpreter may find that he simply cannot determine the meaning of a question, especially when participants use terms particular to their specialized group or subject, or when participants are so familiar with a subject and at ease with one another that they speak in a kind of verbal shorthand. Of course, if the conference lasts long enough, the interpreter does have the opportunity of consolidating his knowledge and familiarizing himself with the particular jargon. Otherwise, he must simply batten down the hatches of his mind and hang on.

   Sometimes delegates are sufficiently aware of the interpreter presence to announce that they regret not having given them a copy of the text but that they will make sure they do not read too quickly. There are two problems here: (a) speakers have a limited time and often read the first half slowly, panic when they see they have little time left and race off at breakneck speed, protesting all the while that they are condensing their presentation, when in fact what they are doing is simply reading twice as fast as they would normally; (b) even when a speaker does speak slowly, the interpreter may still be faced with considerable problems if the information input rate is high, as it usually is when the speaker is reading. In such cases, the stress on the interpreter is intensified when adequate documentation has not been provided beforehand. As a result of excessive stress, efficiency drops while at the same time more and more information has to be processed. All in all, a rather unenviable set of circumstances. It should also be remembered that adequate documentation is at least as necessary to the interpreters as to the delegates at a meeting, since the latter are usually discussing subjects in which they have specialized training, whereas interpretation is one of the few professions in which members are often expected to grasp arguments and terms used in areas in which they have very limited knowledge. It could in fact be argued that far more carefully prepared documentation is needed by the interpreters than by the delegates themselves, whereas the reality is that in some cases the interpreters do not receive even the documentation sent to the delegates who are already presumed to be able to follow the papers being presented.

   Factors such as those mentioned above must be taken into consideration in an evaluation. However, if the rate at which information is provided is reasonable and the subject is not too obscure, the interpreter is expected to understand. Therefore, the next question is whether he transmitted the message clearly and accurately. The problem is that what is scrupulously accurate may not always be clear. I was told that a good translation or interpretation should make the same impression on the reader/listener as the original had on the person reading or listening to it. Logically, therefore, the level of language, education, speech mannerisms or other characteristics identifying an individual should be reflected in the interpretation. In reality, however, I find that government interpreters tend to speak in the somewhat unobtrusive, safe and dignified style expected of civil servants, even if that implies a certain amount of editing or toning down of the original. Like Corneille’s characters, who are said not to behave as people actually do but as they should, government interpreters speak the way the public expects government officials to. As a result, is the flavour of the original sometimes lost? Yes, I think it is. But is the content or message of the original transmitted—even if it is sometimes toned down? I think so.

   We may note that there are some interpreters who give everything and in exactly the same tone as the original, conveying every sigh and nuance. I think this approach may be necessary in court interpretation, where the work often consists of examination of witnesses and the interpreter is usually visible, and where every suggestion of doubt or hesitation in answering a question may be of considerable consequence. However, if simultaneous interpretation of conferences or committees was generally provided in this way, the person having to listen for long periods through an earpiece would soon be exhausted from the very effort of filtering the information and trying to determine where one sentence finished and the next began. Listening to interpretation through an earpiece can be tiring, and the good interpreter realizes this and adapts his presentation accordingly.

   This leads to the next question: why does one person’s interpretation appear clear and convincing whereas another’s is jumbled and nervous? I don’t think this is solely a problem of language; rather, I think it is because some interpreters seem to adapt to the medium better than others. If, notwithstanding the various constraints we mentioned, we presume all interpreters clearly hear and accurately understand the sense of the original, why do some convey the message more effectively than others? To try to answer that question I considered the qualities of those interpreters whom, if necessary, I would find it pleasant and informative to listen to for periods of twenty or thirty minutes at a time. I found that generally their tone struck the right balance: they seemed to combine interest with cool detachment and precision, in the manner of a good network newsreader. This was the case even when they were at meetings which must have been of stultifying boredom to anyone but those directly involved. Even in situations of numbing tedium, they never became numbingly tedious. Their enunciation remained clear and crisp, and ideas were presented in an orderly manner. Their orderliness was often reflected as much by what they did not say as by what they said. Their grasp of the original was such that they were not afraid to pause, and it was those very pauses that often lent considerable authority to their words. Unhurried and precise, they were ready to use pauses judiciously between units of meaning so as to allow the listener time to absorb the ideas. This is something that beginners, who are sometimes afraid of "dead air", might do well to emulate. As suggested earlier, listening with an earpiece to an uninterrupted stream of words can be tiring, and the occasional well-placed pause by the interpreter is very welcome.

   Related to this is the ability of the best interpreters to keep their sentences well organized and often quite short. Essentially, their sentences could be broken down into three parts: subject, verb and object. A clear pattern was followed consistently, but tedium or the impression of repetition was avoided by the suppleness and exactness of their language and the quiet authority with which they spoke. If one word could describe these craftsmen, I think it might be "control". They were in control of the message and knew what they wished to convey. It has been suggested that simultaneous interpreters, after much practice, learn to ignore the sound of their own voice. I do not think this true of the better interpreters, whose effectiveness is, on the contrary, related to their awareness of what they are saying. Some might argue that this impression of control is simply an illusion since simultaneous interpretation involves so many activities taking place at almost the same time. In answer, I can only refer to James Callaghan’s description of Margaret Thatcher: "Serene and swan-like on the surface but paddling like crazy underneath". I found that the same description could be applied to good interpreters, who appear to be in total control even as they may be desperately searching for a reference in a document, a term in a glossary or the name of an Act. As a listener, I found this question of control, which involves pace, tone, clarity and ease of expression, to be far more important than the occasional gallicism, a slight change in a figure or name of a program. To achieve this effect, the message must be understood accurately and quickly, and be presented in such a way that the listener can assimilate it through an earpiece.

   Marshall McLuhan’s distinction between a "hot" and a "cool" medium of communication may be relevant here. He suggested that in the age of a cool medium like television a figure such as Hitler would have had little chance of coming to power because his shrieking and violent gestures would have seemed absurd if presented on television and viewed from a comfortable chair in the intimacy of a living room. Perhaps there is a link here with the earpiece used to listen to interpretation. Is it also a "cool" medium suited primarily to a controlled, calm delivery? The voice can still be interesting and well modulated, but abrupt changes in volume or pace are certainly a hindrance to communication and disturbing to someone who is listening through an earpiece and is thus not always able to anticipate changes in tone through body language or facial gestures.

   I think that if we follow a generally accepted definition of interpretation as the transmission of cognitive content, which, as we have seen, implies familiarity with the concepts and terms involved, it is possible to make a general evaluation of interpreters in working conditions. A more detailed evaluation requires tools such as transcripts, recordings, etc, which are not generally available to the evaluator. In both cases, in view of the constraints mentioned, such as availability of documentation, specialized nature of the subject, terminology specific to one particular group and so on, it is essential to stress that even a broad evaluation has to be conducted by someone with experience of the possibilities and frustrations of interpretation. Somerset Maugham said that an author who wishes to convey the taste of roast mutton should not be required to have devoured a whole sheep but should have eaten the occasional lamb chop. I think the same could be said of evaluating interpreters. The craft is comprehensible to the craftsman who has served the necessary apprenticeship.

2. Does the evaluation serve any purpose?

   I suppose the answer to this question depends on who is doing the evaluation and the attitude of the person evaluated. Obviously, little if any advice can be offered regarding the particular subject of the meeting, which is probably as alien to the evaluator as to the interpreter. However, advice can be suggested on presentation of information. An interpreter who has difficulty establishing that air of control we mentioned earlier might be advised to start slowly but firmly. This is particularly important in the first few sentences since the faster the interpreter tries to speak, the more tense his voice becomes, the more he tends to panic and the less he analyses. So, a conscious effort should be made to speak in a well-ordered and coherent manner from the outset. Clearly, the degree to which one can do that will be influenced by the structure and presentation of the original, but good interpreters do find intelligent solutions to problems of meaning, produce smooth transitions from unstructured thoughts, omit cautiously but rephrase appropriately. I think that these examples are evidence of interpretation strengths and should be mentioned in an evaluation as an attainment or as an objective to be sought.

   I said earlier that the value of an assessment depends partly on the use the person being assessed wishes to make of it. I found that one way of having the interpreter feel more involved in the process was by asking him beforehand what he thought his strengths and weaknesses were. The answers were often far more interesting than my somewhat limp question really deserved. Difficulties such as sudden changes of pace in the presentation (e.g. when the speaker starts quoting from a report or piece of legislation), long series of figures or references to acronyms, shifts in the level of language, non-native speakers of English or French (particularly in international meetings) were some of the points mentioned as areas where interpreters felt training or development courses would be useful.

   The final question I asked myself was whether the evaluation would be more useful if it were conducted by someone other than a senior interpreter. But who, for example? Perhaps by the team leader at one or more assignments. There are certainly advantages to this: a colleague would be far better equipped to assess the level of difficulty of the conference, the documentation provided, the quality of the sound and so on. However, there are problems: colleagues are reticent about assessing one another; a good team must be free of needless tension or disagreements; the criteria for assessment might vary from team leader to team leader. I think it would be difficult to implement such an approach.

   Another suggestion I have heard is to allow the customer to decide whether the service provided by the interpreter is satisfactory. I remember this was the suggestion and reaction of one interpreter to whom advice was offered on ways of enhancing his performance: "They like my work. Nobody has complained, and they often ask me to come back." I suppose an argument could be made that we provide a service, and the customer is always right. So, should we ask organizers at the end of each meeting to complete a form saying they were satisfied with the service provided? It is not an approach which holds much appeal for me since most conference organizers know nothing about interpretation, often thinking that the process is somewhat akin to Morse code as you transfer from one language to another. Really, they have no means of judging the accuracy of the interpretation and no yardstick for comparing the quality of one person’s work with another’s. I must say that I would feel ill at ease if I thought I were to be judged by someone who has not had the same training as I or my colleagues and understands little of the possibilities and limitations involved in the work. But I do think that more could be done to involve the people requesting and using the service, perhaps by sensitizing them to some of the requirements mentioned earlier. Interpreters provide a service, but to do so efficiently they need the cooperation of the people requesting the service. I suggest that it would be reasonable to ask the person requesting the service to provide a brief written explanation of the purpose of the meeting, some background information on previous meetings and decisions, and appropriate documentation. I don’t think this should be considered an excessive demand as the subjects that federal government interpreters deal with are broad in both range and number and yet often very specialised in content, and it is some small comfort to have an idea of what one is going to be talking about.

   As we have seen, the present system of evaluating interpreters at work has shortcomings. However, as with democracy, I think the alternatives are even less appealing. If I can draw one conclusion from the evaluation exercise, it is that the key to ensuring quality is very rigorous selection at the outset. I feel that if people know they have been carefully selected, that very sense of achievement ensures that they show pride in the job and thus seek to keep their skills finely honed. Therefore, for reasons of quality of work and customer satisfaction, I think it is essential that standards of recruitment and accreditation be maintained at very high levels. The people I evaluated were aware of having been carefully selected, and I found that they worked seriously and patiently in often difficult and trying circumstances. If I made suggestions regarding someone’s work, it was not because the work was bad but because the reputation of the profession depends on the quality of the people practising it, and it is very important they be good. And if you don’t want to take my word for it, just recall the advice St. Paul gave (no doubt with interpreters in mind) some two thousand years ago.

   Solikewise ye, except ye utter by the tongue words easy to be understood, how shall it be known what is spoken? For ye shall speak into the air.

   I Corinthians XIV: 9

© Public Services and Procurement Canada, 2024
TERMIUM Plus^®, the Government of Canada's terminology and linguistic data bank
Writing tools – Favourite Articles
A product of the Translation Bureau