To begin an improvement effort, an existing HMI should be assessed against High Performance HMI best practices. The assessment of HMI performance is much more of an evaluation process, “Does my HMI meet all of these various criteria?” than a numeric or statistical analysis process, “What percentage of the surface area of my displays is blank space?” For us engineers, subjective evaluation is usually more difficult than recording data and processing formulas. While there are some measurable quantities factoring in to an HMI assessment, they do not dominate the assessment as they do when evaluating an alarm system’s performance.
Even so, there is a logical process to follow for HMI assessment. The determination is primarily based on observation of operators, discussion, questionnaires, and checklists. Herein, we have adapted a method put forward by the United Kingdom’s Health and Safety Executive (HSE). In using it, you will arrive at one of 5 grades (A, B, C, D, or F), a system which is familiar to most. Here is an overview of the method (Appendix 2 contains the full details and checklists to use).
Methodology Overview:
Beginning Score: F
Satisfy Criteria 1, 2, and 3
New Score: D
Satisfy Criteria 4
New Score: C
Satisfy Criteria 5
New Score: B
Satisfy Criteria 6
New Score: A
HMI Evaluation Methodology
A final performance grade is determined by starting at the bottom grade and working upwards. To advance to the next higher grade, defined criteria must be met. If all of the criteria are not met, the grade is established.
For each grade level, there are certain best practices the HMI must follow. These include HMI design, certain aspects of console design, the control room environment, and control room operating and managerial practices. These are determined by comparison to the detailed checklist in Appendix 2.
A Failing Grade: “F”
“No more TV or video games or hanging out with your friends for you anymore, young man! You’re going to study every night until your grades come up!”
All of these first three criteria must be satisfied to move beyond an “F”.
Criteria One: Operators find it easy to keep track of the process in normal conditions. Potential problem areas which must be addressed to achieve this include:
insufficient process information
unreliability of sensors or displays
other tasks operators are required to perform
distractions, such as maintenance permit writing and telephone calls
Criteria Two: Information about the process and plant condition is adequate for operators to be confident they can effectively monitor abnormal or upset conditions. The critical measure of an operator information system is how well it performs when the demands are greatest – in an abnormal situation.
Criteria Three: During an abnormal, upset, or emergency condition, the operators can keep track of the process using only the HMI, without a burdensome and distracting need to gather relevant information from multiple control room displays, logbooks, procedure manuals, etc.
It is quite common to see operators working from multiple screens or consoles, or paging through several displays or manuals to gather and analyze information, and then take action and monitor results on yet other displays. Remembering to return to a particular display to check on one critical point can be cognitively demanding, especially when under the added pressure of an abnormal situation.
Not Quite Failing – a “D”
“Do you think this is acceptable? Is this the best you can do? We expect far better from you than this! And for now, you’re still grounded!”
A “D” is thus achieved if the operators are always presented the proper information to monitor the condition of the plant in a timely, accurate, and reliable manner – regardless of upset or emergency conditions. To move to a “C” requires more.
The focus now shifts to providing a distraction-free environment. The ultimate goal is a high level of situation awareness (an accurate understanding of the condition and behavior of the plant), which can only happen in a distraction-free environment. To achieve a “C” level, the following must be true.
Criteria Four: During abnormal situations, the unit’s managers, engineers, and supervisors must not interrupt the operator by manipulating the operator’s displays, or even worse, making their own adjustments to the process.
Still Not Good Enough – a “C”
“You got a ‘C’? How do you possibly expect to get into college? Or get a scholarship? Do you think you can live here with us forever when you finish school? Well, you’ve got another think coming! I want to examine your homework every night and no TV until it’s done!”
A “C” is thus achieved when, in upset and emergency conditions, all relevant operators and supervisors can accurately and reliably assess the condition and behavior of the plant within the available time without disturbing each other or blocking each other’s access to information. To achieve a “B” level, the following must be true.
Criteria Five: Non-operating tasks are not required of the operator during upset or abnormal conditions. Whenever there are critical process activities demanding the operator’s attention, there must be no unnecessary tasks, duties, or disturbances.
Once again, the most frequent distraction we find is the telephone, which clearly rings most often during a unit upset. When the surgeon is in the middle of your heart operation, you don’t want him leaving every 5 minutes to discuss his progress with your in-laws!
Finally – a “B”
“Well, that’s more like it! Now you have a chance for a decent future! You’re not grounded anymore. Keep applying yourself because we know you can do even better!”
A “B” grade is thus earned only if the operators have absolutely no responsibility other than monitoring the process during times of abnormal or unusual conditions. This also applies to startups and shutdowns, if the process is normally a continuous one and they are infrequent. There must be a high level of continuity in the operator’s tasks during critical process events. Operators are not required to perform tasks that significantly disrupt their concentration on the process and they are able to delay other activities to minimize distractions.
To achieve an “A” requires the following.
Criteria Six: The HMI must meet all of the major criteria contained in the assessment checklist in Appendix 2 regarding:
General graphic factors
Navigation factors
Workstation factors
Control room and work practice factors
Alarm management factors
All Right! An “A”
“Great going! You know, we’ve been thinking that maybe you’ve been showing enough responsibility that we can talk about getting you that car…”
The “A” grade is reserved for those units with sound operator information strategies for routine as well as abnormal situations, a distraction-free environment, and minimal disruptions from the primary task of monitoring the process unit. The principles of a High Performance HMI are adopted, yielding an HMI specifically designed for effectiveness in both normal and abnormal situations.
Periodic Reassessment
A High Performance HMI is part of a continuous improvement program. Even with high knowledge, diligent effort, and skill, every situation cannot be anticipated. There should be an on-going system of operator feedback on HMI effectiveness and recommended enhancements. The HMI should be under appropriate Management-of-Change (MOC) control so all changes are in accordance with the HMI philosophy.
At most companies, production upsets, incidents, and accidents receive an internal review. Part of the review of such events should be to assess the effectiveness of the HMI during the event.
Disturbing Moments in Cinematic Human-Machine Interfacing:
Young John Connor: “You can’t just go around killing people!”
Model T-800 Terminator: “Why?”
Terminator 2: Judgment Day