Quantifying unquantifiable: The outcome of a clinical case must be quantified to make it Successful

Incentive

The today’s efforts of system engineers to assure adequate and reliable performance of Medical Devices (MD) and instrumentation, not to mention the performance of the medical personnel, are, as a rule, unquantifiable. It is argued by this author that the successful outcome of a medical mission or a more or less typical clinical extraordinary situation cannot be expected, if it is not quantified, and that because of the various inevitable intervening uncertainties, this quantification should be done on the probabilistic basis. Nothing and nobody is perfect. In effect, the difference between a highly reliable and insufficiently reliable medical equipment/instrumentation, or between the performance of a highly qualified medical doctor and a mediocre physician is “merely” the difference in the level of their, actually, never-zero, probability of failure. It is important therefore that such a probability is assessed in advance and made adequate for the medical equipment and clinical tasks of importance. This probability cannot be high, of course, but, as far as a medical instrumentation or devices are concerned, does not have to be lower than necessary either: it has to be predicted and made adequate for a particular medical product and application. Devices that “never fail” are most likely “over-engineered”, i.e., are more robust than they could and should be, and, because of that, could be more costly than necessary.

In the recently published CRC book about modeling of an outcome of an aerospace mission or an extraordinary situation [1] it has been demonstrated how methods and approaches of the applied probability (see, e.g., [2]) could be effectively employed to quantify an aerospace mission outcome, with consideration of both the reliability of the electronic and/or photonic instrumentation and the role of the Human Factor (HF). In the mini-review that follows this effort is brought “down-to-earth” in application to reliability of medical electron devices and to performance of medical personnel, and particularly to the situations, when the reliability of the devices and the HF contribute jointly to the success of a medical mission or an off-normal and often urgent situation. Two major areas are distinguished and addressed in this mini-review, with an objective to quantify, on the probabilistic basis, what is usually viewed as unquantifiable: 1) the Probabilistic Design-for-Reliability (PDfR) of medical electronics (including the role and attributes of accelerated testing); and 2) the role of the HF in situations, when Mental Workload (MWL), Human Capacity Factor (HCF) and, when necessary and appropriate, also his/hers State-of-Health (SoH) are critical from the standpoint of the successful outcome of a clinical effort. The general concepts are illustrated in the referenced information by practical numerical examples.

Review

Probabilistic design-for-reliability (PDfR) of medical electronics: Here are some problems envisioned and questions asked in connection with the today’s practices in the medical electron device engineering (the reference numbers indicate the state-of-the-art in the particular area of importance and publications, in which at least partial answers to these questions and concerns have been given):

• Electronic MDs that underwent HALT (see, e.g., [3]), passed the existing qualification tests (QT) (see, e.g., [4]) and survived burn-in testing (BIT) (see, e.g., [5]) often exhibit nonetheless premature field failures. Are these methodologies and practices, and particularly the accelerated test procedures, adequate [6]?

• Do electronic industries need new approaches to qualify their products, and if they do, what should be done differently [7]?

• Could the existing practices be improved to an extent that if the product passed the reliability tests, there is a way to assure that it will satisfactorily perform in the field [8]?

• Could the operational (field) reliability of an electronic product, and, particularly, in medical electronics, where high reliability of devices is especially critical [9,10], be assured, if it is not predicted, i.e., not quantified [11,12]?

• And if such quantification is found to be necessary, could that be done on the deterministic, i.e. on a non-probabilistic basis [13]?

• Should MD manufacturers keep shooting for an unpredictable and, perhaps, unachievable very long, such as, say, twenty years or so, product lifetime or, considering that every five years a new generation of devices, including MDs, are developed and appear on the market and that such long time predictions are rather shaky, to say the least, should the manufacturers settle for a shorter, but well substantiated, predictable, trustworthy, physically and economically feasible, and, to an extent possible, assured lifetime, with an adequate anticipated probability of failure [14,15]?

• And how such a lifetime should be related to the acceptable (adequate and, if appropriate, even specified) probability of failure for a particular product and application [16,17]?

• Since understanding the reliability physics underlying the possible electronic materials and device failure is critical, and so is the accelerated testing in making a viable electron device into a reliable product, is there and alternative to, or at least a suitable modification of, the currently widely used highly accelerated life testing (HALT), a “black box” that supposedly improves reliability, but does not quantify it, even on a deterministic basis [18]?

• Is a highly focused and a highly cost effective failure-oriented-accelerated test (FOAT) the right accelerated life test [7,18-20] and the right extension and modification of HALT [21]?

• Considering that the principle of superposition does not work in reliability engineering, how to establish the list of the crucial accelerated tests, the adequate, i.e., physically meaningful, stressors and their combinations and levels?

• Boltzmann-Arrhenius-Zhurkov (BAZ) equation [22-29] was recently suggested as a suitable analytical model that could be used to bridge the gap between what one observes as the experimental FOAT data and what will most likely happen in actual operating conditions for the device of interest. What are the merits and the shortcomings of this kinetic model?

• The best engineering product is, as is known, the best compromise between the requirements for its reliability, measurable cost effectiveness and, also measurable, short-as-possible time-to-market [30]; but what about reliability and its appropriate quantifying characteristics? It goes without saying that to make any optimization possible, reliability of such product should also be quantified, but what is the simplest and the trustworthy way for doing that?

• Bathtub curve [31], the experimental “reliability passport” of a mass-fabricated product, reflects the inputs of two critical irreversible processes – the statistics-of-failure process that results in a reduced failure rate with time (this is particularly evident from the infant mortality portion of the curve) and physics-of-failure (aging, degradation) process that leads to an increased failure rate with time (this trend is explicitly exhibited by the wear out portion of the bathtub diagram). Could these two critical processed be separated [32]? The need for that is due to the obvious incentive to minimize the role and the rate of aging, and this incentive is especially significant for products like lasers, solder joint interconnections and others, which are characterized by long wear out portions and when it is economically infeasible to restrict the product’s lifetime to the steady-state situation, when the two irreversible processes in question compensate each other.

• A related question has to do with the fact that real time degradation is a very slow process. Could physically meaningful and cost-effective methodologies for measuring and predicting the degradation (aging) rates and consequences be developed? Could BAZ model [33] be applied to provide the quantitative assessment here?

• What is the possible role of analytical (“mathematical”) modeling predictive modeling [34]? If the predictions based on computer simulations and analytical modeling (these two modeling approaches are based, as is known, on different assumptions and used different calculation methods and techniques) agree, then there is a good reason to believe that the obtained data are sufficiently accurate and trustworthy [35].

• And how the above questions could/should be answered and the taken approaches modified and extended in application to the outer space engineering [36,37]?

HF and Its role

Human error (HE) (see, e.g., [38,39]) affects, to a greater or lesser extent, all aspects of human activity. Ability to understand the nature of various critical HEs and minimize the likelihood of their occurrence is of obvious practical importance. While considerable improvements in various medical and vehicular technologies and HF related missions and situations can be achieved through better ergonomics, better work environment, and other traditional and well established means that directly affect human behaviors and performances, there is also a significant opportunity (potential) for a further reduction in vehicular, medical and other possible HF related casualties through better understanding the role that various uncertainties play in the planner and the performer worlds of work. By employing quantifiable and measurable ways to assess the role of these uncertainties and by treating human-in-the-loop (HITL) as a part, often the most critical part, of the complex man-instrumentation-equipment-environment-object of control (patient, vehicle, piece of instrumentation, etc.) system, one could improve dramatically human’s performance, and to predict, minimize and, when possible and appropriate, even specify the probability of the occurrence of a never-completely-avoidable casualty. It is the author’s belief that adequate human performance cannot be effectively assured, if it is not quantified and, since nobody is perfect, that such quantification should be done on the probabilistic basis. In effect, as has been indicated in the previous section of this write-up in application to various instrumentations, both hard- and software, the only difference between what is perceived as a failure-free and an unsatisfactory human performance is, actually, the difference in the levels of his/hers never-zero probability of failure. Application of the quantitative predictive probability modeling (PPM) concept, which is analogous to the PDfR in medical and other instrumentation addressed in the previous section, should complement, in various HF related situations, whenever feasible and possible, the existing practices. These, as is well known, are mostly qualitative a posteriori statistical assessments.

The long-term HCF should always be considered vs. the elevated short-term MWL that the human has to cope with to successfully complete a critical task or withstand an off-normal (emergency) situation. It is argued that both traditional cognitive/Mental Workload (MWL) [40-50] and Human Capacity Factor (HCF) [51-72] should be considered, when quantifying the most likely outcome of a HITL related mission, medical case, or an extraordinary situation. The famous 2009 US Airways “miracle-on-the-Hudson” successful ditching [73] and the infamous 1998 Swiss Air “UN-shuttle” disaster [73] are good illustration to this statement. The input data in the publication [73] are hypothetical, but realistic, and it is the approach, and not the numbers, that is, in the author’s opinion, the major merit of the analysis. It attracted quite a number of references in the ergonomics literature. As the co-inventor of the calculus, the great mathematician Gottfried Leibnitz put it, “there are things in this world, far more important than the most splendid discoveries—it is the methods by which they were made.” It has been shown particularly that it was the exceptionally high HCF of the captain Sullenberger (“Sully”) and his crew that made a reality what seemed to be, at the first glance, a “miracle”. The highly professional and, in general, highly qualified Swiss Air crew exhibited inadequate performance (quantified in our analysis as a relatively low HCF level) in a much less challenging off-normal situation they encountered with. The Swiss Air crew made several serious HEs and, as a result, crashed the aircraft. In addition to the application of the suggested new Double-Exponential-Probability-Distribution-Function (DEPDF) based approach [62], it has been shown, using a well-known convolution approach in the applied probability [2], that the probability of safe landing/ditching can be evaluated by comparing the (random) operation time (that consists of the decision making time and the actual landing/ditching time) with the “available” anticipated, also random, of course, time needed for landing. A similar approach can be used, when evaluating, say, an outcome of a surgery, and such as effort is considered by the author at present as future work. The developed formalisms, after trustworthy input data are obtained (using, e.g., flight simulators [70] and/or by applying Delphi method (see, e.g., [2]) might be applicable even beyond the vehicular or medical domain and can be employed in various HITL situations, when a long term high HCF is imperative and the ability to quantify it in comparison with the short term anticipated MWL is desirable.

It has been suggested [51-72] that MWL vs. HCF is always considered as a suitable a way to quantify human performance. In the simplest case such a failure should be attributed to an insufficient HCF, when a human has to cope with a relatively high MWL. Our suggested MWL/HCF models and their possible modifications and generalizations can be helpful, after appropriate sensitivity factors are established and sensitivity analyses are carried out, in a number of critical cases, missions and situations: when developing guidelines for personnel selection and training; when choosing the appropriate simulation conditions (these, in the author opinion, should always be considered in any significant undertaking); and/or, in automated driving situations, when there is a need to decide, if the existing levels of automation and of the employed equipment (instrumentation) are adequate in off-normal, but not impossible, situations, and if not, whether additional and/or more advanced and, perhaps, more expensive equipment or instrumentation should be considered, developed, tested and installed, so that the requirements and constraints associated with a medical, vehicular, military, or other mission or a situation of importance that might be encountered, would be successfully coped with. Using an analogy from the reliability engineering field and particularly with the well known “stress-strength” interference model, the MWL could be viewed as a certain possible “demand” (“stress”), while the HCF - as an available or a required “capacity” (“strength”).

The MWL level depends on the operational conditions and the complexity of the mission, i.e., has to do with the significance of the general task, while the HCF considers, but might not be limited to, the human’s professional experience and qualifications, capabilities and skills; level and specifics of his/her training; performance sustainability; ability to concentrate; mature thinking; ability to operate effectively, in a “tireless” fashion, under pressure, and, if needed, for a long period of time (tolerance to stress); team-player attitude; swiftness in reaction, if necessary, etc., i.e., all the critical qualities that would enable him/her to cope with the high MWL. Note that adequate trust that is briefly addressed below is often also an important HCF.

It is noteworthy that the ability to evaluate the “absolute” level of the MWL, important as it might be for numerous existing non-comparative evaluations, is less critical in our MWL vs. HCF approach: it is the relative levels of the MWL and the HCF, and the comparative assessments and evaluations of their levels and likelihoods that are important. The author does not intend, of course, to come up with an accurate, complete, ready-to-go, “off-the-shelf” type of a MWL vs. HCF methodology, in which, as they say, all the i’s are dotted and the t’s are crossed, but rather intends to show how the powerful and flexible PPM methods and techniques could be effectively employed to quantify the role of the HF by comparing, on the probabilistic basis, the actual and/or possible MWL and the available or required HCF levels, so that the adequate, sufficient and quantified, preferably on the probabilistic basis, safety factor is assured. Note that testing on a flight simulator [67] and possible accelerated/preliminary testing in health care are analogous to the HALT and FOAT in electronics reliability engineering, including medical electronics.

Here how the major principles (“the ten commandments”) of our HCF vs. MWL approach could be summarized and formulated:

1. HCF is viewed as an appropriate quantitative measure (not necessarily and not always probabilistic) of the human ability to cope with an anticipated elevated short term MWL;

2. It is the relative levels of the MWL and HCF (whether deterministic or random) that determine the probability of human non-failure in a particular HITL situation;

3. Such a probability cannot be low, but need not be higher than necessary either: it has to be adequate for a particular anticipated application and situation;

4. When adequate human performance is imperative, ability to quantify it is highly desirable, is even a must, especially if one intends to optimize and assure adequate human performance;

5. One cannot assure such a performance by just conducting routine today’s human psychology based efforts (which might provide appreciable improvements, but do not quantify human behavior and performance; in addition, these efforts might be too and unnecessarily costly), and/or by just following the existing “best practices” that are not aimed at a particular situation or an application; the events of interest are certainly rare events, and “best practices” might not be applicable;

6. MWLs and HCFs should consider, to an extent possible, the most likely anticipated situations; obviously, the MWLs are and HCFs should be different for a jet fighter pilot, for a pilot of a commercial aircraft, or for a helicopter pilot, as well as for different healthcare related cases, performers and situations, and therefore should be approached, assessed and, if necessary and appropriate, even specified differently;

7. PPM is an effective means for improving the state-of-the-art in the HITL field: nobody and nothing is perfect, and the difference between a failed human performance and a successful one is “merely” in the level of the probability of their non-failure; this statement is true for practically any field of human activity;

8. FOAT on a flight simulator is viewed as an important constituent part of the PPM concept in various HITL and aerospace engineering related situations, but could and, perhaps, even should be considered and conducted for many healthcare related endeavors; such accelerated testing will certainly improve our understanding of the factors underlying possible failures; this effort might be complemented by the Delphi (experts’ opinion) effort;

9. Extensive predictive modeling (PM) and especially PPM is another important constituent of the approach, and, in combination with the highly focused and highly cost effective FOAT, is a powerful and effective means to quantify and perhaps nearly eliminate human failures in a number of critical missions and off-normal/extraordinary situations;

10. Consistent, comprehensive and psychologically meaningful PPM assessments can lead to the most feasible HITL qualification (certification) methodologies, practices and specifications.

Our HCF vs. MWL approach considers elevated (off-normal) random relative HCF and MWL levels with respect to the ordinary (normal, pre-established) deterministic HCF and MWL values. These values could and should be established on the basis of the existing human psychology practices.

As has been indicated, adequate trust [73-76] is an important HCF constituent. It is shown particularly [76], using the DEPDF based approach, that the entropy of this distribution, when applied to the trustee (a human, a technology, a methodology or a concept), can be viewed as an appropriate quantitative characteristic of the propensity of a decision maker to an under-trust or an over-trust judgment and, as a consequence of that, to the likelihood of making a mistake or an erroneous decision. Since Shakespearian “love all, trust a few” and “don’t trust the person who has broken faith once” and to the today’s ladygaga’s “trust is like a mirror, you can fix it if it’s broken, but you can still see the crack in that mother f*cker’s reflection”, the importance of human-human trust was addressed by numerous writers, politicians and psychologists. It was the 19th century South Dakota politician Frank Craine who seems to be the first who indicated the importance of an adequate trust in human relationships: “You may be deceived if you trust too much, but you will live in torment unless you trust enough”.

The analysis in [76] is, in a way, an extension and a generalization of the recent Kaindl and Svetinovic [75] publication, and addresses some important aspects of the HITL problem for safety-critical missions and extraordinary situations. It is argued that the role and significance of trust can and should be quantified when preparing such missions, including healthcare related (such as, e.g., surgical) missions. The author is convinced that otherwise the concept of an adequate trust simply cannot be effectively addressed and included into an engineering or a medical technology, design methodology or a human activity, when there is a need to assure a successful and safe outcome of a particular engineering or a medical effort or an aerospace or a military mission.

It has been shown, particularly [76], that the calculated entropy of the DEPDF for the random HCF, when applied to the trustee, can be viewed as an appropriate quantitative characteristic of the propensity of a human to an undesirable under-trust or an over-trust. Captain Sullenberger, the above mentioned hero of the miracle-on-the-Hudson event did possess such a quality. He “avoided over-trust”: 1) in the ability of the first officer, who ran the aircraft when it took off La Guardia airport, to successfully cope with the situation, when the aircraft struck a flock of Canada Geese and lost engine power, and took over the controls, while the first officer began going through the emergency procedures checklist in an attempt to find information on how to restart the engines; and 2) in the possibility, with the help of the air traffic controllers at LaGuardia and at Teterboro, he also “avoided under-trust”(as FDR has put it, “the only thing that we should fear, is fear itself”): 1) in his own skills, abilities and extensive experience that would enable him to successfully cope with the situation (57-year-old Capt. “Sully” was a former fighter pilot, a safety expert, an instructor and a glider pilot); that was the rare case when “team work” was not the right thing to pursue; 2) in the aircraft structure that would be able to successfully withstand the slam of the water during ditching and, in addition, would enable slow enough flooding after ditching (it turned out that the crew did not activate the “ditch switch” during the incident, but Capt. Sully later noted that it probably would not have been effective anyway, since the water impact tore holes in the plane’s fuselage much larger than the openings sealed by the switch); 3) in the aircraft safety equipment that was carried in excess of that mandated for the flight; 4) in the outstanding cooperation and excellent cockpit resource management among the flight crew who trusted their captain and exhibited outstanding team work (that is where such work was needed and was useful) during landing and the rescue operation; 5) in the fast response from and effective help of the various ferry operators located near the USS Intrepid museum and the ability of the rescue team to provide timely and effective help; and 6) in the good visibility as an important contributing factor to the success of his effort. As is known, the crew was later awarded the Master’s Medal of the Guild of Air Pilots and Air Navigators for successful “emergency ditching and evacuation, with the loss of no lives…a heroic and unique aviation achievement…the most successful ditching in aviation history.”

Future work

We would like to suggest several possible next steps (future work) that could be conducted using, when necessary, simulators to correlate the accepted DEPDF with the existing practice and to make this distribution applicable for the evaluation of the roles of the MWL and HCF not only to the general field of ergonomics science [77], in various HITL related navigation situations, including avionic [78], automotive driving [79,80], railway obstruction [81], and even outer space related missions [82-85], but to various medical electronic devices and critical health care tasks, missions and problems as well. These areas have a lot in common, as well as, of course, numerous differences, as well as quite a few critical specifics.

1. Suhir E (2018) Human-in-the-Loop: Probabilistic Modeling of an Aerospace Mission Outcome, CRC Press. Link: https://bit.ly/3mWezmD
2. Suhir E (1997) Applied Probability for Engineers and Scientists, McGraw-Hill, New York.
3. Suhir E (2005) Reliability and Accelerated Life Testing, Semiconductor International.
4. Suhir E (2019) Making a Viable Medical Electron Device Package into a Reliable Product, IMAPS Advancing Microelectronics 46.
5. Suhir E (2019) To Burn-in, or not to Burn-in: That’s the Question, Aerospace 6: 29. Link: https://bit.ly/2MhtjA3
6. Suhir E, Mahajan R (2011) Are Current Qualification Practices Adequate?, Circuit Assembly.
7. Suhir E (2018) What Could and Should Be Done Differently: Failure-Oriented-Accelerated-Testing (FOAT) and Its Role in Making an Aerospace Electronics Device into a Product. Journal of Materials Science: Materials in Electronics 29: 2939-2948. Link: https://bit.ly/3mVOpQY
8. Suhir E (2020) The Outcome of an Engineering Undertaking of Importance Must Be Quantified to Assure its Success and Safety: Review. Journal of Aerospace Engineering and Mechanics 4. Link: https://bit.ly/3o2W55y
9. Suhir E, Yi S (2016) Probabilistic Design for Reliability of Medical Electronic Devices: Role, Significance, Attributes, Challenges, IEEE Medical Electronics Symp., Portland.
10. Suhir E, Yi S (2017) Probabilistic Design for Reliability (PDfR) of Medical Electronic Devices (MEDs): When Reliability is Imperative, Ability to Quantify it is a Must. Journal of Surface Mount Technology (SMT) 30.
11. Suhir E, Mahajan R, Lucero A, Bechou L (2012) Probabilistic Design for Reliability (PDfR) and a Novel Approach to Qualification Testing (QT). 2012 IEEE/AIAA Aerospace Conf Big Sky, Montana. Link: https://bit.ly/3mXUOeo
12. Suhir E (2013) Could Electronics Reliability Be Predicted, Quantified and Assured? Microelectronics Reliability 53: 925-936. Link: https://bit.ly/2JvwC5E
13. Suhir E (2014) Electronics Reliability Cannot Be Assured, if it is not Quantified. Chip Scale Reviews.
14. Suhir E (1998) The Future of Microelectronics and Photonics and the Role of Mechanics and Materials. ASME J Electr Packaging (JEP) 120: 1-11. Link: https://bit.ly/37TEW8r
15. Suhir E (2008) How to Make a Device into a Product: Accelerated Life Testing It’s Role, Attributes, Challenges, Pitfalls, and Interaction with Qualification Testing. in E.Suhir, C.-P. Wong, Y.-C. Lee, eds., Micro- and Opto-Electronic Materials and Structures: Physics, Mechanics, Design, Packaging, Reliability 2.
16. Suhir E, Yi S (2017) Accelerated Testing and Predicted Useful Lifetime of Medical Electronics. Handlery Hotel, San-Diego, IMAPS Conf. on Advanced Packaging for Medical Electronics.
17. Suhir E (2013) Failure-Oriented-Accelerated Testing (FOAT), and Its Role in Making a Viable IC Package into a Reliable Product. Circuit Assembly.
18. Suhir E, Bensoussan A, Nicolics J, Bechou L (2014) Highly Accelerated Life Testing (HALT), Failure Oriented Accelerated Testing (FOAT), and Their Role in Making a Viable Device into a Reliable Product, 2014 IEEE Aerospace Conference, Big Sky, Montana.
19. Suhir E (2014) Failure-Oriented-Accelerated-Testing (FOAT) and Its Role in Making a Viable Package into a Reliable Product, SEMI-TERM 2014, San Jose, CA.
20. Suhir E (2010) Probabilistic Design for Reliability. Chip Scale Reviews. Link: https://bit.ly/3aM3r9H
21. Suhir E (2017) Probabilistic Design for Reliability of Electronic Materials, Assemblies, Packages and Systems: Attributes, Challenges, Pitfalls, Plenary Lecture, MMCTSE 2017, Cambridge, UK.
22. Suhir E (2019) Design for Reliability of Electronic Materials and Systems, in Holm Altenbach and Andreas Oechsner, eds., Encyclopedia of Continuum Mechanics.
23. Zhurkov SN (1965) Kinetic Concept of the Strength of Solids. Int J Fracture Mechanics 1: 311-323. Link: https://bit.ly/3aOgzuX
24. Suhir E, Kang S (2013) Boltzmann-Arrhenius-Zhurkov (BAZ) Model in Physics-of-Materials Problems. Modern Physics Letters B (MPLB) 27. Link: https://bit.ly/3pIRA0r
25. Suhir E, Bensoussan A (2014) Application of Multi-Parametric BAZ Model in Aerospace Optoelectronics, 2014 IEEE Aerospace Conference, Big Sky, Montana.
26. Suhir E, Bechou L, Bensoussan A (2012) Technical Diagnostics in Electronics: Application of Bayes Formula and Boltzmann-Arrhenius-Zhurkov Model. Circuit Assembly. Link: https://bit.ly/37VCUVC
27. Suhir E (2014) Three-Step Concept in Modeling Reliability: Boltzmann-Arrhenius-Zhurkov Physics-of-Failure-Based Equation Sandwiched Between Two Statistical Models. Microelectronics Reliability 54: 2594-2603. Link: https://bit.ly/2WUpcM9
28. Suhir E (2019) Failure-Oriented-Accelerated-Testing (FOAT), Boltzmann-Arrhenius-Zhurkov Equation (BAZ) and Their Application in Microelectronics and Photonics Reliability Engineering. Int J Aeronautical Sci Aerospace Research (IJASAR) 6. Link: https://bit.ly/38AZphq
29. Suhir E (2020) Boltzmann-Arrhenius-Zhurkov Equation and Its Applications In Electronic-and-Photonic Aerospace Materials Reliability-Physics Problems. International Journal of Aeronautical Science and Aerospace Research (IJASAR). Link: https://bit.ly/3mU3u5t
30. Suhir E, Bechou L (2013) Availability Index and Minimized Reliability Cost, Circuit Assemblies.
31. Suhir E (2015) Analytical Bathtub Curve with Application to Electron Device Reliability. Journal of Materials Science Materials in Electronics 26. Link: https://bit.ly/2L6Biz2
32. Suhir E (2014) Statistics- and Reliability-Physics-Related Failure Processes, Modern Physics Letters B (MPLB) 28.
33. Suhir E (2018) Probabilistic Design for Reliability (PDfR) of Aerospace Instrumentation: Role, Significance, Attributes, Challenges , 5th IEEE Int. Workshop on Metrology for Aerospace (MetroAeroSpace), Rome, Italy, Plenary Lecture.
34. Suhir E (2015) Analytical Modeling Enables Explanation of Paradoxical Situations in the Behavior and Performance of Electronic Materials and Products: Review. Journal of Physical Mathematics 07.
35. Suhir E (2016) Analytical Modeling Occupies a Special Place in the Modeling Effort, Short Comm. Journal of Physical Mathematics 7. Link: https://bit.ly/2L2bgNw
36. Suhir E (2020) Landing on Mars: Probabilistic Modeling Enables Quantifying the Last Six Minutes of Terror. Acta Astronautica 177. Link: https://bit.ly/3nWuaEj
37. Suhir E (2021) Astronaut's Performance vs. His/Hers Human-Capacity-Factor and State-of-Health: Application of Double-Exponential-Probability-Distribution Function. Acta Astronautica 178: 250-256. Link: https://bit.ly/3nYogSS
38. Reason JT (1990) Human Error, Cambridge University Press, Cambridge, UK. Link: https://bit.ly/3mU3Qcj
39. Bogner MS (1994) Human Error in Medicine, CRC Press, Boca Raton. Link: https://bit.ly/38G4wwY
40. Hancock PA, Mihaly T, Rahimi M, Meshkati N (1988) A Bibliographic Listing of Mental Workload Research. Advances in Psychology 52: 329-333. Link: https://bit.ly/3aXtNp0
41. Hamilton D, Bierbaum C (1990) Task Analysis/Workload (TAWL)-A Methodology for Predicting Operator Workload, Proc. of the Human Factors and Ergonomics Society 34-th Annual Meeting, Santa Monica, CA. Link: https://bit.ly/3hrVpUQ
42. Hancock PA, Caird JK (1993) Experimental Evaluation of a Model of Mental Workload, Human Factors: The Journal of the Human Factors and Ergonomics Society 35. Link: https://bit.ly/2MjBADH
43. Hollnagel E (1993) Human Reliability Analysis: Context and Control, Academic Press, London and San Diego. Link: https://bit.ly/37WdY0g
44. Endsley MR (1995) Toward a Theory of Situation Awareness in Dynamic Systems. Human Factors 37: 32-64. Link: https://bit.ly/3aQ0yok
45. Ericsson KA, Kintsch W (1995) Long Term Working Memory. Psychol Rev 102: 211-245. Link: https://bit.ly/3hq6fub
46. Reason JT (1997) Managing the Risks of Organizational Accidents, Ashgate Publishing Company. Link: https://bit.ly/3pxdHqg
47. Endsley MR, Garland DJ (2000) Situation Awareness Analysis and Measurement. Lawrence Erlbaum Associates, Mahwah, NJ. Link: https://bit.ly/3aOKYcp
48. Lebiere C (2001) A Theory Based Model of Cognitive Workload and its Applications, Proc. of the Interservice/Industry Training, Simulation and Education Conf., Arlington, VA INDIA.
49. Kirlik A (2003) Human Factors Distributes Its Workload. In E.Salas, ed., Advances in Human Performance and Cognitive Engineering Research 1.
50. Diller DE, Gluck KA, Tenney YJ, Godfrey R (2005) Comparison, Convergence, and Divergence in Models of Multitasking and Category Learning, and in Architectures Used to Create Them, In Gluck, K.A. and Pew, R.W. (eds.), Modeling human behavior with integrated cognitive architectures, Mahwah, NJ, Lawrence Erlbaum Associates. Link: https://bit.ly/3mZLRRQ
51. Suhir E (2009) Helicopter-Landing-Ship: Undercarriage Strength and the Role of the Human Factor, ASME Offshore Mechanics and Arctic Engineering (OMAE) Journal 132. Link: https://bit.ly/37SX3eT
52. Suhir E, Mogford RH (2011) Two Men in a Cockpit: Probabilistic Assessment of the Likelihood of a Casualty if One of the Two Navigators Becomes Incapacitated. J Aircraft 48. Link: https://bit.ly/37Vb6Aq
53. Suhir E (2011) Human-in-the-Loop: Likelihood of a Vehicular Mission-Success-and-Safety, and the Role of the Human Factor, Paper ID 1168, 2011 IEEE/AIAA Aerospace Conference, Big Sky, Montana.
54. Suhir E (2012) Likelihood of Vehicular Mission-Success-and-Safety. Journal of Aircraft 49.
55. Suhir E (2014) Human-in-the-loop (HITL): Probabilistic Predictive Modeling of an Aerospace Mission/Situation Outcome. Aerospace 1: 101-136. Link: https://bit.ly/2WTAEYs
56. Suhir E, Bey C, Lini S, Salotti JM, Hourlier S, Claverie B (2014) Anticipation in Aeronautics: Probabilistic Assessments, Theoretical Issues in Ergonomics Science.
57. Suhir E (2014) Human-in-the-loop: Probabilistic Predictive Modeling, its Role, Attributes, Challenges and Applications. Theoretical Issues in Ergonomics Science (TIES). Link: https://bit.ly/3mXXrwM
58. Suhir E (2014) Human-in-the-loop (HITL): Probabilistic Predictive Modeling (PPM) of an Aerospace Mission/Situation Outcome. Aerospace. Link: https://bit.ly/34Tv9gP
59. Suhir E (2014) Human-in-the-loop: Probabilistic Predictive Modeling, Its Role, Attributes, Challenges and Applications. Theoretical Issues in Ergonomics Science (TIES). Link: https://bit.ly/3rym1YF
60. Suhir E (2015) Human-in-the-Loop and Aerospace Navigation Success and Safety: Application of Probabilistic Predictive Modeling. SAE Conf. Seattle, WA.
61. Suhir E (2015) Human-in-the-Loop: Could Predictive Modeling Improve Human Performance? J Phys Math.
62. Suhir E (2017) Human-in-the-Loop: Application of the Double Exponential Probability Distribution Function Enables Quantifying the Role of the Human Factor. Int J Human Factor Modeling and Simulation 5. Link: https://bit.ly/3aRpyeO
63. Rosenfeld A, Kraus S, Brachman R, Stone P (2018) Predicting Human Decision-Making: from Prediction to Action. Morgan & Claypool. Link: https://bit.ly/3aQbi63
64. Suhir E (2018) Editorial, Quantifying Human Factors: Towards Analytical Human-in-the-Loop. Special Issue of the Int J Human Factor Modeling and Simulation 6.
65. Suhir E (2018) Human-in-the-Loop: Probabilistic Modeling of an Aerospace Mission Outcome. CRC Press. Link: https://bit.ly/3mX1tW8
66. Suhir E (2018) Aerospace Mission Outcome: Predictive Modeling, editorial, Special Issue Challenges in Reliability Analysis of Aerospace Electronics. Aerospace 5. Link: https://bit.ly/3aRi4sq
67. Suhir E (2019) Assessment of the Required Human Capacity Factor (HCF) Using Flight Simulator as an Appropriate Accelerated Test Vehicle. Int J Human Factor Model Simul 7. Link: https://bit.ly/2WS5fWl
68. Suhir E (2019) Failure-Oriented-Accelerated-Testing and Its Possible Application in Ergonomics. Ergonomics Int J 1-3. Link: https://bit.ly/3aOMdZ7
69. Suhir E (2019) Mental Workload (MWL) vs. Human Capacity Factor (HCF): A Way to Quantify Human Performance. in Gregory and Inna Bedny, eds., Applied and Systemic-Structural Activity Theory, CRC Press. Link: https://bit.ly/2MjDmol
70. Suhir E (2019) Assessment of the Required Human Capacity Factor (HCF) Using Flight Simulator as an Appropriate Accelerated Test Vehicle. Int J Human Factor Model Simul 7. Link: https://bit.ly/3aM7mmV
71. Suhir E (2021) Astronaut's Performance vs. His/Hers Human-Capacity-Factor and State-of-Health: Application of Double-Exponential-Probability-Distribution Function. Acta Astronautica 178. Link: https://bit.ly/3mXcLtM
72. Suhir E (2013) Miracle-on-the-Hudson: Quantified Aftermath. Int J Human Factors Modeling and Simulation 4: 35-62. Link: https://bit.ly/3hqGESb
73. Madhavan P, Wiegmann DA, ‘Similarities DA (2007) Differences Between Human-Human and Human-Automation Trust: an Integrative Review, Theoretical Issues in Ergonomic Science 8.
74. Hoff KA, Bashir M (2015) ‘Trust in Automation: Integrating Empirical Evidence on Factors that Influence Trust. Hum Factors 57: 407-434. Link: https://bit.ly/38MkV2U
75. Kaindl H, Svetinovic D (2019) Avoiding Undertrust and Overtrust’, in Joint Proceedings of REFSQ-2019 Workshops, Doctoral Symp., Live Studies Track and Poster Track, co-located with the 25th Int. Conf. on Requirements Engineering: Foundation for Software Quality (REFSQ 2019). Essen, Germany.
76. Suhir E (2019) Adequate Trust, Human-Capacity-Factor, Probability-Distribution-Function of Human Non-Failure and its Entropy. Int J Human Factor Modeling and Simulation 7. Link: https://bit.ly/37TjzEu
77. Suhir E, Paul G (2021) When Instrumentation and Human Performance Contribute Jointly to the Outcome of a Human-System-Interaction (HIS) Mission, 21st Triennial Congress of International Ergonomics Association - IEA, Vancouver, Canada. 
78. Suhir E (2020) Risk-Analysis in Aerospace Human-Factor-Related Tasks: Review and Extension. Journal of Aerospace Engineering and Mechanics (JAEM) 4. Link: https://bit.ly/3aQ2z3S
79. Suhir E, Paul G (2020) Avoiding Collision in an Automated Driving Situation, Theoretical Issues in Ergonomics Science. Link: https://bit.ly/3nXhH34
80. Suhir E, Scaraglini S, Paul G (2020) Extraordinary Automated Driving Situations: Probabilistic Analytical Modeling of Human-Systems-Integration(HSI) and the Role of Trust. AHFE, San-Diego, CA. Link: https://bit.ly/2L3Y6iY  
81. Suhir E (2020) Head-on Railway Obstruction: Probabilistic Model. Theoretical Issues in Ergonomic Science. Link: https://bit.ly/3nSaDVk
82. Suhir E, Karwowski W, Bedny I, Paul G (2021) Some Major Human Issues in Aerospace Engineering: Review and Extension, 21st Triennial Congress of International Ergonomics Association - IEA Vancouver, Canada.
83. Salotti JM, Hedmann R, Suhir E (2014) Crew Size Impact on the Design, Risks and Cost of a Human Mission to Mars. 2014 IEEE Aerospace Conference, Big Sky, Montana. Link: https://bit.ly/2KAMWmh  
84. Salotti JM, Suhir E (2014) Some Major Guiding Principles for Making Future Manned Missions to Mars Safe and Reliable, 2014 IEEE Aerospace Conference, Big Sky, Montana. Link: https://bit.ly/38M4Z0u
85. Salotti JM, Suhir E (2014) Manned Missions to Mars: Minimizing Risks of Failure. Acta Astronautica 93: 148-161. Link: https://bit.ly/3nWdtsl