Physician Accountability for Physician Competence: Summit V

The Alliance for Good Medical Practice

Related Information

The following materials are being provided as additional resources for participants of the December 5-7, 2007 Summit in Westlake, Texas. These materials may be helpful in providing background information. They may also provide some stimulating and thought provoking concepts. Currently there are materials related to:

Prior to each Summit since the first Summit in March 2005 we have been providing links to articles and documents as both background materials and thought stimulators. If you are interested in reviewing the resources provided for each Summit as well as the Discovery Workshop from this past August, please click here and use the link entitled: Online Resources Related to this Project.


Track 1: Sustaining the Effort

Introduction to Complex Adaptive Systems
In his introduction to Complex Adaptive Systems at the Discovery Workshop in August, Bryan Coffman presented key characteristics of systems, of adaptation and evolution, and of Complex Adaptive Systems.

Diversity and Leadership
At the Discovery Workshop in August, Norman Johnson gave a detailed presentation on the importance of diversity, the nature of leadership, and of key attributes of collective organizations

Attributes of Ecosystems
As part of his presentation at the Discovery Workshop, Norman Johnson reviewed key concepts from the presentation made by Lauren Buckley during a conference call that was held in preparation for the workshop.

Strategies of Ecosystems
At the Discovery Workshop in August, Norman Johnson presented a second talk in which he reported on his research with ant collectives and their strategies for finding food, and the implications of this work for organization design and governance.

Positive Deviance
Curt Lindberg discussed the process of positive deviance, a tool to help communities to find their top performers, learn the traits that make them successful, and then transmit those traits to the rest of the community to improve both individual and overall performance.

The Wisdom of Crowds
There has been a lot of research done recently on the principles of decision making and the capacity of individual experts in comparison with groups. The 2002 book by James Surowiecki entitled, naturally, “The Wisdom of Crowds,” provides an excellent overview of current thinking in this area. During the pre-workshop conference call led by Michael Mauboussin of Legg Mason, a number of key ideas were discussed, and during the workshop itself Norman Johnson summarized many of these.

Complexity and Healthcare
Bill Rouse discussed key elements of his work relating to complexity and healthcare, and discussed key aspects of leadership in complex systems undergoing rapid change.

Collective Decision Making
Marko Rodriguez of Los Alamos National Laboratory presented an overview of recent trends and findings in the field of Collective Decision Making Systems and Prediction Markets

Going to Scale
Kirsten Moy of the Aspen Institute discussed her work relating to increasing the scale of an organization, and leadership models that are presently being used in different types of non-profits and collectives.

The Structure of Good Ideas
Keith Sawyer • June 22, 2007
I’ve just been reading a fascinating new doctoral dissertation by a recent Ph.D. from Washington University’s Olin School of Business: Chihmao Hsieh, now a professor at the University 0f Missouri. He asks a fascinating question about innovation: what is the structure of successful new ideas? If that sounds abstract, I’ll try to explain by drawing on some of the research from my book GROUP GENIUS.

The See-Through CEO
Fire the publicist. Go off message. Let all your employees blab and blog. In the new world of radical transparency, the path to business success is clear.
By Clive Thompson • Wired Magazine Issue 15.04 - March 2007
Pretend for a second that you're a CEO. Would you reveal your deepest, darkest secrets online? Would you confess that you're an indecisive weakling, that your colleagues are inept, that you're not really sure if you can meet payroll? Sounds crazy, right? After all, Coke doesn't tell Pepsi what's in the formula. Nobody sane strips down naked in front of their peers. But that's exactly what Glenn Kelman did. And he thinks it saved his business.

META-LEADERSHIP AND NATIONAL EMERGENCY PREPAREDNESS: STRATEGIES TO BUILD GOVERNMENT CONNECTIVITY by Leonard J. Marcus Harvard University; Barry C. Dorn Harvard University; Joseph M. Henderson U.S. Centers for Disease Control and Prevention
The answer to that question could very well lie in what is introduced in this paper as “meta-leadership.”
The prefix “meta” as used here refers to overarching leadership that connects the purposes and the work of different organizations or organizational units. Just as “meta-research” refers to identification of broader themes and conclusions that emerge from a body of related investigation, and “meta-analysis” refers to a frame of reference that joins diverse thinking into a coherent framework, “meta-leadership” refers to guidance, direction, and momentum across organizational lines that develops into a shared course of action and a commonality of purpose among people and agencies that are doing what appears to be very different work.

Explaining the Wisdom of Crowds: Applying the Logic of Diversity
by Michael J. Mauboussin, Legg Mason Capital Management
Understanding diversity and leveraging its potential requires deeper understanding than we currently possess. We won’t get far with compelling anecdotes and metaphors . . . We need a logic of diversity.

Diverse Perspectives - How we See Things
Chapter 1 of The Difference by Scott Page
We all differ in how we see and interpret things. Whether considering a politician’s proposal for changes in welfare policy, a new front-loading washing machine, or an antique ceramic bowl, each of us uses a different representation. Each of us sees the thing, whatever it is, in our own way. We commonly refer to the ways we encode things as perspectives. But if asked what a perspective is, most of us would have only a crude idea. In this chapter I provide a formal definition, but before I get to that I’ll present an example of a famous perspective: the periodic table.

Leverage Points: Places to Intervene in a System
by Donella Meadows • Sustainability Institute 1999
Folks who do systems analysis have a great belief in 'leverage points.' These are places within a complex system (a corporation, an economy, a living body, a city, an ecosystem) where a small shift in one thing can big changes in everything.

The Power of Positive Deviance
by David R Marsh, Dirk G Schroeder, Kirk A Dearden, Jerry Sternin and Monique Sternin • BMJ VOLUME 329 13 NOVEMBER 2004
Identifying individuals with better outcome than their peers (positive deviance) and enabling communities to adopt the behaviours that explain the improved outcome are powerful methods of producing change


Track 2: Measuring Competencies

Welcome to LifeKnowledge®
The LifeKnowledge Center for Agricultural Education was created by the National FFA Organization as part of a strategic initiative to help agriculture students develop skills in leadership, personal growth and career success. The LifeKnowledge program offers a comprehensive approach to leadership training that is designed to reach every student in every classroom. Through the program, students learn the rewards of taking more active roles in life, work and their communities and how to make wise life choices now and throughout their lives.

Counting and Recounting: Assessment and the Quest for Accountability
Lee S. Shulman • From Change January/February 2007 • Accounting is essentially a form of narrative. Dina Shulman
When my daughter Dina returned from her first class in managerial accounting early in her MBA program, I innocently asked how it had gone. I fully expected her to describe her boredom with the rigors of accounting, since pursuing an MBA was decidedly an afterthought for my iconoclastic daughter, who already held degrees in theatre and social work.

Dr. Deming's Seven Deadly Diseases of Western Management

The 7 Deadly Sins of Performance Measurement [and How to Avoid Them]
By Michael Hammer
Operational performance measurement remains an unsolved problem. Despite the relatively little attention it gets in the management literature, designing and using metrics to track and improve operating performance is one of the most persistent problems that organizations face. In my interactions with companies in virtually every industry, I scarcely ever encounter one that believes it has an effective set of metrics for their operations: manufacturing, customer service, marketing, procurement and the like. To be sure, companies do have measurements for these areas that they employ every day, but few managers or staff believe that these metrics are the right ones or that they help the company improve its performance and achieve its strategic goals.

The Medical School Objectives Project (MSOP)
The Medical School Objectives Project (MSOP) is an AAMC initiative designed to reach general consensus within the medical education community on the skills, attitudes, and knowledge that graduating medical students should possess.

Coping with complexity: educating for capability
This is the last in a series of four articles
Sarah W Fraser, visiting professor, Middlesex University; Trisha Greenhalgh, professor of primary health care
Recent high profile scandals in the United Kingdom have highlighted the changing values by which the National Health Service is judged.1 The public expects, and the government has promised to deliver, a health service that is ever safer, constantly up to date, and focused on patients' changing needs. Successful health services in the 21st century must aim not merely for change, improvement, and response, but for changeability, improvability, and responsiveness.

Residents’ Perception of Effectiveness of Twelve Evaluation Methods for Measuring Competency
Kay K. Cogbill, M.D., Patricia S. O’Sullivan, Ed.D. and James Clardy, M.D.
OBJECTIVE: The authors assessed residents’ perceptions of techniques used to evaluate competency. METHODS: Psychiatry residents from a single program rated 12 evaluation techniques for their effectiveness to measure resident competency. They rated each method for 25 selected skills reflecting the six general competencies. RESULTS: Sixteen residents (70%) completed the survey. Responses indicated that different methods were effective for different competencies. Residents saw objective structured clinical examinations and standardized patients as equally effective. They favored the 360° evaluation method for all competencies except medical knowledge. Resident perception of the effective techniques differed from preferred techniques identified by measurement experts. CONCLUSIONS: Residency program directors using guidelines from the Accreditation Council for Graduate Medical Education (ACGME) to select preferred evaluations may find that residents do not have the same perceptions.

Measuring the Competence of Healthcare Providers
by Neeraj Kak, Bart Burkhalter, and Merri-Ann Cooper
This paper provides a framework for understanding the key factors that affect provider competence. Different methods for measuring competence are discussed, as are criteria for selecting measurement methods. Also, evidence from various research studies on measuring the effectiveness of different assessment techniques is presented.

Are specialist certification examinations a reliable measure of physician competence?
High stakes postgraduate specialist certification examinations have considerable implications for the future careers of examinees. Medical colleges and professional boards have a social and professional responsibility to ensure their fitness for purpose. To date there is a paucity of published data about the reliability of specialist certification examinations and objective methods for improvement. Such data are needed to improve current assessment practices and sustain the international credibility of specialist certification processes.

Physician Clinical Performance Assessment - Prospects and Barriers
Bruce E. Landon, MD, MBA; Sharon-Lise T. Normand, PhD; David Blumenthal, MD, MPP; Jennifer Daley, MD • JAMA. 2003;290:1183-1189.
The performance of physicians in their day-to-day clinical practices has become an area of intense public interest. Both patients and health care purchasers want more effective means of identifying excellent clinicians, and a variety of organizations are discussing and implementing plans for assessing the performance of individual clinicians. In this article, we review the current state of physician clinical performance assessment with a focus on its usefulness for competency assessment. We describe recommendations for a physician clinical performance assessment system for these purposes, and identify ways in which current methods of performance assessment fall short of these. We conclude that important technical barriers stand in the way of using physician clinical performance assessment for evaluating the competency of individual physicians. Overcoming these barriers will require considerable additional research and development. Even then, for some uses, physician clinical performance assessment at the individual physician level may be technically impossible to accomplish in a valid and fair way.

Evaluating physician competence
by Avedis Donabedian • Bulletin of the World Health Organization
What comes immediately to mind, in evaluating physician competence, is a jumble of methods that jockey for position and clash in what, at times, seems to be a battlefield of contending parties. We must return to first principles, if we are to understand this surface confusion and choose wisely from the many methods offered to us. And perhaps the most fundamental question to ask is: "Competence to do what? What are the objectives, and what are the preferred means for attaining these objectives?". I shall attempt an answer under three headings: the area of responsibility, the level of responsibility, and the means.

Physician Profiling, Guiding Principles
The AAFP believes physician profiling is an analytic tool that uses epidemiological methods to compare physician practice patterns across various quality of care dimensions (process and clinical outcomes). Cost, service and resource utilization data are dimensions of measuring quality, but should not be used as independent measures of defining quality care. The ultimate goal is to improve clinical outcomes.


Track 3: Infrastructure for Information Sharing

No Laughing Matter: An Open Letter to Google Health Marissa Mayer
by Scott Shreeve, MD
Congratulations on your new assignment to head up Google Health. This will certainly be a challenging task given Google's much anticipated entry into the Healthcare industry. I believe your own legendary success with product management and the user experience will be very useful as you launch. I wanted to be among the first to welcome you to the industry.

GoogleFirst Google Health Screenshots
Google Health, codename “Weaver”, is Google’s planned health information storage program. Google’s Vice President of Engineering Adam Bosworth lobbies for the program for quite a while now. Adam said the current US health care system is challenged when it comes to “supporting caregivers and communicating between different medical organizations.” Adam went on to say that people “need the medical information that is out there and available to be organized and made accessible to all ... Health information should be easier to access and organize, especially in ways that make it as simple as possible to find the information that is most relevant to a specific patient’s needs.” Adam adds that this – making information accessible – happens to be along Google’s mission.

Welcome to HealthVault
Imagine controlling the flow of your health information. Whether you need to search the Web for the most up-to-date treatments, catalog existing health records, receive test results, or monitor current physical readings — HealthVault gives you the control you need.

An infrastructure for seamlessly and securely sharing healthcare data
By E. Dellavalle, D. Cerri, D. Cerizza, A. Ghioni
This article is based on a presentation given at Med-e-Tel as part of the session on Electronic Records and Data Transmission

Massachusetts Health Data Consortium
Massachusetts SHARE (Simplifying Healthcare Among Regional Entities) is a regional collaborative initiative operated by the Massachusetts Health Data Consortium. MA-SHARE seeks to promote the inter-organizational exchange of healthcare data using information technology, standards and administrative simplification, in order to make accurate clinical health information available wherever needed in an efficient, cost-effective and safe manner.

Healthcare Services Bus (HSB): A pragmatic integration platform for healthcare services
by Raghu Havildar and Dr. John Hsu
According to 2007 Annual HIMSS Leadership Survey, among the top priorities for healthcare organizations are clinical automation and integration/standardization. Providers have been deeply involved in implementing EMR for the past several years and will be severely challenged to invest in new technologies and solutions. However, they see an acute need to integrate and make disparate systems interoperable. The healthcare enterprise is very disconnected. Applications and services were developed in silos. Even the few integration efforts are isolated, use non-standard and proprietary mechanisms leading to high sunk costs and inflexibility.