Project Teams
Michael Kaufman: Any wild thoughts that you woke up with this morning that you want to share here?
- President Obama won the Nobel Peace Prize
We bombed the moon
MK: Anything about our work here?
- The research topics for patient outcomes are in general comparisons of whether certified nurses avoid bad things or do good things more frequently than non-certified nurses. That’s really for the certifying boards. The content of that is indistinguishable from stakeholder value. We can just say that’s the way it is. This might be something for us to look for in the report outs.
MK: Okay then let's get started.
Report Out
Team 10 - Do certified nurses have better critical thinking skills
I was noticing that the brain guy was at least 25 years old.
We looked at certified nurses and the difference between non-certified nurses in terms of critical thinking. If you look at our diagram here you see here the overlap of different areas. We also came up with a definition.
The objective of this was to conduct a preliminary study to determine if there is a significant difference between the CT skills of certified versus non-certified nurses. We want to make our linkages between the work environment and certified practitioners. Our stats would be ANOVA and descriptive. We want to use Kolb’s adaptive competency acquisition tool.
Our partners would ABSN member organizations, practice settings, and major conferences for specialty groups. We don’t want to be too homogeneous.
Our plan is to review the literature, look at our sampling power and refine our definitions, tease out our research questions and do our analysis. We will have to use the IRB and we all know how that works.
We need to look at implications for further research. We thought this might be the first step if there is a difference and then we can take it further such as Jim said early about adverse outcomes. If we look at our resources we figure we need a statistician, research assistant, Kolb tool, situational scenarios, in kind time, grant writer and a transcriptionist. We could potentially use a tool that works for sample across the specialties.
Comments
Q: Are these voluntary certifications?
A: Yes
There is always another approach. At the point someone commits, things dramatically change. When you want to keep it clean it takes some staff development. We need to tap into that level of enthusiasm but when it comes down to it, you have to let people who have the experience to develop it.
I might fund a doctoral student to do this down the road.
We need to find the enthusiasm if we’re going to really back this up. If we’re going to do it, it has to get real.
Q: When you’re designing your scenarios, we have to be consistent if we want a general scenario or for the specialties. Would they each write their own scenario?
A: We did talk about this. If you want to twist this project, you can do it for the specialties so they recognize their environments.
Why would expect that certified nurses have more critical thinking than non-certified? Some component of that has to be built in to the certification process. A better strategy for a favorable outcome is to put resources towards the educational and learning process. I’d invest the resources towards that into the current process. The critical issue here is prior to this study. The focus on certification is under the assumption is that it may measure a number of different things. Some of the focus should go on the assessment process itself. That’s what guarantees the outcomes.
You could do feasibility study or a pilot study with one organization to work out the bugs before it gets sent out to everyone else. You may want to have a nice size sample here. We need to start somewhere. I have the hypothesis that certified nurses have better critical thinking skills but we need to show that. I don’t think we need to do intervention work until we know where we stand descriptively.
There is a critical thinking part of the exam process. It is in the blueprint. Why would someone participate in this if their not certified? Do we need to look at this with more variables? Maybe having higher education experience provides critical thinking and we can take it out of the us/them situation.
We talked about having a pilot study and how we need to package the questions so that we don’t create an us/them scenario.
There is a whole body of literature out there about education and its impact on critical thinking.
The discussion is very enlightening. What we’re trying to do here is to find out where the interest is. If we have critical thinking as one of our wings in the blueprint then we need to incorporate it in our work going further. So I think we can just say that it merits moving forward in our research agenda.
We could link critical thinking from a blueprint to an actual test they take. That would be a phenomenal accomplishment.
We need an exploration of what critical thinking is to guide the work. I would start with the blueprint but there can be a substantial difference between that and its specificity. We need to move it from the blueprint to the certification process. That needs to be clear. Everything needs to be guaranteed.
I came to this and said ‘so what’ if our nurses have critical thinking. How does it impact the care and adverse events?
We may learn something along the way and for this to be successful; the nursing profession has to develop some critical thinking to see that it isn’t there and isn’t measured right now.
Entry-level exams tests critical thinking.
Why I don’t think it needs to move forward here is that I think it would be a good idea but doesn’t belong at a general level. It could be in the specialties.
My concern is that this is the first presentation and I’m not sure we can do it with all the other things we have to do.
Team 1 - Certification and Symptom Relief
Symptom relief depends on which specialty you’re in. We looked at a couple of different questions, for example: do patients experience less interdialytic hypotension when cared for by a certified nurse? As well as looking at it from a more general perspective.
We identified our methodology and our partners. It will depend upon whether we do a simulation or another method. We would need partnership with member organizations, a statistician, research assistants, and a lot of money for things like travel and IT.
Some things that need to be considered are controls for education level, demographics of nurse and patients, facility and experience.
We made a rough timeline which just has the order of how things need to occur but not with dates.
Comments
There are already developed tools that can be used. Even less experienced researchers could do this. It could have an interesting outcome and we could take advantage of that.
It seems that we’re losing a critical piece about what is really important for nurses.
There is a study about oncology nurses and symptom relief. There were some good outcomes there. They looked at charting and how often the team came together. This could be useful.
There was a study done on continence management in adolescents and there was some trending between certified and non-certified nurses, so there are some data out there that we can start piggybacking on.
You have to be careful with how you craft your questions. There are many factors that influence the outcomes.
I like this because it’s very nurse-y. It seems that this would have to be tied to other studies.
Team 4 - Effect of certification on failure to rescue
Our definition is the failure to notice symptoms or respond adequately or swiftly enough to clinical signs when a patient is dying of preventable complications in a hospital.
Our objective is to demonstrate the negative correlation between proportion of certified nurses and rate of failure to rescue. Do specialty differences exist? Can we reduce costly risk management?
We would implement this by a retrospective chart review of code patients, and we would determine specific pre-code interval. We would look at the nurse responsible for that patient and determine whether they were certified or not. Or we could do this by simulations. There has been some work done in this area by Aiken.
We would partner with IHI and get information from the situations. We would get information from Aiken’s database so we wouldn’t have to duplicate the data.
In our research plan we assume that we would have great collaboration and access to charts for review. For our timeline we would develop simulations based on most frequently encountered scenarios in chart review concurrently.
We would test our certified and non-certified nurses by partnering with ABNS and ask them to do simulation testing. We would compile and analyze the data and publish brilliant papers from the research. We would receive consultations from psychometricians.
An end product we would have from this is having unique item types for certified exams and materials to develop training materials.
If the simulation results to not agree with the results of chart review, we would review all the components of the simulation document, administration, scoring and do a re-test.
This project needs some re-working and we advise that we go forward with it to do some more work.
Comments
Q: Should we continue this project?
A: I think so but it really needs more work.
If something like this were to occur and we did a chart review and it was shown that certified did a much better job with all things being equal. Then if we did a simulation and it came out with other results, maybe it leads to other questions. Maybe we have to look at this in a completely different way.
You could do things at the unit level. I would want to look more deeply at this issue because this is very controversial. I’m wary of this if it has issues.
It’s a questionable indicator particularly because this is a rare event.
I know a number of people have pointed to the issue of certified and non-certified and looking for differences. Maybe we could look at variations. It may be more reasonable to think that we can find what we’re looking for there. Maybe the variation is smaller than we expect and we might find some things there.
Usually this situation revolves around a bigger team than just the nurse.
Team 5 - Impact of certified nurses on adverse events
We looked at adverse events and my group did a good job with the task we had and we weren’t all in agreement but we got a lot done. First we had to identify what is in the adverse event bucket. This is the list we came up with. Most of this information is gettable.
The non-pc title is do units with a higher proportion of certified nurses have fewer adverse events. And what are the costs of certified nurses in prevention of adverse events.
The first thing you need is a strong principle investigator because you need a lot of money and then you can go after funding. We looked at the controls and determined the number of certified nurses and adverse events. A big debate began about how some specialties are better able to prevent adverse events. If so, we need to get that data.
We’ll be looking at education, specialty, and years of experience among others. How can we examine the knowledge question? Maybe a nurse fails a question on a test but actually has the knowledge. There might be a tipping point if we had good enough data.
There is a time line shown here. We figured that it would take six months groundwork and finding PIs. Then we develop a proposal and get funding. From there we get in front of the Institutional Review Board and start doing data collection which takes a year and another year of data analysis.
Comment
From a layman’s perspective, this is cool. This is something a lot of stakeholders care about. This has got me the most excited. This also gets us away from the us versus them scenario too. This concept is cool. I think the funding resources could be tremendous.
I would make it percentage of adverse events.
The national group has published indicators and we need to look at things that are already shown to be nurse-sensitive.
At first I thought this doesn’t apply to me, it may not be measurable in my specialty but we’re paid to analyze adverse events and the more ways I have for looking at it can be really helpful.
Team 11 - Impact of certification on patient/family education
Our main question is: Do patients cared for by certified nurses have a higher knowledge level of their health status and self care needs than those cared for by non-certified nurses?
We need to link to patient outcomes and our research partners are healthcare organizations, patients, nursing specialty organizations and employers. Our methodology is a pretest when possible and a post test once all is complete. We’ll do a comparison of the post-test scores of patients cared for by certified nurses versus non-certified nurses. We need to include self-care behaviors as well. We realize there might be children included and other people who could benefit from this.
We set a timeline for about four years unless of course we can get a really large grant. We would look for grant writing and identify specialties and conditions to participate. We would need to align with research partners, do the IRB protocol refinement, do patient recruitment and collect the data. We would evaluate it and then publish.
Comments
I have system issues. If I’m understaffed I don’t really want to do this. I see a lot of issues with this one.
We didn’t think this was realistic to do this on an in-patient basis.
This has a ‘so what’ aspect to me. So what if the patient has self-knowledge.
We would probably get more bang for our buck if there were more elements that we could control.
There are very different settings depending on whether people are going through surgery or not. This is very difficult to measure.
Maybe we could just look at admission rates.
There is good data out there for the heart failure cases in terms of readmission and trips to the emergency room. We had to look at something we could develop.
Disease management has become a for-profit business in the U.S. so maybe we could take advantage of the call centers and we could impact their costs.
That’s probably true for the diabetes community but it will be unethical in terms of selecting nurses who are not certified or those we think are less-qualified.
We could deliver the same information, but style and ability to teach is also a skill-set.
Team 2 - Recertification
We combined four projects into one project. We had several objectives to look at. We want to consider the links to nursing licensure. WE want to identify the feasibility of a multi-modal approach. Should we look at CE practice or simulations, etc.?
We have a very ambitious list of objectives. Originally we titled this "best practices". We want o show the linkages with the NOCA standards.
Firstly we want to build upon the work of NOCA and to add to the benchmarking standards. What we found in here we would project what are the best practices for nursing. We would go back and identify what is the best for the multi-modal approach. Some of this could be part of the certifying test. We would ask the question what a certified nurse should do in order to craft the education and training.
We would recruit graduate students and train them to help with this research. We would look at the psycho-motor, cognitive and affective domains. We would use the research committees.
We considered sectioning out base-practice nursing. Our timeline is very ambitious because we are going to get funded and pay our research team. People get things done much faster if they’re getting paid. (laughter) We will identify the project team.
The feasibility study will be done in terms of what needs to be included. We will have a common site and tie into the organizations to get the emails.
This is something that could be done in an expeditious way and could bring the foundations together to help decide what certification should look for.
Comment
Many of those variables are something that my organization is putting together with 2000 of our general practitioners. We get over 95% of participation. We have an iterative process with a self-assessment exam and other options and we’re getting ready to publish this.
Team 9 - Recertification
Ours is very similar to the previous group. Our title is identifying the strengths and weaknesses of different recertification methods for assessing continuing competency assessment.
We want people to see notes and the portfolio method is not so good if you have to recertify 2000 people a year but here there are pros and cons. Our objective is the development of a toolkit of pros and cons of different methods to assess specific aspect of continuing competency.
Our method would include the literature review. There are some studies that have already done some of this. Probably it would have to be updated but we could leverage a lot of that work. We would compile and disseminate the toolkit with ANS.
Our ultimate goal is to estimate the reliability, credibility and validity of different methods for nursing continuing competency certification. This is a bit pie in the sky work but maybe it could stretch us to better results.
We would define our gold standard when it comes to competencies but this is a long-term goal. Right now we see that people need to have a toolkit to help to decide where to go.
Comments
There have been speakers recently which show certain aspects that are invalid such as that as a professional can’t know what they don’t know. It would be helpful to have valid and reliable tools. There are several organizations which could benefit from a toolkit. Some of the organizations are not invested in what there is in play.
Those of us who have been involved in the alliance know that one of the proposed requirements is certification. This is a philosophical concept that we determine what the best practices are for recertification. We can provide this as they go forward with their regulation models.
Is this truly research or is this the development of a toolkit? This could change the funding model and it might be differently placed on the research agenda.
The concept piece is not quite clear. I’m not sure this is the right strategy. How does this connect with the current testing environment and expectations? I’m absolutely for the concept but not sure how we should get there.
One of the things our NOCA study found is that when people work in any field they tend to specialize. Those informal specializations play into how people perform on an entry-level position. We need to address that particular need in the assessment profile.
Team 6 - Value of certification by public/consumer 1
We looked at what the public values. We wanted to determine if the public recognizes what professional certification means. Can the public identify the impact of a certified professional? One of the recommendations we got is that the patient does not capture what we want to measure. This would be a descriptive study done with interviews and public surveys that could be used for tool development.
In terms of research partnerships, we would look to ABNS and the Gallup poll. We would look only to places where certified nurses were employed. The timeline is fairly ambitious from the review of literature to making a report would take about a year and a half.
The required responses from a people standpoint is that we would need a tool, subjects and administrators giving us approval. We have the roles and players laid out in this chart. Our overall point of contact would be ABNS.
Comments
In determining who the public is it might be better to title it as “patient”.
When we first put it together we identified patient with public, but we see now that would have to be distinguished.
The general public thinks that nurses should be certified.
If you really do what to get patients to do a Gallup poll you can just ask Gallup to add a few questions to a poll they are doing already.
This will also tell us whether the public even notices whether someone is a certified nurse or not.
Team 7 - Value of certification by public/consumer 2
We are very similar to the previous team, so I will just highlight the differences. We identified the consumer is the public. Our objective is to determine consumers’ level of knowledge about nursing certification and to determine value to consumers and how it impacts their care and healthcare experience.
We would link to the CAC/David Swankin work. You may be aware that ABMS did a Gallup poll in 2005 and it was surprising to find that the patients considered it very important that their doctors were board certified but very few of them actually knew what that meant.
We wanted to tap into some private sector studies such as by Johnson & Johnson who care about nursing and have them help us access more of the public community through places like Ladies Home Journal, the Readers’ Digest for more publicity. We thought about having people do some rank ordering of priorities in terms of how important it is for your doctor, nurse or car stereo installer to be certified.
We want to take advantage of previous studies, find funding from a variety of sectors, private, public, government and nursing associations, and develop the surveys.
Comments
I think you’ve finally hit on a good way to really distinguish what the public knows about certification. Of course doctors and nurses are already licensed and that certification is something extra. This is the information we really don’t communicate. We want to know how important it is that a practitioner has gone through this extra work to become certified.
One of the suggestions is a marketing and education plan. That resonates with me. This isn’t really about research but this is an important topic for ABNS.
I think that’s right that the public doesn’t know about certification but I find also that nurses don’t know that either. In a survey I’ve seen there were a surprisingly low number of nurses who really understood what that meant. I think this is a marketing scheme, but where it does fit into research is that you need to explore what it means for them. When we ask well-placed questions before the certification ones, we get better results.
The ANCC has put up a list of certifications and it’s a monster with over 200 options. It is complex in understanding the difference between a certificate and a certification. You have to give them instructions about who is certified.
We need to have a consumer-focused campaign. The little bit we have done for consumer awareness takes a lot of effort. It is a multi-year project. We have to decide whether we’re talking about nurse-awareness or consumer-awareness.
I don’t think it is an either/or. When you get consumers down that line educated about understanding the significance of certification it becomes much easier because they start asking for certification in the professionals they need.
We could capture a lot of this by having this as part of a regular curriculum. We could just have a quick PowerPoint about this.
I can speak to that as an educator and I can tell you that even the seniors are not ready to hear about this. It needs to go into the hospitals.
Team 3 - Value of certification by employers
We want to demonstrate the ROI for hiring and retaining specialty certified nurses. ROI is determined by the length of stay for patient, the risk reduction for malpractice among others.
If the employer finds there is more return on investment then employers will want to hire more certified nurses. There is a tendency to hire more certified nurses but we don’t know why they do that. Recently the Veterans Administration has put an emphasis on hiring certified nurses but we don’t really know why. It would be good to know.
We want to look at how certified nurses interact with patients to effectively use resources. One of the groups that came in to identify hospitals or organizations that started doing this five years ago and calculate the ROI from that. We could also look at Magnet hospitals and compare them to their pre-magnet state.
Comments
It is hard to calculate this because it’s difficult to track the ROI based on certified versus non-certified nurses.
This is important to do this for benchmarking purposes. I always get frustrated with this in terms of certification boards. The incentives are questionable. This could create a compendium of these examples. The VA will not only pay for the certification process but even sometimes provide stepping pieces for the process. It would be good as an administrator to know what’s going on.
There is a large push now because of all the veterans who are coming back and there is a push to have more of the hospitals be Magnet hospitals. There is more integration with the private sector. Part of this is to put them further up in the quality care. The incentive is built in and the Magnet status is to help move this forward. We have a huge research base that’s targeting nurses and it will be interesting to see what comes out in a couple of years.
The VA story is good to look at. They linked certification to length of stay. The VA has evidently bought into the idea of hiring certifications. If we’re going to get more nurses certified it would be good to know what their process was.
I think there are other studies that are more scientifically framed.
If we put this with the research agenda, how many of you think we should now put it as part of the research agenda.
Team 8 - Defining competence relative to IOM
Our objective was to establish definitions of competency and continuing competencies for specialty nursing. Our research method is a review and synthesis of the current literature which would include your own organizations and how you define competency as well as drawn upon the IOM.
We want to abstract out of this the elements of the survey results for continuing competency. We would include nurses, employers, patients and consumer, regulators and educators.
We had interesting feedback on our first thought of a three-year timeline. We focused on doing the up front work including the development of methodologies. We did get some feedback that this was really bigger in scope than would be required. Maybe we could eliminate stage 3 or both 2 and 3. Maybe we would just focus on the literature review and synthesis which could be done in 6 months and then turning it over to ABNS members.
These are two different studies and models.
Comments
If you remember having grand theory which would explain all of nursing, you know that we moved to middle range theories. We’re not going to find THE answer to continuing competency. We will find something good but it will change again in ten years. To spend a lot of energy on something that will be contextual to your own organization is a waste.