The PA-CAT seems to be on everyone’s mind. Do I need to take the test? Is it replacing the GRE? How do I study for it? We went straight to the source with this interview with Dr. Scott Massey, a PA with tons of educational experience and a PhD. Dr. Massey has spent his career researching PA education and trying to improve it, and he shares his involvement in the development of the PA-CAT and why this test is the next step in PA admissions.
P.S. This blog post is also a podcast episode! Listen to it here.
Dr. Massey: My name is Scott Massey, and I’ve been a PA for 36 years. I’ve been a PA educator for 29 years. To make a long story short, I've been in leadership positions in 4 PA programs, and I’ve been an active researcher over the years in particular about predictive factors for success in PA education. A couple of years ago, Exam Master approached me and the executive of Exam Master came to one of my presentations and asked if I would be interested in participating in this exciting project, which was developing and launching a PA admissions exam. He asked me and Dr. Johnna Yealy to be the co-investigators of this project, as there would be an 18 month research component for that. I won’t go into everything else as far as my PA career, but I’ve worked in numerous settings including more emergency medicine than anything else. I also worked internationally for a year, and I’ve worked in several countries around the world participating in projects as well. At this point in time, I’m focusing on research and education, and another part of my time is spent working on this project. So that’s a summary of me!
Savanna: So a little bit of everything but sounds good! Yeah so I'm going to be perfectly honest. I am not exactly sure what my thoughts are on the PA-CAT, so I'm excited to learn more. I have been a PA for 5 years now, and I think if I was in a pre-PA’s shoes, I would be a little bit terrified. But as someone who has been through it and has helped pre-PA students throughout the process, I can see the benefits of a test that is designed like the PA-CAT to be more specific than say, the GRE. So I'm excited to hear more and see if it helps my thoughts on it a little bit more. So you said it wasn't your idea; you just got involved, is that safe to say?
Dr. Massey: I was one of the principal investigators. When you launch something of that magnitude, there has to be some robust research behind it. The exam was launched in a research phase in 2018, and it’s been administered to over 2,000 students, both who are first years matriculated into PA programs and to participants who are in the interview phase. So obviously, because this has to be approved by Institutional Review Board, we cannot ethically administer this exam to pre-PA students in the case that people would use it for the purpose of making decisions. Up until now it has been in the research phase, and this will continue into the future as well.
As far as what you mentioned about the concern from pre-PA students regarding PA-CAT vs. GRE, I can tell you honestly as a PA educator, I’ve been involved with admissions for now 28 years. I can tell you that whatever tools the educators are going to use to admit students, they are going to use. I’ve read virtually all the research that PA education and PA educators have done on the GRE and predictive factors and on the PANCE and its success. And it’s not strong. It’s not a great instrument. The comment I want to make to all those students is that do you want PA educators to use a tool that’s not very strong vs. one that is a better identifier of students’ success in the PA program. There's nothing we can do to really to allay their fears, it's going to be a process but we'll get into that more detail as you ask questions.
Savanna: Looking at applications and talking to students, I find it hard to help students understand that PA schools have to standardize patients to a degree. There is so much variety in where they are taking classes and getting their experience, there has to be some way to even the playing field a bit. So what have schools used traditionally up until this point?
Dr. Massey: Unfortunately, they use science GPA quite a bit. We just got the data from a research phase for this project, which looks at several hundred students and their relationship to prerequisite science and admissions, basically like the first year of a PA program and the first year PACKRAT. The relationship between those variables are very, very weak. And what research shows is that science GPA, which intuitively we would think that looking at grades and performance in chemistry would be an indicator, and it hasn’t panned out. I’ve done my own assessment, so I’m involved with a lot of assessment. I have assessed several years of data looking at, here at Central for example, looking at how well science GPA predicts PANCE performance in an academic format, and there is a negative relationship, which means it is not strong. So PA educators are going to use that, even though it is not a great predictor. Interviews and other personal things, I think, are probably one of the more important things. We try to identify that by using MMIs, group interviews, and personal interviews. I think looking at personal statements are important.
There are qualitative types of variables and the quantitative. 51% of the PA programs, as of last year, are using the GRE. So they plug the GRE score into a spreadsheet and formula, and they say, oh here is the score on the GRE, we will give the student however many points. So that’s kind of what happens. It’s not an exact science to admit students, and I’m the first to tell you that my whole career has been about student success. I work with students who have failed PA school. I worked with students who are trying to remediate and develop their remediation tools. At the end of the day, not every student who applies is able to handle PA school. It’s our moral and ethical responsibility to not admit students who are not going to be successful. So it’s tough, it’s a tough job.
Savanna: So what are the end goals and end points of the PA-CAT? Is it seeing a better performance on PANCE, better GPAs in PA school, less matriculation – how do you figure out if it is effective?
Dr. Massey: I believe the best way to find out if it is effective is to see a slight decrease an attrition. Whether someone gets 450 or 500 on PANCE, I could care less. But you know, attrition rates in PA programs ranges from virtually 0% in some that only accept students in the top 10% of GPAs to much higher in programs that are trying to admit candidates from disadvantaged backgrounds and those kinds of things. So, if this tool at the end of the day, lowers some attrition, then that will be successful. So it's really about student success. Basically to me, it's the best tool in the arsenal that will probably be available to programs to look at cognitive factors. So the other thing I'll say is that some programs that have a mission to admit students that might get lower scores on this exam could still use this exam to try to identify those that need a pre-matriculation type of program as well. So, they have multiple uses.
Savanna: That's interesting. So can you even compare the PA-CAT to the GRE?
Dr. Massey: Not necessarily because it is testing different types of skills. There is the verbal and quantitative. I think there is some overlap with quantitative because it is testing some statistics and some chemistry. But for the most part, it’s really testing basic science knowledge and social science knowledge. It is most closely aligned with the pharmacy CAT. There’s a PCAT (Pharmacy College Admission Test). During the development phase, the committee looked at the PCAT because that was probably the closest aligned exam that already existed. The MCAT, which has undergone some evolution, is basically testing the same thing. It is testing the typical undergraduate courses that PA students will have already taken. It does not assume that students have advanced biological science courses, for example. It’s based on the average prerequisites of all PA students in the country. We did a large assessment of that to make sure we are not overtesting content that is beyond the scope of what PA students are experiencing.
Savanna: Okay. And I listed out the subjects on my website. There's a lot of them, so I don't know if you want to list them, but we can either direct people there or if you want to go over them, you can. But I want to know more the logistics of the test, like how long is it, how is it scored, that type of thing.
Dr. Massey: So the exam is going to be 225 questions. During the launch this year, 25 of those questions are going to be what are called “field test” questions. 200 questions will actually be scored. It will be based on the one minute per question framework. As far as scale, the scale process is very similar to what the PACKRAT uses. So with scaled scores, basically all the students in the country, the lowest score might be 200 and the highest might be 800, and students find out where they are in terms of their percentile. If they scored higher than 61% of people taking the exam, it will be 61. So they will be total scale percentile scores and subscores, which we are combining the subscores for biological science and AP chemistry and social sciences.
Savanna: I guess with it being so new, is there anyway to tell someone the score they should aim for or what schools are looking for?
Dr. Massey: Not yet, it will probably take several years to get to that point. I will use the example of PAKRAT again, which has been used for 20 years now. There is research out there that looks at scoring PACKRAT. It doesn’t say this is the score you want; you research it, look at the data. Same thing here, it will take time. Each year, applicants will be scored against other applicants. It’s not going to be, here’s the score. Programs, just like they do now, take that data and begin to identify how the scores relate to their own specific departments. You can’t generalize all programs. They will also look at the relationship of the PA-CAT with the PACKRAT, EORs, and the PANCE. Like any instrument, it will take time to get to that point. The PACKRAT is administered on a psychometric model called the The Rasch model. Rasch is the framework of developing a scale and looking at performance within a group. We are not looking at everyone who has ever taken the exam; we are looking at all students within a specific testing window that year.
Savanna: Okay, and I know you said every school uses their own parameters. Do you see this as something that could potentially replace the GRE or become something that is mandated for every single PA program?
Dr. Massey: No admissions exam can be mandated unless the program decides to. Does PAEA condone the PA-CAT? PAEA is neutral. It doesn’t condone it one way or other other; it doesn’t say PA-CAT is better than GRE or vice versa. Programs will choose to adopt this exam and choose to use it in their cycle if they want. It is available.
Savanna: I’ve seen some programs have started to implement the PA-CAT in this upcoming cycle; do we know how many programs?
Dr. Massey: There is estimated to be about a dozen right now. There may be as many as 25-30. Upcoming years, it is really grassroots. As more PA educators hear about this exam, for example I’ve given webinars to PA educators, so they know about it. I don’t know exactly the percentage of programs that will use it; but I think it will probably grow.
Savanna: Alright, so what is going to be the cost of the test, and are there going to be fee waivers? I’ve gotten questions about cost being a limiting factor. What does that look like?
Dr. Massey: The cost of the actual exam is $149. When students register, they're also going to be paying the prometric testing center fee at the same time. So with the total cost, it’s comparable to the GRE. As far as the waivers right now, Exam Master is developing a program to support economically disadvantaged students, so they're connecting with various constituency groups. I know that the executive director is talking to the president of AAPA, trying to reach out to see how that can be done on as well as in state chapters. Certainly, that is the desire if students are challenged economically, to be able to have a scholarship program and offer it free of charge. I honestly don’t know how much that is happening with current exams, like the GRE or CASPER. But that is the intent of Exam Master.
Savanna: Okay, and how would students send scores? I know with the GRE, students are able to send it to 3 schools for free, then you can add more later. Is it similar to that?
Dr. Massey: When students register, there is an articulation with CASPA. The number of schools the students apply to, CASPA would then forward the scores. It’s a process, and I can’t tell you exactly how it is transmitted, but it will similar to the process the GRE uses. There will be a school code, and there is a transcript students can download. So that’s how it’s going to work.
Savanna: Okay, cool. So another question I’ve gotten is that students who have seen certain programs require the PA-CAT now are asking how to prepare. When I was studying for the GRE, I went to the library and checked out some books. But how can students actually study and prepare?
Dr. Massey: First of all, I would say the preparation is not going to be necessarily as robust as looking at setting up and preparing for the GRE because it's a different kind of exam. Students will get you know review books for the GRE and the more that you practice those exam questions, the better you will do – the customer test phenomenon. Because the PA-CAT is based upon the typical undergraduate courses, Exam Master now has a downloadable document, which is like the outline of the exam. Students can go to their own content and their own textbooks and use that document as a guide. There may be some more truncated review books for certain subjects like biology and that’s up to them. Along with the payment for the exam, students do get access to a free practice exam, and that will give them an estimation for their scaled score. So if they go ahead and they take the practice exam. If they score a 450 for example, which is lower, that’s an indication they may have to study more. I can tell you that the outline is quite detailed, they can go into chemistry and review those concepts. I know that depending on when students have taken certain courses, I mean there are certain students that get to their senior year and applying to PA school – it’s been 3 years since they took organic chemistry. So they may have to go back and spend more time focusing on that. Because the concentration of the exam is a bit less on chemistry as opposed to biology and physiology, it’s not going to have as much of an impact on the scaled score.
Savanna: Okay great, so there is some guidance. Last time I checked the website, it wasn’t up yet, but I’ll link that so everyone can find it and feel more reassured. I think you have covered all of my questions. Are there any questions that you are getting regularly?
Dr. Massey: Yeah we look at blogs sometimes, and there are concerns that come up. Let’s see… how heavily weighed will it be in the application process? That depends on the program and how heavily they want to weigh it. Some programs may use a percentile program; some may use a watch-and-see approach and gather all the data first. The other concern I’ve seen is that it’s being implemented under people’s feet, as in they already took the GRE, now they have to take the PA-CAT. The process of putting a new requirement for an applicant is based on, if it’s early in the cycle, it’s pretty permissible to include. Once before the application is due, that’s how programs will look at it and will put it on their website so applicants will know ahead of time. Applicants have to realize that individual programs have the ability to require various elements from their students, so same thing here. I’ve spoken to a few directors that are choosing to make it voluntary this year and phase it in.
Savanna: Do you think there are any programs that will require the GRE and PA-CAT, or is that overkill?
Dr. Massey: I don’t know that for sure. Everyone I’ve spoken to so far is choosing to go to PA-CAT instead of GRE. Again I think the cost to students is concerning if you require both exams. Also, I’ve gotten the comment that if the PANCE score is high, why would you another instrument? Again, I’m going to go back to wanting to reduce attrition. If it helps reduce overall some attrition, then it’s worth it. So the message I want to get across the pre-PA applicants, is that this is about trying to ensure success. It is not about adding more barriers or eliminating people. It’s about ensuring the people who are in programs have a higher probability of succeeding. They’ve also got to realize that when you take 5-10% attrition across states – we are talking about hundreds of people that are being dismissed from programs across the country! So if that's lowered, it's worth it.
Savanna: Yeah, I completely agree, and I think sometimes too it's difficult from a pre-PA standpoint, when you haven't been to a PA school yet. With my class, we started with 44 and ended with 40. One person from my class who graduated never passed the PANCE. I mean, it sounds small, like 4 or 5 people, but that’s life-changing for those people. So, I think if there was something that could have prevented them or kind of indicated that maybe they weren’t going to be able to succeed in PA school, that would have been helpful in the long run.
Dr. Massey: Making programs, myself included, we will look at like any predictive variables that may indicate where students might need more remediation along the way. So if we look at that, what programs do is that they look at applicants who get lower scores in certain classes. And we ask, what do these students need? Do they need study skills? More support? Sometimes it’s a number of things. Sometimes it’s just study skills they need to adjust. Sometimes it’s science knowledge and science aptitude or the ability to handle the fast pace of the program. Some programs also look at those scores and implement a pre-matriculation program review process or some form of intervention for those students. Some medical schools already do this. So I think it opens the door to those things too, not that Oh, I’m just going to reject you because your score is X.
Savanna: Yeah, just the more information, the better. I think that's great. Where can people find out more about the PA-CAT and ask their questions?
Dr. Massey: They can head to the PA-CAT website: https://www.pa-cat.com/pa-cat-candidate-page/. There are a lot of resources on there now.
