Reviewing grant proposals for federal agencies has proven to be a boon to researchers at the USC School of Social Work. A perfect example is Suzanne Wenzel, a professor and current chair of the school’s Research Council. Since 2001, she has served as a reviewer for the National Institutes of Health (NIH) on multiple occasions. She recently accepted an invitation to join a four-year review panel tasked with examining proposals related to the multiple personal, social and community factors that influence HIV risk and preventive behaviors.

“I think it has helped me to craft stronger proposals and to give my colleagues and doctoral students better advice on writing their own,” she said. “The level of precision and scientific rigor that is expected by the reviewers and the institutes is really quite pronounced.”

The grant review process can be mystifying for many in academia. Application guidelines are long and complex, arguments and approaches have to be framed in a certain way, and funding priorities are constantly shifting.

Having an insider’s perspective on how federal agencies such as the NIH and National Science Foundation (NSF) assess grant applications is critical to putting together a successful proposal, said Charles Kaplan, associate dean of research at the USC School of Social Work.

“You can send proposals in, but you can’t get them funded unless you are very aware of the dynamics of the review process,” he said. “You really have to have your finger on the pulse of what of what NIH and NSF are doing in terms of their review and to try to get on those committees. That’s the best way of learning. It’s a stepping-stone to excellence.”

The school has increasingly emphasized mentoring and internal review of grant proposals before they are submitted. Each of the school’s research clusters—which bring together faculty, student and postdoctoral scholars with similar research interests to stimulate intellectual productivity—has at least one faculty member with extensive experience reviewing federal grant proposals. That knowledge appears to be paying off.

Three recent proposals earned perfect scores during the review process, a relatively rare occurrence in federal funding circles. However, Kaplan said he would like to see more leadership among faculty members with review experience in terms of mentoring younger or less experienced researchers.

“We definitely have lots of room for improvement,” he said.

Practice makes perfect

The most effective way to gain knowledge of the review process, at least at the NIH, is to be invited to join a study section at the federal Center for Scientific Review. Members of these study sections or peer review groups typically serve for four years and meet three times a year to review proposals that correspond with their area of expertise.

Due to her extensive research in the area, Wenzel has primarily assessed applications for funding that focus on HIV/AIDS prevention and treatment and behavioral health issues. During a typical review period, she is assigned as the primary reviewer of a handful of proposals.

“Your role as a reviewer is to review the applications carefully, present your point of view to the study section, listen to the other reviewers, and as a group evaluate the applications in terms of their overall impact on the field and beyond,” she said. “It is important that you address whether the application is answering an important question within an area of science and how the aims, if successful, will move the field forward.”

Before meeting with other reviewers, however, Wenzel meticulously reads each proposal, including sections on the research and data analysis strategy, procedures for protecting participants, the budget, and biographical information for the research team.

She then drafts a written review and assigns a preliminary score to proposals that warrant further discussion at the group level. After she presents the application to the study section, other reviewers weigh in and vote on a final score.

Recommendations are then forwarded on to a separate advisory council, which helps the federal agency decide which proposals to fund. Adding to the complexity of the process is the varying set of guidelines for different types of proposals. Some focus on the significance and innovation of the research, the rigor of the methods and analysis plan, and the scientific environment in which the project will occur. Others are focused on early career development and may emphasize training opportunities for less experienced researchers.

“If I had to describe my mindset when I approach review of any given application, it would be objectivity,” Wenzel said.“Fairness. Openmindedness. Impartiality must be a hallmark of review. Without that, the process is doomed.”

Staying power

The commitment required to be a federal grant reviewer is staggering. John Brekke, the Frances G. Larson Professor of Social Work Research, has been consistently involved as a scientific reviewer at the federal level since 1989, serving as a standing member of a mental health services and epidemiology review committee at the National Institute of Mental Health for six years and as an ad hoc reviewer on many occasions.

Although each round of review involves at least two weeks of intensive work prior to gathering with other committee members to discuss the proposals, Brekke said he never questioned whether he should dedicate his time to the process.

“It really is a notable time commitment, but it’s just something that people who are in the research community and take the research and scientific world seriously know is something we have to do,” he said.
“It’s all about shaping a scientific agenda, and that’s a wonderful thing to be a part of.”

He also noted that the position is considered to be very prestigious. Only individuals who have exemplary records of scientific exploration and productivity are invited to serve as reviewers, Brekke said.

He is thankful for several senior mentors who offered guidance during the early years of his career, and he has since created a template for others to follow in terms of building a successful grant application.

“I think it’s absolutely perilous for someone to attempt an NIH proposal without mentoring from someone who has gone through the process,” Brekke said. “That can mean the difference between funding and no funding.”

By capitalizing on the strength and knowledge of experienced researchers, he said the USC School of Social Work is setting up its junior faculty members for success. It’s also helping senior researchers maintain their edge in an increasingly competitive funding environment and adapt to new funding streams.

Brekke recently applied for and received a $1.2 million award from the newly established Patient-Centered Outcomes Research Institute, a nonprofit organization created by Congress through the Patient Protection and Affordable Care Act. He said advice and guidance from colleagues who had submitted successful proposals to the institute in earlier rounds of funding was critical to his success.

“I’ve had good success following a really disciplined approach,” he said. “You can tell the proposals that have done this and those that haven’t. It’s immediately obvious, just in reading the abstract, who has had some mentoring and who has not.”

Helping the next generation

Another strong believer in the power of mentoring and peer review of proposals prior to submission is Eric Rice, an assistant professor at the USC School of Social Work. As an early career researcher at UCLA, he wanted to pursue a specific NIH funding mechanism designed for career development known as a K award. Because none of his close colleagues had experience with that award, his mentor referred him to a friend on a federal review committee, Concepción “Concha” Barrio, now an associate professor at the USC School of Social Work.

Barrio, who several years prior had applied for and received a K award on her first submission and had been reviewing grant proposals related to mental health services, was more than happy to provide feedback on Rice’s application.

“No hyperbole, I got that grant because of Concha,” Rice said. “Knowing what one of these things should look like and how it should read and having someone who has sat on those review committees is invaluable. It’s an inside bit of knowledge you wouldn’t otherwise have.”

He has since served as an ad hoc reviewer, called in by federal officials because of his expertise in social network research. He said the experience has given him a better sense of how to craft his own proposals, and he is sharing that knowledge with postdoctoral scholars and colleagues.

However, Rice acknowledges that he still has plenty to learn. He turned to Brekke for advice on how to leverage data from a pilot study into a larger project on social networks among young men and women experiencing homelessness. Wenzel, who also specializes in homelessness issues, helped him refine his application, and he successfully secured a $2 million grant from the National Institute of Mental Health.

“I completely attribute my early career successes to having been in an environment with people like Concha and Suzanne and John who were willing to help me,” Rice said. “The resources are there; you just have to reach out and ask for that help.”

Another early career researcher who is benefiting from the school’s focus on internal review and mentoring related to grant proposals is Mercedes Hernandez, a postdoctoral scholar who recently completed the school’s doctoral program.

She worked closely with Barrio as a PhD student and received valuable feedback on an application for funding from the National Institute of Mental Health to support her dissertation. Barrio served for four years as a reviewer for the institute and has since been invited to participate in ad hoc reviews for various federal agencies.

“She was just amazing in helping me to navigate the whole application process, which can be a little overwhelming and confusing for someone who is not familiar with it,” Hernandez said. “When I was trying to develop ideas for my dissertation, she read my drafts from the beginning, she directed me in terms of what I needed to include, and she directed me to the NIMH website and informed me of what its priorities are and how my work related to that.”

Barrio helped her set up a mock review with experts in serious mental illness and put together other required information beyond her research plan such as her project budget and bio sketch. Hernandez received the dissertation grant, which allowed her to provide incentives to participants in her study and hire a professional to transcribe interview data.

As a postdoctoral scholar, Hernandez is helping Barrio with various grant applications and research projects, developing academic manuscripts, and applying for her own funding.

“Now that I’m a little more familiar with the process, I think I’m able to contribute more,” she said. “Having the NIH experience is the model for other types of grants, and being involved in these projects is furthering my awareness of what’s available in terms of funding. She’s just a wonderful mentor and has treated me as an equal in terms of sharing her resources and knowledge.”

Gaining experience

Building that understanding of the grant review process is critical for early career success, Barrio said, especially in the profession of social work. When she first served as an NIH standing committee member in 2006, she was one of only two reviewers at San Diego State University. Most of her fellow committee members were medical doctors, psychologists and nurses, with a handful of economists and anthropologists.

“I think what made me stand out is I’m a Latina with a lot of practice experience, a social worker, and my area of focus on serious mental illness and psychosis among Latinos and multicultural populations,” she said.

Barrio said her experience as a reviewer has improved her grant-writing skills and continues to afford her a sneak preview of federal funding priorities. Before each review session, she said officials provide an overview of the government’s focus for that particular period. She also feels that being a reviewer has given her a boost of self-confidence, especially when she comes across grant proposals submitted by people whose work she studied and are recognized as top scholars in the field. However, Barrio noted that although gaining this knowledge has benefited her career, the ultimate goal of embracing the role of reviewer is to improve the quality of life and well-being of vulnerable individuals.

“We are invested in this process to improve the rigor of our research and ultimately benefit our community,” she said.