SciTS 2013 Conference: Sessions
Team Science & Evaluation
This panel presents state-of-the-art work that illustrates the quality and breadth of emerging evaluation efforts. The presentations include discussions of novel methods for measuring team interactions, new ways to link survey research with collaboration productivity, and applications of experimental designs to test team science interventions. The first presentation describes evaluations that involve equipping each team member with an electronic badge (sociometric sensors), about the size of a smart phone, which record data as the team members interact. While the badges do not record the content of conversations, they can tell whether two scientists are face to face, how close they are to one another, the intensity of the conversation and other information. The second presentation shows a creative way to combine survey research with bibliometric data to evaluate the complex interactions of factors that affect team performance. Surveys are used to collect the independent variables related to team science: resource strategy variables (the inputs of a research group, human capital and funding), management variables (rewards, communication, quality control), leadership variables (time allocation, research involvement) and networking activities. The third presentation illustrates how randomized experimental designs can be used to assess the effects of a brief intervention designed to encourage collaboration on pilot grant applications for a CTSA. By randomly assigning participants to a symposium on grant opportunities to presentations of potential project ideas the evaluators were able to estimate the likelihood of collaboration for pairs of researchers and show that being in the same room at the event significantly increases the probability of subsequently appearing as coinvestigators on a final grant application. Evaluation approaches like the ones considered here have the potential to advance both evaluation practice and the field of team science. The panel will discuss both the methodological and substantive implications of their work.
Wednesday, June 26, 3:15pm - 4:45pm
Presenters
William Trochim
Session Chair
William Trochim, Ph.D., received his doctorate degree from the Department of Psychology at Northwestern University in the area of Methodology and Evaluation Research. His research is in the area of applied social research methodology, with an emphasis on program planning and evaluation methods. He is known for the development of a number of methodologies in the behavioral, social and medical sciences. He is a prolific writer, including several widely used introductory research methods texts, and articles that have appeared in American Journal of Evaluation, New Directions for Program Evaluation, Evaluation and Program Planning, Evaluation Review, the American Journal of Public Health, Journal of Clinical Epidemiology, Consulting and Clinical Psychology, Controlled Clinical Trials, Performance Improvement, and Medical Decision Making, among others. He is actively engaged in research with the National Science Foundation (NSF) incorporating systems approaches in the evaluation of Science, Technology, Engineering and Mathematics (STEM) education programs. He is working with Dr. Gilbert Botvin, Weill Cornell Medical Center, on an NIH project in studying the use of Life Skills Training and its role in the dissemination, adoption, implementation, and sustainability (DAIS) of this evidence-based program. He is the Director of Evaluation for the Weill Cornell Clinical and Translational Science Center -- a collaboration of Weill Cornell Medical College, Memorial Sloan Kettering Cancer Center, The Hospital for Special Surgery, Hunter College and Cornell Cooperative Extension -- and participates actively in the NIH Clinical and Translational Science Awards national evaluation. He has also served multiple terms on the Board of Directors for the American Evaluation Association, including serving as the 2008 AEA President.
John R. Hollenbeck
John R. Hollenbeck holds the positions of University Distinguished Professor at Michigan State University and Eli Broad Professor of Management at the Eli Broad Graduate School of Business Administration. Dr. Hollenbeck received his Ph.D. in Management from New York University in 1984. He served as the acting editor at Organizational Behavior and Human Decision Processes in 1995, the associate editor of Decision Sciences from 1999 to 2004, and the editor of Personnel Psychology from 1996 to 2002. He has published over 90 articles and book chapters on the topics of team decision-making and work motivation. According to the Institute for Scientific Information, this body of work has been cited over 2,900 times by other researchers. Dr. Hollenbeck has been awarded over $7 million in external research funding, most of which was granted by the U.S. Department of Defense and the National Science Foundation. Along with Daniel R. Ilgen, he founded the Michigan State University Team Effectiveness Research Laboratory, and this facility has been dedicated to conducting large sample team research since 1991. Dr. Hollenbeck has been awarded fellowship status in both the Academy of Management and the American Psychological Association, and was recognized with the Career Achievement Award by the HR Division of the Academy of Management (2011) and the Early Career Award by the Society of Industrial and Organizational Psychology (1992). At Michigan, State, Dr. Hollenbeck has won several teaching awards including the Michigan State Distinguished Faculty Award, the Michigan State Teacher-Scholar Award, and the Broad MBA Most Outstanding Faculty Member.
Karim R. Lakhani
Karim R. Lakhani is an associate professor in the Technology and Operations Management Unit at the Harvard Business School. He is also a faculty affiliate of Harvard’s Institute for Quantitative Social Science and a faculty fellow of Harvard’s Berkman Center for Internet and Society. He specializes in innovation management and leadership in firms and communities. He has extensively studied the emergence of open source software communities and their unique innovation and product development strategies. He has also investigated how critical knowledge from outside of the organization can be accessed through innovation contests. He is the principal investigator and director of the Harvard-NASA Tournament Laboratory, which aims to systematize the use of innovation tournaments for the space agency. He also works extensively with the Harvard Medical School on designing incentives and programs related to encouraging scientific collaboration. He teaches in both the MBA and Executive Education programs at Harvard Business.
 Peter van den Besselaar
Peter van den Besselaar is professor of organization science at the VU University Amsterdam, Faculty of the Social Sciences. He holds the Rathenau Institute Royal Netherlands Academy of Arts and Sciences endowed chair of organization and dynamics of science. Within the VU Universities’ Network Institute, he collaborates with computer scientists to improve data and analytical tools for the science of science.
Previously, he was research director of the science system assessment department at the Rathenau Instituut in Den Haag (2005-2010), professor of communication studies at the University of Amsterdam (2004-2009), director of the Netherlands Social Science Data Archive (2002-2005), and associate professor of social informatics at the University of Amsterdam (1986-2002).
|
