Achieving Consensus Deal with Methodological Issues in the Delphi Technique
Subject Areas : Environmental policy and management
زهرا
گودرزی
1
(دانشجوی دکتری ترویج و آموزش کشاورزی، دانشگاه تربیت مدرس، تهران، ایران)
عنایت
عباسی
2
(استادیار گروه ترویج و آموزش کشاورزی، دانشگاه تربیت مدرس، تهران، ایران)
همایون
فرهادیان
3
(دکتری، گروه ترویج و آموزش کشاورزی، دانشگاه تربیت مدرس، تهران، ایران)
Keywords: Delphi Technique, Consensus level, Methodological criticism,
Abstract :
Delphi is a powerful technique used to seek answers to appropriate questions. The purpose of this paper is to provide an overview of the Delphi technique as a research method. This paper discusses the scientific merit of the Delphi technique by investigating on 41 studies of Journal of Agricultural Education from 1981 to 2013, and 2 studies of Journal of Agricultural Science and Technology. The results showed that there is no general agreement on using indexes in different rounds of Delphi technique; however, according to the frequencies of using indexes in different studies, the following suggestions are presented. The favourable number of panel of experts is between 10 to 20. Purposive sampling method is used for selecting the panel members. Usually a three rounds of Delphi method is used. One question is designed in round one. Mean and standard deviation indexes are used for passing from round two to round three and agreement level of 70 present is used for achieving expert's consensus in round three.
Akers, C. L., Vaughn, P. R., & Haygood, J. D. (2003). High school agricultural communications competencies: A national Delphi study. Journal of Agricultural Education, 44(4), 1-10.
Akers, C., Vaughn, P. R., & Lockaby, J. D. (2001). High school agricultural communications competencies: a national Delphi study. Journal of Southern Agricultural Education Research. , 51(1), 124-137.
Beech, B. (1997). Studying the future: a Delphi survey of how multi-disciplinary clinical staffs view the likely development of two community mental health centers over the course of the next two years. Journal of Advanced Nursing, 25(2), 331-338.
Beretta R. (1996) A critical review of the Delphi technique. Nurse Researcher 3(4), 79±89.
Boyd, B. L. (2003). Identifying competencies for volunteer administrators for the coming decade: a national Delphi study. Journal of Agricultural Education, 44(4), 47-56.
Brown, B. B. (1968). Delphi process: A methodology used for the elicitation of opinions of experts (No. RAND-P-3925). Rand Corp Santa Monica CA.
Buriak, P., & Shinn, G. C. (1989). Mission, initiatives, and obstacles to research in agricultural education: A National Delphi Using External Decision-Makers. Journal of Agricultural Education, 30(4), 14-23.
Camp, W. G., Clarke, A., & Fallon, M. (2000). Revisiting supervised agricultural experience. Journal of Agricultural Education, 41(3), 13-22.
Clayton, M. J. (1997). Delphi: a technique to harness expert opinion for critical decision‐making tasks in education. Educational Psychology, 17(4), 373-386.
Cline, A. (2005). Prioritization process using Delphi technique. Carolla Development 2000, Retrieved March, 8. 2005.
Conner, N., & Roberts, T. G. (2013). Competencies and experiences needed by pre–service agricultural educators to teach globalized curricula: A modified Delphi study. Journal of Agricultural Education, 54(1), 8-17.
Connors, J. J. (1998). A regional Delphi study of the perceptions of NVATA, NASAE, and AAAE members on critical issues facing secondary agricultural education programs. Journal of Agricultural Education, 39 37-47.
Creswell, J. W. (2005). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (2nd Ed.). Upper–Saddle River, NJ: Pearson Education.
Dalkey, N. (1969). An experimental study of group opinion: the Delphi method. Futures, 1(5), 408-426.
Dalkey, N. C. (1972). The Delphi method: An experimental study of group opinion. In: N. C. Dalkey, D. L. Rourke, R. Lewis, & D. Snyder (Eds.). Studies in the Quality of Life: Delphi and Decision-making (Pp. 13-54). Lexington, MA: Lexington Books.
Dalkey, N. C., Rourke, D. L., Lewis, R., & Snyder, D. (1972). Studies in the quality of life. Lexington. DC Heath and Company.
Dalkey, N., & Helmer, O. (1963). An experimental application of the Delphi method to the use of experts. Management science, 9(3), 458-467.
Delbecq, A. L., Van de Ven, A. H., & Gustafson, D. H. (1975). Group techniques for program planning: A guide to nominal group and Delphi Processes1975, Glenview. IL: Scott, Foresman.
Dobbins, T. R., & Camp, W. G. (2003). Clinical experiences for agricultural teacher education programs in North Carolina, South Carolina, and Virginia. Journal of Agricultural Education, 44(4), 11-21.
Dyer, J. E., & Breja, L. M. (2003). Problems in recruiting students into agricultural education programs: A Delphi study of agriculture teacher perceptions. Journal of Agricultural Education, 44(2), 75-85.
Dyer, J. E., Breja, L. M., & Ball, A. L. (2003). A Delphi study of agriculture teacher perceptions of problems in student retention. Journal of Agricultural Education, 44(2), 86-95.
Franklin, E. A. (2011). Greenhouse facility management experts’ identification of competencies and teaching methods to support secondary agricultural education instructors: a Modified Delphi study. Journal of Agricultural Education, 52(4), 150-161.
Franklin, K. K., & Hart, J. K. (2007). Idea generation and exploration: Benefits and limitations of the policy Delphi research method. Innovative Higher Education, 31(4), 237–246.
Frick, M. J. Kahler, A. A., & Miller, W. W. (1991). A definition and the concepts of agricultural literacy. Journal of Agricultural Education, 32(2),49-57.
Gordon, T. J. (1994). The Delphi Method. Futures Research Methodology, 2. Retrieved July 6, 2007, from /http://www.futurovenezuela. org/_curso/5-delphi.pdfS.
Green, B., Jones, M., Hughes, D., & Willimas, A. (1999) Applying the Delphi technique in a study of GP's information requirements. Health and Social Care in the Community 7(3), 198-205.
Green, P. (1982, March). The content of a college-level outdoor leadership course. 3572-3572. Paper presented at the Conference of the Northwest District Association for the American Alliance for Health, Physical Education, Recreation, and Dance, Spokane, WA.
Harder, A., Place, N. T., & Scheer, S. D. (2010).Towards a competency-based extension education curriculum: A Delphi Study. Journal of Agricultural Education, 51(3): 44.
Hasson, F., Keeney, S. & McKenna, H. (2000). Research Guidelines for the Delphi Survey Technique. Journal of advanced nursing, 32(4): 1008-1015.
Hung, H. L., Altschuld, J. W. & Lee, Y. F. (2008). Methodological and Conceptual Issues Confronting a Cross-country Delphi Study of Educational Program Evaluation. Evaluation and Program Planning 2008, 31(2), 191-198.
Ishikawa, A., Amagasa, M., Shiga, T., Tomizawa, G., Tatsuta, R., & Mieno, H. (1993). The max-min Delphi method and fuzzy Delphi method via fuzzy integration. Fuzzy sets Sets and systemsSystems, 55(3), 241-253.
Jenkins III, C. C., & Kitchel, T. (2009). Identifying Quality Indicators of SAE and FFA: A Delphi Approach. Journal of Agricultural Education, 50(3), 33-42.
Jenkins III, C. C., Kitchel, T., & Hains, B. (2010). Defining Agricultural Education Instructional Quality. Journal of Agricultural Education, 51(3), 53-63.
Jenkins III, C. C., Kitchel, T., & Hains, B. (2010). Defining agricultural education instructional quality. Journal of Agricultural Education, 51(3), 53-63.
Johnson, D. M., & Schumacher, L. G. (1989). Agricultural Mechanics Specialists Identification and Evaluation of Agricultural Mechanics Laboratory Management Competencies: A Modified Delphi Approach. Journal of Agricultural Education, 30(3): ), 23-28.
Keeney, S., Hasson, F., & McKenna, H. (2006). Consulting the Oracle: Ten Lessons from Using the Delphi Technique in Nursing Research. Journal of Advanced Nursing, 53(2): ), 205–212.
Keeney, S., Hasson, F., & McKenna, H. P. (2001). A critical review of the Delphi technique as a research methodology for nursing. International Journal of Nursing Studies, 38(2), 195-200.
Landeta, J. (2006). Current validity of the Delphi method in social sciences. Technological forecasting Forecasting and social Social changeChange, 73(5), 467-482.
Lang, T. 1994. An Overview of Four Futures Methodologies. Retrieved July 6, 2007, from http://www.soc.hawaii.edu/future/j7/LANG.html.
Lawrence, L. D., & Mallilo, A. T. (1981). Identification of Specific Areas of Vocational Agriculture Teaching in Need of the Greatest Improvement: A Modified Delphi Approach. Journal of the American Association of Teacher Educators in Agriculture, 22(1), 24-28.
Linstone, H. A., & Turoff, M. (1975). (Eds.) The Delphi Method: Techniques and applications, 29, Reading, MA: Addison-Wesley Publishing Company.
Linstone, H. A., Turoff, M., & Helmer, O. (2002). The policy Delphi. The Delphi method: Techniques and applications. Reading, MA: Addison-Wesley Publishing Company, Pp. 80-97.
Loughlin, K. G., & Moore, L. F. (1979). Using Delphi to achieve congruent objectives and activities in a pediatrics department. Academic Medicine, 54(2), 101-6.
Ludwig, B. G., & Barrick, R. K. (1996). Internationalization of extension: What does it mean? Journal of Agricultural Education, 37, 40-46.
Ludwig, L., & Starr, S. (2005).Library as place: Results of a Delphi study. Journal of the Medical Library Association, 93(3), 315.
Mantooth, L. J., & Fritz, C. A. (2006). Challenges of service-learning in tennessee 4-H Youth Development: A Delphi Study. Journal of Agricultural Education, 47(3), 94-104. DOI: 10.5032/jae.2006.03094.
Martin, M. J., Fritzsche, J. A., & Ball, A. L. (2006). A Delphi study of teachers’ and professionals’ perceptions regarding the impact of the no child left behind legislation on secondary agricultural education programs. Journal of Agricultural Education, 47(1), 100-109.
McCampbell, W. H., & Stewart, B. R. (1992). Career ladder programs for vocational educators: Desirable characteristics. Journal of Vocational Education Research, 17(1), 53-68.
McKenna, H. P. (1994). The Delphi technique: a worthwhile research approach for nursing? Journal of Advanced Nursing, 19(6), 1221-1225.
Mitchell, V. W. (1991). The Delphi Technique: An exposition and application. Technology Analysis & Strategic Management, 3(4), 333–358.
Moore, G. E. (1987). A day late and a dollar short: Doctoral research in agricultural education. Presented to the National Agricultural Education Research Meeting. Las Vegas, NV.
Mundt, J. P., & Connors, J. J. (1999). Problems and challenges associated with the first years of teaching agriculture: A framework for pre-service and in-service education. Journal of Agricultural Education, 40, 38-48.
Murphy, T. H., & Terry, H. R. (1998). Opportunities and obstacles for distance education in agricultural education. Journal of Agricultural Education, 39, 28-36.
Myers, B. E., & Thompson, G. W. (2009). Integrating academics into agriculture programs: A Delphi study to determine perceptions of the national agriscience teacher ambassador academy participants. Journal of Agricultural Education, 50(2), 77-88.
Myers, B. E., Dyer, J. E., & Washburn, S. G. (2005). Problems facing beginning agriculture teachers. Journal of Agricultural Education, 46(3), 47-55.
Namdar, R. & Sadighi, H. (2013). Investigation of major challenges of rural development in Iran utilizing Delphi technique. Journal of Agricultural Science and Technology, 15(3), 445-455.
Nistler, D. L., Lamm, A. J., & Stedman, N. (2011). Evaluating the influences on extension professionals' engagement in leadership roles. Journal of Agricultural Education, 52(3), 110-121.
Park, T. D., & Rudd, R. (2005). A description of the characteristics attributed to students' decisions to teach agriscience. Journal of Agricultural Education, 46(3), 82-91.
Powell, C. (2003). The Delphi technique: Myths and realities. Journal of Advanced Nursing, 41(4), 376–382.
Ramsey, J. W., & Edwards, M. C. (2011). Entry-Level technical skills that agricultural industry experts expected students to learn through their supervised agricultural experiences: A Modified Delphi study. Journal of Agricultural Education, 52(2), 82-94.
Rasouli, F., Sadighi, H., & Minaei, S. (2009). Factors affecting agricultural mechanization: A Case study on sunflower seed farms in Iran. Journal of Agricultural Science and Technology, 11, 39-48.
Rayfield, J., & Croom, B. (2010). Program needs of middle school agricultural education teachers: A Delphi study. Journal of Agricultural Education, 51(4), 131-141.
Roberts, T. G. (2006). Developing a model of cooperating teacher effectiveness. Journal of Agricultural Education, 47(3), 1-13.
Roberts, T. G., & Dyer, J. E. 2004. Characteristics of effective agriculture teachers. Journal of Agricultural Education, 45, 82-95
Sackman, H. (1974). Delphi assessment: Expert Opinion, Forecasting, and Group Process (No. Rand-R-1283-Pr), Rand Corp Santa Monica California.
Saucier, P. R., McKim, B. R., & Tummons, J. D. (2012). A Delphi Approach to the preparation of early-career agricultural educators in the curriculum area of agricultural mechanics: Fully qualified and highly motivated or status quo? Journal of Agricultural Education, 53(1), 136-149.
Shih, C. C., & Gamon, J. A. (1997). Using the Delphi Technique to assess educational needs related to extension’s 4-h beef program. Journal of Agricultural Education, 38(1), 14-20.
Shinn, G. C., Briers, G., & Baker, M. (2008). Forecasting Doctoral-Level Content in Agricultural Education: Viewpoints of Engaged Scholars in the United States. Journal of Agricultural Education, 49(1), 121-131.
Simon, L. A., Haygood, J. D., Akers, C. L., Doerfert, D. L., & Davis, C. S. (2005). Master's level agricultural communications curriculum: A National Delphi study. Journal of Agricultural Education, 46(3), 56.
Skulmoski, G., Hartman, F., & Krahn, J. (2007). The Delphi Method for graduate research. Journal of Information Technology Education: Research, 6(1), 1-21.
Slusher, W. L., Robinson, J. S., & Edwards, M. C. (2011). Assessing the animal science technical skills needed by secondary agricultural education graduates for employment in the animal industries: A Modified Delphi study. Journal of Agricultural Education, 52(2), 95-106.
Smalley, S. W., & Retallick, M. S. (2011). Purposes, activities, and documentation of early field experience in agricultural teacher education: A National Delphi study. Journal of Agricultural Education, 52(3), 100-109.
Snyder-Halpern, R. (2002). Indicators of organizational readiness for clinical information technology/systems innovation: a Delphi study. International Journal of Medical Informatics, 63, 179-204.
Sprenkle, D. H., & Piercy, F. P. (Eds.). (2005). Research methods in family therapy. Guilford Press. New York City, NY.
Sutphin, H. D., & Newcomb, L. H. (1983). Positions held by teachers, teacher educators, and state supervisors about selected national issues in agricultural education. Journal of the American Association of Teacher Educators in Agriculture, 24(2), 53-63.
Trexler, C. J., Parr, D. M., & Khanna, N. (2006). A Delphi study of agricultural practitioners' opinions: Necessary experiences for inclusion in an undergraduate sustainable agricultural major. Journal of Agricultural Education, 47(4), 15.
Ulschak, F. L. (1983). Human resource development: The theory and practice of need assessment. Reston Publishing Co, United States.
Vamadore, W. L. & Iverson, M. J. (1991). Projecting meat industry characteristics in the 21st century using Delphi: Extrapolating curriculum content in agricultural education. Journal of Agricultural Education, 32(2), 29-33.
Van Zolingen, S. J., & Klaassen, C. A. (2003). Selection processes in A Delphi study about key qualifications in senior secondary vocational education. Technological forecasting and social change, 70(4), 317-340.
Warner, W. J., & Washburn, S. G. (2009). Issues facing urban agriscience teachers: A Delphi study. Journal of Agricultural Education, 50(1), 105-115.
Williams, P. L., & Webb, C. (1994). The Delphi Technique: A Methodological Discussion. Journal of Advanced Nursing, 19(1), 180-186.
Witkin, B. R., & Altschuld, J. W. (1995). Planning and Conducting Needs Assessment: A Practical Guide. Thousand Oaks, CA: Sage Publications, Inc.
Wooten, K., Rayfield, J., & Moore, L. L. (2013). Identifying STEM concepts associated with junior livestock projects. what a degree in agricultural leadership really means: exploring student conceptualizations. Journal of Agricultural Education, 54(4), 31-44.
Wooten, K., Rayfield, J., & Moore, L. L. (2013). Identifying STEM concepts associated with junior livestock projects. Journal of Agricultural Education, 54(4), 31-44.
Woudenberg, F. (1991). An evaluation of Delphi. Technological Forecasting and Social Change, 40(2), 131-150.
Young, W. H. & Hogben, D. O. N. A. L. D. (1978). An experimental study of the Delphi technique. Education Research Perspective, 5, 57-62.