An Analysis of Sampling in Quantitative research of urban planning based on the survey (Case study: Journal of Research and Urban Planning in year 2019, no. 36 and 37)

Subject Areas :

Saeid amanpour ¹ , ALIREZA RAHIMI chamkhani ²

1 - Associate Professor of Geography & Urban Planning, Shahid Chamran University, Ahvaz, Iran.
2 - PhD student of Geography & Urban Planning, Shahid Chamran University, Ahvaz, Iran.

Received: 2019-10-07 Accepted : 2020-05-24 Published : 2020-10-22

Keywords: validity, Sampling, Quantitative research, Generalizability, Journal of Research and Urban Planning,

Abstract :

Sampling is one of the practical items of methodology in urban planning researches during recent decades that because of its significant role in improving the quality and credit of researches, analyses of design procedure, applying and reporting based on logics of sampling theory, seems to be essential. This research concerns with the articles based on sampling where have been published in Journal of Research and Urban Planning 2015 as the sample, with quality method. Here, the methods of sampling of this report have been evaluated analytically. The results show great considerable errors in reporting the methodology of the sampling in most of the articles. For example, in the ten studied articles regarding the methods of explaining, the sampling criterions (100% not reported), sampling methods (55% not reported and 45 % incomplete reports, incorrect or unclear), determining the statistics society (45% not reported 30% incomplete or unclear), determining the ways of access to samples (70% not reported 5% incomplete or unclear), seems not to be trustworthy. On the other hand, the only index which could be almost acceptable is reporting the numbers of samples (%80 exact and complete and %20 incomplete and unclear). In general, the method of reporting the procedure of sampling in these articles seems to be incomplete and unclear (in %50 of cases), incomplete and incorrect (in %10 of cases), and incomplete, incorrect, and unclear (in %40 of cases). Therefore, none of them includes a complete, clear, and correct report (in all indexes at the same time) of the method of sampling.  Therefor lacking of a clear scientific pattern in reporting the sampling method based on rules and assumptions of the sampling theory in the studied researches, could influence the scientific quality (i.e. validity, reliability and generalizability) of them. This problem requires a serious attention of science member’s society in fields of urban planning to methodology of sampling in teaching process, searching, analyzing and publishing scientific researches.
Extended abstract
Introduction:
The main important sections of any recent researches is widely believed to be the methodology, quality of the report and the style of writing. These are the main important and gist of any recent research in urban planning studies as well. The aforementioned factors, provide the required information for the repetition, the structure, ability to be examined, generalization and other characteristics of the study. Only by thorough consideration of the mentioned factors, a reliable outcome would be eventually obtained. Therefore, one can evaluate the reliability of the study by inspecting the sampling method and the reporting style. Sampling is one of the many stages of urban planning studies and methodologies. The sampling method has huge importance in analysis and performing the given plans in a study. It must be considered any lack of precision and ambiguity can severely rule out the scientific reliability of the study. Hence, the presented article aims to analyze and evaluate the sampling methods in urban planning surveys.
Methodology:
The presented study is of the qualitative research in terms of the identity and research approach. Data are gathered from the available references and citation from the resources. The analysis method is descriptive-analytical and based on the content analysis of the resources. The goal society of the research is the quantitative researches based on the urban planning surveys which they have used sampling methods at any level. To simplify the study the 36 and 36 issues of the 1398 edition of “Journal of Urban Planning” including 10 eligible articles are selected to be analyzed (Table 1). The number of variables, theoretical definition of the statistical population, the number of statistical populations, frame sampling population size and the method of accessing the samples are analyzed according the deductions given in their report.
Table 1: Sources of survey articles based on quantitative sampling published in the Journal of Research and Urban Planning in the first half of the year 2019





code


SOURCE


 


code


Source


 




F


Ghorbanpour et al (2019)


6


A


Heidarzadeh & Behzadfar (2019)


1




G


Arzaghi et al (2019)


7


B


Haghpanah et al (2019)


2




H


Zarin kaviyani et al (2019)


8


C


shamsoddini & nasibi (2019


3




I


PourAhmad et al (2019)


9


D


Miralaei et al (2019)


4




J


Badiee et al (2019)


10


E


Moradian et al (2019)


5





Results:
It is found that there is a severe lack of precision and meticulousness in methodology of sampling reports in most of the inspected articles. That is, in terms of explanation and reporting the measures of sampling frameworks: 100% not reported, sampling: 40% not reported and 60% uncompleted and ambiguous reports, accessing the sample members: 80% nor reported, defining the number of populations: 40% not reported and 30% ambiguous report are seen. The only acceptable reported measure in the articles was the number of samples: 80% exact and complete and 20 ambiguous or uncompleted. The style of reporting and sampling process are uncompleted in 50% of the articles. Meaning the all or few of the inspected measures are not reported or explained superficially. 10% of articles have wrong or misleading reporting style, meaning, having not mentioning some of the measure, others- for example sampling type, are reported in a wrong way and what is reported have conflicts with the actual outcomes of the study. Moreover, 40% of the articles have all the mentioned sampling methodology issues together. Sampling methodology is not mentioned in some of them at all and others in a very ambiguous way. In some of the articles, a different sampling methodology rather than the actual method used in the articles are presented. Therefore, 100% of the articles have methodology and sampling issues and none of them can be regarded as a complete and correct report in all aspects.

Table 2: Summary of the results of the evaluation of the sampling method report in the studied articles





Report status


Statistical Society


Sample size


Sampling method and access method to sample members




Theoretical definition


Statistical definition
(Number)


Calculation method


Number of samples


Sampling frame


Sampling method


Access method




Frequency
(Percentage)


Frequency
(Percentage)


Frequency
(Percentage)


Frequency
(Percentage)


Frequency
(Percentage)


Frequency
(Percentage)


Frequency
(Percentage)




Not reported


1 (10)


4 (40)


2 (20)


0 (0)


10 (100)


4 (40)


8 (80)




Incomplete or ambiguous or incorrect report


5 (50)


3 (30)


3 (30)


2 (20)


0 (0)


6 (60)


0 (0)




Complete and correct (exact) report


4 (40)


3 (30)


5 (50)


8 (80)


0 (0)


0 (0)


2 (20)




Total


10 (100)


10 (100)


10 (100)


10 (100)


10 (100)


10 (100)


10 (100)





Conclusion:
To sum up, the results indicate the extreme lack of a pure and systematic scientific modeling in the field of sampling method and reporting based on theoretical principles and rules of sampling techniques. The results has severely damaged the reliability of the studies and it cannot be said whether the results and conclusions made in the studies are correct or false and not even how much correct or how much false. Also, there exists numerous and significant weaknesses and inadequacies in sampling method and its reporting style in urban planning researches. Such conclusions are based on quantitative survey in terms of compatibility with the principles and characteristics of the scientific research method, such as reproducibility, validity, reliability and generalizability, which their quality and scientific validity are somehow overshadowed by them. The reason of this problem is largely due to the lack of full knowledge of the theoretical principles and rules of sampling theory. Moreover, the characteristics of the scientific research method or neglect of it during the research can also be reasons for this problem. This problem points out the necessity for serious attention and examination of the scientific community members in the field of urban planning to the existence of accuracy in choosing and planning sampling methods and to the morality and scientific explicitness in compiling and reporting it in the processes related to education, research, evaluation and publication of scientific research in line with scientific monitoring in this field.

References:

1. Abedi, A., Shavakhi, A.R.. (2010), The Comparison Between Quantitative and Qualitative Research in Behavioral Science, Rahbord, 19(54): 153-168.


2. Afshari, M., Mahram, B., Noughani, M. (2013). Studying and identifying indicators to evaluate the quality of scientific research papers in the field of humanities, science-based norms theory Merton, Journal of Science & Technology Policy, 6(1): 49-66.


3. Arab Mzar A., Hosseini Nejad S.M. (2006). Selectivity Bias in Standard of Living Studies, Economic Resarch Review, Vol. 5, No. 4 (19): 81-113.


4. Arzaghi, A., Aminpoor, A., Toghyani, S. (2019). Measuring and Evaluating the Effects of Green Spaces on the Sustainability of Residents' Lifestyles in Residential Complexes (Case: One and Six Shiraz Regions). Quarterly Research and Urban Planning, 10(37): 171-181.


5. Asefzadeh, S. (2005). Critical analysis of studies in medical science. Research in Medicine, 29(3): 195-201.


6. Attia, A. (2005). Bias in RCTs: confounders, selection bias and allocation concealment, Evidence-based medicine, 10(3): 258-261.


7. Badiee, L., ezatpanah, B., soltani, A. (2019). Explain and analyze the development of urban sustainability based on environmental components: case study of Sanandaj city. Quarterly Research and Urban Planning, 10(36): 75-86.


8. Baker, T. (2000). Doing Social research, Translated by Hooshang Nayebi, fourth edition, Ney Publications, Tehran.


9. Barjak, F. (2006). Research productivity in the internet era, Scienometrics, 68: 343-360.


10. Battaglia, Michael P. (2011).  Nonprobability Sampling. Encyclopedia of Survey Research Methods. SAGE Publication, New York.


11. Berk, Richard A. (1983). An Introduction to sample selection Bias in sociological data, American Sociological Review, vol 48: 386-398.


12. Blaikie, N. W.H. (2000) Designing social research, Translated by Hassan Chavoshian, Third edition, Ney Publications, Tehran.


13. Colton, G. (2013). How do we sample? Introduction to Survey Sampling, Translated by Behrouz Granpaye, Ney publications, Tehran.


14. Cuddeback, G., Wilson, El., Orme, John G., Combs-Orme, T. (2004): Detecting and Statistically Correcting Sample Selection Bias, Journal of Social Service Research, Vol. 30(3), 19-33.


15. Delavar, A. (2004). Theoretical and practical research in the humanities and social sciences, Third edition, Roshd Publication, Tehran.


16. Delavar, A. (2012). Research method in psychology and educational science, Thirty-Sixth published, Fourth Edition, Nashr Virayesh nstitute, Tehran.


17. Evans, K.M. (1982). Introduction to Research Methods in Education,Translated by Mohammad Ali Farjad, Elham Publications, Tehran.


18. Finch, S., Gordan, L. (2013). Random sampling- A guide for teachers – Years 11 and 12; Probability and statistics: Module 23, Published by Education Services Australia.


19. Ghorbanpour, M., Molavi, M., Zali, N. (2019). Analyzing the Economic Criteria of Urban Sustainable Tourism (Case Study: Zarjub River of Rasht). Quarterly Research and Urban Planning, 10(37): 129-140.


20. Hafez Nia, M. (2004). An introduction to the research methodology in humanities, SAMT Publication, Tehran,


21. Haghpanah, Y., Rafieian, M., kamyabi, S., arghan, A. (2019). Defining the role of the urban land management pattern based on the development of the physical and spatial development of metropolises Case study: Region 12, Tehran Metropolitan. Quarterly Research and Urban Planning, 10(37): 67-82.


22. Hassanzadeh, R. (2004). Research method for behavioral sciences (A Practice guide for Research), Savalane Publication, Tehran.


23. Heidarzadeh, E., Behzadfar, M. (2019). The Impact of Population Density on Urban Quality of Life Indicators, Case Study: District 3 of Tehran. Quarterly Research and Urban Planning, 10(37): 1-12.


24. ILO (2004). Consumer price index manual; Theory and practice, Publications of the International Labor Office, First published (2004), www.ilo.org/publns


25. Khatibian, M., Abed Saeidi, Zh., Ashtari,I. (2007). Sampling in Quantitative Research, Quarterly faculty of Nursing & Midwifery, 16(56): 53-60.


26. Kish, L. (1965). Survey Sampling, New York: John Wiley and Sons Inc.


27. Kousha, K., Sadeghiani, J., Haidari, M.S., Zainali Kermani, A. (2010). A Study on the Rate of Articles Derived from PhD Dissertations in University of Tehran (2001-2005), Journal of Academic Librarianship and Information Research, 44(2): 45-69.


28. Levy, Paul S., lemeshow, S. (2004). Sampling of populations: methods and applications, Translated by Giti Mokhtari, Institute of Statistics publications, Tehran.


29. Malkomian, L., Shirani, P. (1985). the application sampling methods in the social sciences, Kalama publications.


30. Mardani, A.H., Nikkar. M. (2009). Self-citation of Faculty Members in University of Tehran in Journals. Research on Information Science & Public Libraries, 15 (3) :149-167


31. Mills, Harriet L, Johnson, Samuel, Hickman, Matthew, Jones, Nick S. & Colijn, Caroline (2014). Errors in reported degrees and respondent driven sampling: Implications for bias, Drug and Alcohol Dependence 142(2014) 120 – 6.


32. Miralaei, S., mohamadi, M., SAMETI, M. (2019). Housing choice, based on demographic characteristics of households, using discrete choice experiment method, in Isfahan city. Quarterly Research and Urban Planning, 10(37): 97-112.


33. Moradian, O., rakhshandehroo, M., abdolahzade fard, A. (2019). The role of urban parks in the social sustainability of cities, case study Azadi Park, Shiraz. Quarterly Research and Urban Planning, 10(37): 113-128.


34. Morgan, R.D. (1975). Introduction to Sampling Methods, Translation by Kamran Sepehri, Farous Publications.


35. Moser, S.K., Colton, J. (1989). Research Methodology, Translation by Kazem Izadi, Keyhan Publications.


36. Mousavi, Y., Zaer Sabet, F. (2011). A sociological survey about humanity and structural obstacles of the growth of applied social researches in Iran, 2010-2011, 3(2): 7-34.


37. Naderi, E., Seif Naraghi, M. (1985). Research Methods in Human Sciences (With an emphasis on Education), Badr publications.


38. Omurcheartaigh, C., Wong, S. T. (181). The impact of sampling theory on survey practice: a review, Bulletin of the International Statistical Institute, 49(1): 465-493.


39. PourAhmad, A., ziari, K., Abdali, Y., Sadeghi, A. (2019). Analysis of resiliency criteria in urban worn out texture of Tehran 10 municipality against earthquake with emphasis on physical resilience. Quarterly Research and Urban Planning, 10(36): 1-21.


40. Raj, D., The design sample surveys, Translated by Abolghasem Bzrgnya and Seyed Mojtaba Hoseyniyon, first volume.


41. Safaei Pour, M., Rahimi Chamkhani. A.R. (2018). Critical Analysis of Sampling Designs in Human Geography Research by Focusing on Selection Bias in Probability Samples. Geographical research, 32 (4): 47-68. URL: http://georesearch.ir/article-1-365-en.html.


42. Saraei, H. (2003). Introduction to survey sampling, samt publication, Tehran.


43. Saroukhani, B. (2009). Research methods in social science, vol. 1: Principles & Foundations, Publications Institute for Humanities and Cultural Studies, Tehran.


44. shamsoddini, A., nasibi, S. (2019). The study of urban furniture layout on the urban area's vitality (case study: the whole area of Shiraz). Quarterly Research and Urban Planning, 10(37): 83-96.


45. Tabatabaee, S.H.R., Fazalzadeh, O. (2009). Assessing the Principle of Scientific Writing in Published Articles in Shiraz University of Medical Sciences, Iranian Journal of Epidemiology, 5(2): 28-34.


46. Tajdari, A.R. (2010). Selection and Measurement Bias in Estimates of Hidden Populations Size: The Case of Drug Use Researches in Iran, Social Development and welfare Planning, 1(1): 181-196.


47. tamimdari, A. (2010). The Requirement of Developing Research Methods in Persian Literature. Literary Text Research, 14(44): 43-61. doi: 10.22054/ltr.2010.6499


48. Zarin kaviyani, A., Kalantari, M., Meshkini, A., piry, I. (2019). Structural analysis of barriers to community-based Enabling in informal settlements (Case study: Ban Barze neighborhood in Ilam city). Quarterly Research and Urban Planning, 10(36): 22-32.