Usage of statistical and substantive significance in Pedagogická orientace and Pedagogika in the last 10 years: Statistical insight

The aim of this paper is to highlight the problems in the use of statistical and substantive significance in the last decade in two leading educational journals, particularly in the journal Pedagogika and Pedagogická orientace. After a brief introduction the article offers a description of the content analysis performed on two pedagogical journals. The results of the research are divided into three areas: the incorrect use of statistical analysis, its incorrect interpretation and the application of measures of substantive significance (effect sizes). In conclusion the author in addition to traditional summary of results and discussion of their limitations offers possible improvements for publishing and educational practice in the Czech educational community. The research showed that quantitative research is not very widespread (two pedagogical journals published only 84 quantitatively oriented articles). More than half of these texts uses the concept of statistical significance for data that do not allow this practice. Errors in the interpretation of statistical significance are quite rare, the most common include mechanical work with the data (most often use asterisks). Contrary to current guidelines Czech educational scientists use quite few measures of substantive significance (effect sizes) and if so they use only the simplest ones. It is also alarming that about a third of the articles analyzed lacked substantive interpretation of the results.