{"id":711,"date":"2020-10-05T06:30:00","date_gmt":"2020-10-05T13:30:00","guid":{"rendered":"https:\/\/edtechuvic.ca\/edci565\/?p=711"},"modified":"2020-10-03T09:39:22","modified_gmt":"2020-10-03T16:39:22","slug":"reliability-and-validity","status":"publish","type":"post","link":"https:\/\/edtechuvic.ca\/edci565\/reliability-and-validity\/","title":{"rendered":"Reliability and Validity"},"content":{"rendered":"

Validity<\/em> and reliability<\/em> are two important ideas that must be considered when thinking about assessment in any context, and especially in learning environments. An assessment is considered valid if it directly measures the product of the stated outcomes, or, stated another way, if it measures whether learners have learned what we wanted them to learn. For example, after completing a course on painting using water colours, a valid assessment might be to show the different ways to layer colour to create texture. It would not be a valid assessment of the water colour students to have them arm wrestle! This is why writing clear outcomes is so important. Conrad and Openo (2018, p. 65)<\/a>\u00a0 make note of Yogi Berra’s famous quote here:<\/p>\n

If you don’t know where you’re going, you might end up somewhere else.<\/p><\/blockquote>\n

More specifically, Finch and French (2019) explain that validity is about whether the evidence we have gathered supports a particular inference about what a learner has achieved. For instance, if we discover that learners who perform well on a licensing assessment for accountants make fewer mistakes in their accounting practice than learners who perform poorly on that assessment, then we can say that the assessment is valid. Our inference (that the learner is qualified to be an accountant) is supported by the evidence we gathered (their performance on the assessment), so, it isn’t so much that the assessment<\/em> is valid, but that our inference<\/em> is valid.<\/p>\n

An assessment is considered reliable if the results are considered accurate. This usually means that given the same learning conditions and activities, repeating an assessment would produce the same results. Finch and French (2019) talk about consistency of results between repeated assessments without the learners remembering the previous assessments. Repeated assessments are often impractical for determining reliability because learners will remember their mistakes from the previous assessments and correct them on subsequent attempts, and they may also become fatigued (these two factors wold have opposite effects on reliability).
\n<\/span><\/p>\n

Reliability and validity can be complicated by many different factors<\/a><\/span>. If learning conditions change in some way, or there are external factors impacting your learners, it will be difficult to know if your assessment is reliable unless you provide opportunities for retesting. Validity of assessments can be impacted by the assessment tool you use, how you design your assessment, and even the way in which any questions are asked.<\/p>\n

Hidden Curriculum<\/span><\/h5>\n

Our assessment protocols tend to speak loudly about that which we value most. If work habits and compliance are held as the highest standards, we tend to include marks for meeting or demerits for missing due dates.\u00a0<\/span><\/p>\n

Effort is another area that appears at times on assessment data. How is this assessed? Everyone can learn but the effort may look different. How will you adjust your assessment to reflect the data you collect? Does multiple attempts at mastery (more effort) mean lower marks because it took longer to achieve mastery? Is a task that is completed quickly by someone with very little effort rewarded because it arrived early (valuing work habits?) Effort is not something that can readily be assessed. A designer may include reflective questions for the learner to consider their work over the course of the learning experience to prompt the learner to set goals for their next learning experience.\u00a0<\/span><\/p>\n

Pre-assessment<\/span><\/h5>\n

If a designer chooses to include a pre-assessment before embarking on their planned instructional approach, they may have a better understanding of the prior knowledge of their learners. This could inform the designer about misconceptions to address, outcomes that will need adjusting, and outcomes that are already met. The designer may also face the conundrum of managing the data of knowing that their learners are in ten different places in their understanding!\u00a0<\/span><\/p>\n

Types of Assessment<\/span><\/h5>\n

There are almost unlimited options for how we assess learning. The important consideration is if the assessment piece aligns with the outcomes. If the outcome is to \u201cbe able to\u201d do a particular task, the assessment should be testing the learner\u2019s ability to do that task. Match the language of the assessment with the verbs used in the outcomes. Using language from Bloom’s Taxonomy<\/a> can assist a designer with this.\u00a0<\/span><\/p>\n

References<\/h5>\n
\n
Conrad, D., & Openo, J. (2018). Assessment Strategies for Online Learning: Engagement and Authenticity<\/i>. Athabasca University Press. https:\/\/doi.org\/10.15215\/aupress\/9781771992329.01<\/a><\/div>\n<\/div>\n
\n
Finch, W. H., & French, B. F. (2019). Educational and psychological measurement<\/i>. Routledge.<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

Validity and reliability are two important ideas that must be considered when thinking about assessment in any context, and especially in learning environments. An assessment is considered valid if it directly measures the product of the stated outcomes, or, stated… Continue Reading →<\/a><\/p>\n","protected":false},"author":3,"featured_media":752,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"class_list":["post-711","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-assessment"],"_links":{"self":[{"href":"https:\/\/edtechuvic.ca\/edci565\/wp-json\/wp\/v2\/posts\/711","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/edtechuvic.ca\/edci565\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/edtechuvic.ca\/edci565\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/edtechuvic.ca\/edci565\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/edtechuvic.ca\/edci565\/wp-json\/wp\/v2\/comments?post=711"}],"version-history":[{"count":23,"href":"https:\/\/edtechuvic.ca\/edci565\/wp-json\/wp\/v2\/posts\/711\/revisions"}],"predecessor-version":[{"id":955,"href":"https:\/\/edtechuvic.ca\/edci565\/wp-json\/wp\/v2\/posts\/711\/revisions\/955"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/edtechuvic.ca\/edci565\/wp-json\/wp\/v2\/media\/752"}],"wp:attachment":[{"href":"https:\/\/edtechuvic.ca\/edci565\/wp-json\/wp\/v2\/media?parent=711"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/edtechuvic.ca\/edci565\/wp-json\/wp\/v2\/categories?post=711"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/edtechuvic.ca\/edci565\/wp-json\/wp\/v2\/tags?post=711"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}