What is the purpose of assessment? Looking at assessment of, for, and as learning

Wednesday, 19th January 2022

Image courtesy of Lagos Techie. Unsplash

Assessment always has a purpose. We need to be clear on that purpose in our learning design work. But that purpose can vary. In this article aimed at educators, we explore assessment of, for, and as learning to think about the purpose of assessment and to help us think about integrated summative assessment.

In a previous article about online assessment, we asked ‘How do we know if students are learning?’ We spoke about the value of formative assessment as part of activity-based teaching and learning. We suggested that formative assessment activities form an important part of an integrated summative assessment strategy.

But what do we mean? Traditionally,

"Formative assessment occurs before or during teaching. It is a way of assessing students’ progress, providing feedback and making decisions about further instructional activities. It is assessment for learning purposes. Summative assessment is conducted after instruction primarily as a way to document what students know, understand and can do. It is an assessment of learning and its aim is to ‘sum up’ the learning that has taken place." (Waspe, 2020)

But Waspe goes on to say:

"There isn’t always a clear split between formative and summative assessment: some activities may fall somewhere in the middle. For example, a test at the end of a section of material may be used for marks (summative) but the lecturer may also analyse it to identify which competences need strengthening going forward (formative)." (Waspe, 2020)

Assessment and learning

We know there is an integral relationship between assessment and learning. We can see this when we unpack three important forms of assessment:

Assessment of learning

Assessment of learning measures the knowledge, skills, attitudes, and values students have demonstrated at the end of a learning cycle.

Assessment for learning

Assessment for learning checks what students understand and can do as part of the learning process. It helps to identify any misunderstandings, difficulties, or gaps in knowledge so educators can adjust teaching to address these.

Assessment as learning

Assessment as learning involves the student ‘thinking about how they are thinking’ and using what they discover to make adjustments to how they approach learning. Assessment as learning helps students notice their own thoughts and processes, called metacognition, and make changes to those thoughts and processes, called self-regulation.

Let’s think about these three forms of assessment in relation to formative and summative assessment.

In his article about assessment for learning, Wiliam (2011) agrees with Bennett (2009) that it is unhelpful, and simplistic, to equate assessment for learning with formative assessment and assessment of learning with summative assessment. Bennett suggests that assessments designed primarily to serve a summative function may also function formatively, while those designed primarily to serve a formative function may also function summatively.

Let’s look at an example:

This is an example of an activity in which students are learning about the impact of globalisation in an economics course. Let’s analyse the activity for opportunities for assessment of learning, assessment for learning, and assessment as learning.

Activity: The impact of globalisation
Purpose	Create a diagram which reflects the impact of globalisation on an industry of your choice
Resources	Reading A Reading B
Task [60 minutes]	In small groups, recall what you already know about globalisation, and share your thoughts. Individually research and read theories of globalisation, summarising the key ideas. Again in small groups, share your research findings. Use the theories to agree on a definition of globalisation and unpack different aspects of globalisation. Individually analyse each aspect of globalisation in relation to your chosen industry. Create a diagram which shows what you have learned. Write a short narrative summary to accompany the diagram, which shows your understanding of the impact of globalisation in your selected industry.
Reflect, share and respond	Share your final diagram and narrative with your group peers.
Feedback	Give and receive feedback from each other. Refine your work if necessary. Receive and read feedback from your educator.

Example adapted from OER Africa: https://www.oerafrica.org/supporting-distance- learners/case-studies-using-asynchronous-communication

In this example, we believe we can identify all three forms of assessment:

Assessment of learning	Assessment for learning	Assessment as learning
What students already know Agreeing on a definition and aspects of globalisation Creating a diagram Writing a narrative summary	Discussing with peers and critiquing final diagrams and narratives, students will identify what they don’t know and adapt their work accordingly. The educator could ask students to submit both their first revised diagrams and narratives, for comparison purposes. From the diagrams and narratives the educator will identify gaps and misconceptions to integrate into future teaching.	As part of the discussions, reflections, responses and feedback with each other and from their educator, students will be able to think about how they learned, what helped them to learn and how they can improve their learning strategies in future.One way of enhancing this aspect of assessment and learning would be to ask students explicit reflection questions about their learning.

Integrated activity-based summative assessment

What does this mean for how we think about integrated summative assessment?

Most definitions of integrated assessment include ideas about gathering and presenting evidence for judgement against standards, outcomes and criteria, using a combination of assessment methods and instruments, in different contexts, supporting learners to demonstrate understanding of theory in practice.

When we integrate assessment of, for, and as learning into activity-based teaching and learning, we design integrated (summative) assessment that supports integrated learning. In certain cases, it might be useful to think about summative (and formative) assessment as gathering evidence for the purposes of marking, recording, and promotion. But when we want to ‘stop worrying about testing and start thinking about learning’ and we are challenged to think about assessment differently for whatever reason, we can use activity-based teaching that integrates assessment for, of, and as learning to support and guide students’ learning towards success.

In the ‘Impact of globalisation’ example above, we saw the integration of assessment of, for, and as learning in a single activity. But integrated summative assessment could also be the integration of formative and summative assessment over a series of activities for a whole unit or module of study, or even a whole course.

The same basic rules of assessment will apply, so that activities are fair, reliable, and valid. Let’s consider what criteria and elements activity-based design would need to incorporate in order to constitute valid, fair, reliable, and integrated assessment of, for, and as learning:

Do the activities have a clear purpose that is clear to students?
Are the activities aligned to one or more outcomes?
Are the activities logically sequenced along a learning pathway?
Are the activities and tasks fit for purpose for different students in different contexts?
Is there a range of activities or tasks that give students the opportunity to engage and learn in different ways?
Are there clear guidelines that help students understand what they are expected to do individually or collaboratively?
Are all the activities accessible to all students, whether they have an Internet connection or not, whether they are on campus or not, whether they have access to devices or not?
Do all students have access to the resources necessary to do the activities?
Do the activities provide sufficient opportunities for students to collaborate?
Do the activities provide sufficient opportunities for students to reflect on their own and each other’s learning (self and peer reflection)?
Do the activities encourage students to give and receive feedback in meaningful ways?

Consider an assessment activity or task you have recently given to students. To what extent does the activity promote assessment of, for, and as learning? Which of the above criteria and elements does the activity address? What tasks can you add to the activity to ensure students have opportunities for assessment of learning, for learning, and as learning? What activities could you add, before or after this activity, to create a more integrated activity-based learning and assessment pathway as part of your materials design?

References

Bennett, R. E. (2009). A critical look at the meaning and basis of formative assessment (ETS RM-09-06). Princeton, NJ: Educational Testing Service.

Waspe, T and Louton, B. (2020). Rethinking TVET Assessment, Advanced Diploma Technical and Vocational Training, DHET. (See also https://nols.gov.za/dhetnols/)

Wiliam, Dylan. (2011). ‘What is assessment for learning?’ Studies in Educational Evaluation. 37. 3-14. 10.1016/j.stueduc.2011.03.001.

Access the OER Africa Communications Archive here

What's New

Examining technology’s role in education: UNESCO’s inaugural Digital Learning Week

September, 4, 2023

From 4 to 7 September 2023, we celebrate the inaugural Digital Learning Week – a reframing of what was previously known as Mobile Learning Week. The United Nations Educational, Scientific and Cultural Organisation (UNESCO) will convene in-person events for policymakers, practitioners, educators, private sector partners, researchers, and development agencies.

Image courtesy of Siphosihle Mkhwanazi, Wikimedia (CC BY-SA)

This week, we celebrate the inaugural Digital Learning Week – a reframing of what was previously known as Mobile Learning Week. From 4 to 7 September 2023, the United Nations Educational, Scientific and Cultural Organisation (UNESCO) will convene in-person events for policymakers, practitioners, educators, private sector partners, researchers, and development agencies.

Under the theme ‘Steering technology for education’, the event will explore public digital learning platforms and generative AI, examining how both can be steered to reinforce and enrich humanistic education.

Plenary sessions, panel discussions and public lectures will be livestreamed. The full programme, including links to each session, is available here.

Some highlights to look forward to include:

Release of Guidance for Generative AI in education and research
Presentation of AI Competency Frameworks for Students and Teachers
Release of UNESCO policy guidelines on digital learning and AI in education, including AI and Education: Guidance for Policymakers, Guidelines for ICT in Education Policies and Masterplans, Education and Blockchain, and K-12 AI curricula: a mapping of government-endorsed AI curricula
Progress report on Gateways to Public Digital Learning

For more information, visit the Digital Learning Week page.

Related articles

African Librarians Support the UNESCO OER Recommendation

August, 29, 2023

A William and Flora Hewlett Foundation grant to NBA funded the research and writing entailed in producing the Overview.

Figure 1: AfLIA poster on the UNESCO OER Recommendation

The UNESCO Recommendation on Open Educational Resources (OER) is significant to all those who are interested in and committed to ensuring that all learners have access to appropriate high-quality educational content, including librarians. It was approved unanimously by UNESCO member states in November 2019.

In August 2023, the African Library and Information Associations and Institutions (AfLIA) and Neil Butcher & Associates (NBA) co-published an Overview for African Librarians on the UNESCO OER Recommendation and Open Knowledge.[1] A William and Flora Hewlett Foundation grant to NBA funded the research and writing entailed in producing the Overview.

The Overview explores how the OER Recommendation’s five action areas are relevant to librarians and what librarians can do to support their implementation. It examines how the OER Recommendation relates to the different library types in Africa and the user communities the libraries represent.

It further aims to help African librarians develop a deeper understanding of OER, including the kinds of open content that will resonate with library users. OER is consonant with other equally important principles for librarians—access to equitable, suitable, and relevant content for easy sharing and interoperability of knowledge within Africa. All these facets are included in the Overview.

The Overview is filled with insights and stories from librarians on open knowledge and open licensing, including how traditional knowledge, culture, and languages can be used in creating and adapting openly licensed content.

AfLIA also produced a comic strip to explain to librarians why OER and UNESCO’s OER Recommendation are so important. Comic strips on teaching are becoming increasingly popular; Google has a full page of images, as does OER Commons. But we could not find a comic strip to explain open licensing or OER…until AfLIA came along and created one.

If you would like more information on the Overview or would be interested in joining related discussions, please write to Nkem Osuigwe, AfLIA’s Director of Human Capacity Development and Training. Her email address is neosuigwe@aflia.net.

Figure 2: AfLIA poster on collecting and opening up Africa's heritage

Related articles

[1] The document is available on both the AfLIA and NBA websites: https://web.aflia.net/wp-content/uploads/2023/08/UNESCO-OER-Recommendation-Overview-for-librarians-_230802_093332.pdf and https://www.nba.co.za/resource/unesco-oer-recommendation-and-open-knowledge-overview-african-librarians

Three ways Artificial Intelligence could change how we use Open Educational Resources

July, 28, 2023

Over the past year, news about Artificial Intelligence (AI) has abounded. Information about breakthroughs and new applications have become commonplace, and we have been thrust into a world where AI-enabled technologies are starting to change how we work and live.

In this article, we consider three ways that AI might change how we use OER.

Introduction

Many are asking what the implications of this might be for the education sector. Will it affect teaching and learning positively or negatively? How can AI-enabled technologies personalize education – and will this be educationally beneficial? What organizations are already working on AI in education and how, if at all, is this work regulated? An air of uncertainty pervades the sector in terms of the benefits and risks of harnessing AI in education.

At OER Africa, we have written extensively on how Open Educational Resources (OER) might improve aspects of education, including access, relevance, and quality. Likewise, the intersection of OER and AI necessitates greater exploration, particularly given the opportunities that it offers to scale access to high quality education.

In this article, we consider three ways that AI might change how we use OER.

How AI could change our engagement with OER

1. OER Content Development

AI tools can be used to develop new OER using natural language processing and machine learning capabilities. They might be able to help educators create interactive learning materials, assessments, and learning simulations, expanding the range of available OER and enabling novel teaching practices.

One of the most popular platforms that demonstrates such capabilities is ChatGPT (or Chat Generative Pre-Trained Transformer), an AI chatbot developed by OpenAI. Since its release in November 2022, is has amassed over 100 million users.

There is no doubt that generative tools like ChatGPT hold great potential to save time and effort for OER creators. With a few well-crafted prompts, ChatGPT can: generate thousands of words on a subject; create dozens of sample questions that could be included in an open textbook for learners to be able to self-evaluate their own learning; create lesson plans and assignments; and develop question prompts that can be used as asynchronous discussion prompts in discussion forums.^[1] However, it is crucial that OER creators use their own expertise to check that what is generated by ChatGPT is indeed correct.

For educators working with OER, copyright ownership of AI-generated works is important to determine as, by definition, OER are materials unencumbered by legal restrictions that may prevent the reuse, sharing, redistribution, and adaptation of copyrightable works. While some are using the rise of generative AI to question the validity of copyright itself, the question of who owns the copyright when a work is created by AI is a very murky area, both legally and ethically.^[2,3] When we asked ChatGPT whether the content it produces is openly licensed, it had the following to say:

As of my last update in September 2021, the content generated by ChatGPT and similar AI language models is not openly licensed. AI language models, including ChatGPT based on the GPT-3.5 architecture, are proprietary technologies developed by organizations like OpenAI.

…Therefore, when using content generated by ChatGPT or any other AI language model, it is essential to review the terms of service, usage policies, and any specific guidelines provided by the organization that owns the AI model to ensure compliance with their requirements.

It's worth noting that the field of AI and its legal and ethical implications are continually evolving, and there might be changes or developments in the licensing and usage of AI-generated content beyond my last update. I recommend checking with the organization that provides the AI service for the most current and accurate information regarding the licensing and usage of their AI-generated content.^[4]

We recommend that users of these technologies stay abreast of these kinds of debates, read terms of service of the organizations that create these technologies, pay close attention to licensing conditions, and state clearly when they have used AI tools to generate intellectual property.

2. Personalized Learning

Some AI algorithms can develop tailored recommendations for OER based on a learner's performance, learning preferences, and development areas. This implies that learners can use OER that meet their requirements, making the learning pathway more engaging and effective.

For example, Siyavula is a South African organization that provides personalized and adaptive learning platforms. Siyavula has produced book titles from Grades 4-12. These are high quality OER that are aligned with the South African curriculum for mathematics, physics and chemistry. Learners can now also access Siyavula’s adaptive learning software, which adjusts the difficulty levels of exercises through machine learning to cater to each learner’s individual needs.^[5]

3. Translation and Localization

AI can enable translation and localization of openly licensed content. Software with machine translation capabilities, such as Google Translate, can translate OER into different languages, facilitating knowledge sharing. It is always recommended that users state when they have used these kinds of tools for translation purposes.

The United Nations Educational, Scientific and Cultural Organization (UNESCO) has been exploring the use of AI for the translation and localization of educational content. It has collaborated with partners to develop machine translation systems and tools that bridge language gaps in accessing OER. For example, the Global Digital Library (GDL) collects existing high-quality openly licensed reading resources and makes them available on the web, mobile and for print. The platform also supports translation and localization of GDL-resources to more than 300 languages. UNESCO partnered with the GDL team under the auspices of the Norwegian Programme for Capacity Development in Higher Education and Research for Development (known as NORAD) and the Global Book Alliance to launch the GDL in Asia. Reading materials in 41 Asian languages, including seven Nepali languages were launched.^[6]

Despite the benefits of integrating AI into OER, there are several potential challenges and concerns. For example, the issue of data privacy has received a lot of attention recently, as the use of AI algorithms often entails the collection and analysis of user data. Ensuring that such data is stored securely and used responsibly is critical to maintaining the trust and privacy of both learners and educators.^[7]

A second challenge is the potential for AI to exacerbate existing inequalities in education. As AI-powered OER become more widespread, there is a risk that those who cannot access such resources or platforms may be left behind due to unstable internet connections for example. There may also be inherent biases in the data that is used to train AI models, such as a lack of data from Sub-Saharan African countries. Thus, introducing measures to ensure that AI-driven educational tools are accessible to learners regardless of their geography or socioeconomic contexts is key to promoting educational equity.^[8]

Conclusion

Regardless of one’s outlook on the impact that AI could have on society over time, its integration into most spheres of our lives in some shape or form is progressing fast. With regard to OER, AI offers exciting opportunities to augment the production, dissemination, and access to quality educational resources. However, rolling out such capabilities means that we need to consider potential shortfalls, including that we might inadvertently inhibit access to such platforms for those who face educational barriers.

Some further reading on this topic:

Lalonde, C. (2023). ChatGPT and Open Education. BC Campus. Available at: https://bccampus.ca/2023/03/06/chatgpt-and-open-education/
Downes, S. A look at the future of Open Educational Resources. An Introduction to Open Education. Available at: https://edtechbooks.org/open_education/a_look_at_the_future
Wiley, D. AI, Instructional Design, and AI. Improving Learning. Available at: https://opencontent.org/blog/archives/7129

Related articles in OER Africa’s archive

[1] Lalonde, C. (2023). ChatGPT and Open Education. BC Campus. Retrieved from: https://bccampus.ca/2023/03/06/chatgpt-and-open-education/

[2] Lalonde, C. (2023). ChatGPT and Open Education. BC Campus. Retrieved from: https://bccampus.ca/2023/03/06/chatgpt-and-open-education/

[3] See article here

[4] Conversation with ChatGPT on 24 July, 2023. OpenAI's ChatGPT, based on the GPT-3.5 architecture.

[5] See article here

[6] See a rticle here

[7] Frackiewicz, M. (2023). AI in Robotic Open Educational Resources. Retrieved from https://ts2.space/en/ai-in-robotic-open-educational-resources/#:~:text=AI%2Dpowered%20chatbots%2C%20for%20instance,thinking%20and%20problem%2Dsolving%20skills.

[8] Frackiewicz, M. (2023). AI in Robotic Open Educational Resources. Retrieved from https://ts2.space/en/ai-in-robotic-open-educational-resources/#:~:text=AI%2Dpowered%20chatbots%2C%20for%20instance,thinking%20and%20problem%2Dsolving%20skills

OER Africa Menu