OBOW exams at Wharton
This blog was pointed out me recently by Wing Lam which shows that OBOW-type exams are not something exclusive to U21Global.
This blog was pointed out me recently by Wing Lam which shows that OBOW-type exams are not something exclusive to U21Global.
Source: Sunday Times (Singapore)
Quite by chance today, I opened up a newspaper in a coffee shop and saw this man’s face in the Obituary column. I wonder how many Singaporeans under the age of 40 have heard of this man. If you are Singaporean and do not know who he is, ask your parents or grandparents and they will likely tell you about a brilliant young political activist in the 1960s whose talents as an orator were such that he attracted a huge following. A co-founder of the People’s Action Party (PAP), he was supposedly introduced in 1955 to David Marshall, then Singapore’s Chief Minister, as the future Prime Minister of Singapore. The introduction was made by a person named Lee Kuan Yew, someone Singaporeans will have heard of. An edited volume on Lim Chin Siong, Comet in our Sky, is well worth a read. The chapters contributed by T.N. Harper and Greg Poulgrain are particularly interesting in that they draw on British colonial records (released after a 30 year embargo) to explain how and why Lim fell from grace so dramatically. A review of this book was published in Pacific Affairs in 2003.
The National Endowment for Science Technology and the Arts (NESTA) Futurelab produce periodic literature reviews on education and technology. One of the more recent reports looks at e-assessment. (A pdf version is available here.) It’s a little UK-centric, but a jolly good read all the same, especially as ‘open web’ examinations get an honourable mention. A short critique of the report is also available at elearning-reviews.org.
One of the challenges faced by institutions offering global educational services using globally distributed faculty is arriving at globally acceptable grading standards. There are 7-point grading scales, 4-point grading scales, alpha-grading systems and plain old percentages. Each schema is a different ‘language’ in that if one has grown accustomed to one system, it is not always a straightforward process ‘translating’ your grades (and associated standards) into another system.
Take “70%” for example. What information does this convey? In the United States, a student would most likely be disappointed with such a low mark. In India, on the other hand, a student would be delighted to have performed so well. Herein lies the problem. If you have an Indian professor with a class largely comprising North Americans (or vice versa!) then, left to their own devices, it is a recipe for disaster.
One way around this is to grade to a standard set of assessment criteria and work from a standard set of grade descriptions. While not a perfect solution, it does make life a little easier come grade moderation time.
When attempting to articulate the virtues of edublogs today, someone asked how they offered anything different from discussion boards. Fumbling for a response, I observed that the learner has much more control than they do in a linear, pre-determined discussion thread. An article published in 2004 by James Farmer provides a more erudite explanation, part of which is reproduced below:
‘While discussion boards can be placed alongside content in packaged courses and with limited opportunities to use the technology in ways unforseen by the designer, a weblog is essentially free-form and there is little, besides providing templates, guidelines and facilitating the group as a whole that the teacher can do to actively impact on the technical structure of their experience. … [T]his is not to say that an anarchistic structure is appropriate but rather to suggest that one of the key attributes of weblogs is that they have within them “incorporated subversion” (Squires 1999) which allows learners to express themselves and explore their context in ways independent of the original designers intentions.’
Thus, educators might design not with constraint in mind, but with freedom and flexibility in mind … ‘this emphasises the active and purposeful role of learners in configuring learning environments to resonate with their own needs …’ (Squires 1999 p. 1).
I gave a seminar today at U21G on OBOW (open book, open-web) examination design principles. The ppt file is pretty large (7.55MB), so I have also saved it in pdf format (still big at 2.51MB!). The main purpose of this presentation is to provide practical advice on the construction of this type of assessment instrument.
A useful resource on authentic assessment has been developed by Jon Mueller of North Central College. The Authentic Assessment Toolbox web site won one of the 2004 Merlot Classics awards, and deservedly so. The focus is on the secondary education sector as is usually the case, but many of the key principles underpinning the authentic assessment movement are more broadly applicable. The ‘What is Authentic Assessment‘ link leads to a nice easy-to-absorb piece by Mueller that deals with the basics.
Often branded ‘sadistics’ by students, finding ways of engaging students to foster deep learning in statistics has always presented a challenge. The introduction of more authentic learning environments for the study of statistics within the MBA program at the Brisbane Graduate School of Business at QUT has been sufficiently successful to encourage students to opt for the follow on statistics subject! This paper to be presented at this year’s Online Learning and Teaching conference in Australia reports on the progress made.
I presented a paper on a methodology for authoring open book open web exams recently at the 2004 ASCILITE conference in Perth. A pdf version of my PowerPoint slides is available here.
In her widely acclaimed book Rethinking University Teaching (now in 2nd edition), Diana Laurillard has arguably provided the most persuasive theoretical case to date on how the various learning technologies might be effectively employed in the tertiary education sector. This is what she has to say about assessment:
‘There is an ongoing debate about whether we should assess what students know, or what they can do. The traditional modes of assessment of knowledge are seen as inadequate because they fail to assess students’ capability in the authentic activities of their discipline. The authentic assessment movement would instead reflect the complex performances that are central to a field of study; e.g. writing a position paper on an environmental issue, investigating a mathematical concept. The debate continues, questioning the validity of the claim that authentic assessment is a true measure of students’ capacity to generalise their learning to new situations. Given that students orient their study towards their perception of the assessment, the solution offered is to find more challenging forms of assessment. They must link to the learning aims and reveal what students have learned at a general level, rather than simply assess the technicalities, which leads to a more instrumental form of learning.’
– Diana Laurillard, Rethinking University Teaching (2002, p. 204).