Using tangible user interfaces for technology-based assessment–Advantages and challenges

Please download to get full document.

View again

All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Share
Transcript
    Page 1 Using tangible user interfaces for technology-based assessment – Advantages and challenges Eric Ras, Valérie Maquil, Muriel Foulonneau, Thibaud Latour Public Research Centre Henri Tudor eric.ras@tudor.lu Abstract Assessing higher order thinking skills should measure knowledge,  procedure, as well as attitudes and dispositions. It can be considered as multidimensional regarding the cognitive processes involved to solve a complex problem, for instance. This paper summarises first experiences made with a so called tangible user interface (TUI) for assessment purposes. Due to the haptic dimension and the shared space of a TUI, the understanding of a  problem can be improved and skills can be assessed in a collaborative context. A simple matching item, suitable for measuring recall of factual knowledge, as well as a simulation item with the potential to assess higher order thinking skills are  presented. Introduction Innovation cycles get shorter and shorter. This impacts the way in which products and services are developed, but also influences how people access, process, and use information during daily work. The need and pressure to develop innovative solutions put more light on specific skills and competencies. These so-called 21 st  century skills refer to skills, such as, complex problem solving, creativity 1 , critical thinking, learning to learn, decision making, etc. (Binkley et al., 2012). Besides this, learning and 1  Center for Creative Learning, http://www.creativelearning.com/  (accessed 22 June 2012).  Using tangible user interfaces for technology-based assessment – Advantages and challenges   Page 2 working in a collaborative, connected context requires communication skills, negotiation skills, and skills for learning in the digital network (Griffin, Care, & McGaw, 2012). Educational frameworks on a global or national scale support teachers to cope with these new challenges – however, the knowledge about how to assess these skills still suffers of a very low level of understanding. Bennett and Gitomer state that “… knowledge about acquisition of 21 st  century skills and their development is very limited. Developers of assessment do not yet know how to create practical assessment using even this partial knowledge effectively (Bennett & Gitomer, 2009)”. Technology-based assessment may avoid existing time constraints, ease the comparison of results, reduce measurement errors, etc.(Csapó, Ainley, Bennett, Latour, & Law, 2012; Grundwald Associates LLC Report, 2010). Nevertheless, most of the research in technology-based assessment dealt with the improvement of assessment of traditional skills (Binkley et al., 2012). The purpose of this paper is to present first experiences of a so-called tangible user interfaces used for technology-based assessment. Exploratory, design-focused studies have suggested that TUIs provide some learning benefits, due to the additional haptic dimension, the better accessibility, and the shared space that can be used in collaborative situations (Marshall, 2007). Section 2 refers to 21 st  Century skill frameworks developed recently and reflects upon higher order thinking skills (HOTS). It will allow us to elaborate for which skills a TUI can be useful for assessment and how it should evolve in the future from a technical perspective. Related work on TUI for learning and assessment is presented in Section 3. Section 4 summarises first experiences made with a so-called tangible user interface (TUI) for assessment in science. Section 5 concludes our work. 21 st  Century skills The OECD 2  developed several programs such as PISA (Programme for International Student Assessment), PIAAC (Programme for the international Assessment of Adult Competences to assess skills on a large scale. A program for Assessment of Higher Education Learning Outcomes (AHELO) is planned for the future. Binkley et al. have organised the 21 st  Century skills into four groups and have proposed a model to assess those (Binkley et al., 2012): ways of thinking, ways of working, tools for working, and living in the world. For our work, ways of thinking is the most relevant. These skills refer to a) creativity and innovation, b) critical thinking, problem solving, decision making, c) learning to learn, metacognition. Ways of working covers communication and collaboration (i.e., teamwork). Projects, such as, the ATC21S 3  are developing methods to assess skills, which are part of 21st-century curricula. It concentrates on collaborative problem-solving and learning in digital networks (Haladyna, 2004): a) Collaborative problem solving   is defined as the capacity to recognise the perspective of other persons in a group, participate in a group and contribute knowledge, experience and expertise in a constructive way, recognise the need for contributions and how to manage them, identify structure and procedure in resolving a problem, and build knowledge and 2  Organisation for Economic Co-operation and Development, http://www.oecd.org/  (accessed 22 June 2012). 3  Assessment and Teaching of 21 st  Century Skills, http://atc21s.org/  (accessed 22 June 2012).  Using tangible user interfaces for technology-based assessment – Advantages and challenges   Page 3 understanding as member on a collaborative setting; b) Learning through a digital network   consist of learning as consumer and producer of information, learning in the development of social and intellectual capital (Griffin et al., 2012). JISC 4 , for example, funds a huge number of technology-based assessment projects in the UK. The Learning Literacies for the Digital Age   project (LLiDA) reviewed the kinds of capabilities valued, taught for and assessed and defined in competence frameworks. Their “framework of frameworks” emphasises on components of digital and learning literacy, which covers also communication and collaboration skills, as well as problem solving (Leighton, 2011). A comprehensive report on assessment in the digital age   highlights outcomes of case studies conducted in the UK. The authors conclude that assessment should also capture the process in addition to the learning outcome, that it is important to explore the potential of technology to enable more efficient use of practitioners time and effort, and should provide a digital environment in which all learning and assessment related activities take place where evidence is consolidated (JISC e-Learning team, 2010). Higher order thinking The term thinking is difficult to define and depends on the view on thinking and the purpose it should serve. In the book of Schraw et al. on assessing higher order thinking skills (HOTS) they state that thinking and human cognition in general is a goal-oriented activity: “Thinking is related to gather and evaluate information that is relevant to this goal; It requires constructing a meaning and conceptual representations that support the analysis of events around us; and it requires that we engage in strategic decision making and judgments that enhance our ability to self-regulate (Schraw & Robinson, 2011)”. According to Schraw et al. HOTS can be refined to reasoning   skills,  problem solving   and critical thinking  , argumentation   skills, and finally metacognition   (i.e., thinking about thinking). The assessment of thinking skills is related to three different outcomes: knowledge, procedure, as well as attitudes and dispositions (Schraw & Robinson, 2011). The first focuses on the assessment of facts, concepts, or mental models. Procedural outcomes refer to specific thinking skills and strategies, self-regulatory processes, or as stated by Schraw et al., content-based procedural skills. For real-time problem solving, for example, so-called learning traces, which log the actions performed by a learner. Analysis of notes taken, think aloud protocols, or eye tracking are just a few examples for assessing real-time problem solving skills. Computer-based simulation can assess complex thinking skills that cannot be measured by more traditional assessment methods (Lane, 2011). An additional value of simulations is the way how students interact with the tools provided in the problem solving space and the possibility to record how students use them. Both outcomes, as well as strategies and processes can be measured. In the following, the tangible user interface will be explained by means of a matching and a simulation item. Tangible user interfaces for learning and assessment While current, web-based e-assessment frameworks exploit multimedia capabilities of graphical user interfaces to support a large range of different questions types, 4  Joint Information Systems Committee, http://www.jisc.ac.uk (accessed 22 June 2012).  Using tangible user interfaces for technology-based assessment – Advantages and challenges   Page 4 their possibilities for supporting and measuring team-oriented, creative, and communication skills in a collaborative context are extremely limited. Even if some platforms (e.g., TAO (Ras, Swietlik, Plichart, & Latour, 2010) 5  which is currently used in PISA and PIAAC) are designed to accommodate multi-test-taker networked assessments together with log collection and analysis capacities, they still lack delivery modes that can capture human-to-human collaboration in a natural setting. Hence, we need to explore new technologies that better support and allow collaborative activities, enhance understanding through more natural activities and realistic problems, and finally allow us to gather activities traces. If proven effective, they might open new avenues for innovative modes to be included in the assessment platforms. Using a new technology in assessment requires first a deep understanding with regard to the actions the users perform, the group dynamics in collaborative settings, as well as the required digital literacy to use such a new technology. These aspects may impact in which the results of assessments are obtained (i.e., measured) or they may produce undesired effects (i.e., measurement errors). Tangible user interfaces (TUIs) are an approach to create new types of interactions combining physical artefacts with digital visualisations in a common interactive space. The idea of TUIs is to make computer bits tangible and to allow users to grasp and manipulate them with their hands (Antle, 2007). This concept has a number of advantages for social and contextual interactions, such as collaboration (Hurtienne, Weber, & Blessing, 2008). The use of TUI is twofold: 1) using the TUI for innovation and modelling tasks: creating new services is an innovative, creative, and collaborative way; using TUIs for complex modelling tasks in industrial settings, for example, modelling business process models; 2) the previously mentioned tasks rely on 21 st  Century skills and hence provide an ideal setting with realistic problems to assess those skills. In literature, we can find a number of examples, demonstrating learning environments using TUIs, without emphasising on assessment. For example, the Chromatorium (Rogers, Scaife, Gabrielli, Smith, & Harris, 2002) is an environment where children may discover and experiment with mixing of colours. In a similar type of setup students may learn about the behaviour of light (Price, Pontual Falcão, Sheridan, & Roussos, 2009). Through physically manipulating a torch and blocks on a table surface, the students could explore how projected light beams react and understand the concepts of reflection, absorption, transmission, and refraction. Another interactive tabletop called TinkerLamp provides a simulation environment for learning purposes where students design a warehouse (Do-Lenh, Jermann, Cuendet, Zufferey, & Dillenbourg, 2010). A kind of learning system implements the concept of digital manipulatives (Resnick et al., 1998), computationally enhanced building blocks, which allow the exploration of abstract concepts. This principle is followed by SystemBlocks and FlowBlocks, two physical, modular interactive systems, which children can use to model and simulate dynamic behaviour (Zuckerman & Arida, 2005). The approach of concept mapping for self-regulated learning is used in (Oppl, Steiner, & Albert, 2010): A tangible tabletop allows users to reflect and evaluate their learning tasks through externalising and representing their knowledge on concept maps represented through physical and digital elements. Users can, for instance, 5  http://www.tao.lu  Using tangible user interfaces for technology-based assessment – Advantages and challenges   Page 5 place physical tokens, assign names, make connections, and use tokens as containers. Only a few systems go beyond the learning aspects, and implement possibilities for assessment. The Learning Cube (Terrenghi, Kranz, Holleis, & Schmidt, 2006) acts as a tangible learning platform where multiple choice tests can be done. By turning the cube, the user selects the right answer for a given question and then shakes it to select it. Another example provides new possibilities for assessing spatial and constructional ability. The Cognitive Cubes (Sharlin et al., 2002) are a set of construction cubes that are provided for building a 3D shape shown on a screen. The change of each shape is recorded and scored for assessment. Although a number of publications can be found that describe different kinds of strengths of TUIs for learning environments, there is still a lack of knowledge for using TUIs as tool for assessment. To the knowledge of the authors, there is no prior work addressing the development and measurement of higher order thinking skills in a mixed physical – digital environment, such as TUIs. First results with a tangible user interface Two example test items with a TUI are presented next. The first one, a matching item assesses recall of factual knowledge, whereas a second example is a performance item using simulation, which shows potential to measure HOTS in the future. The purpose of both studies was explorative, which means to observe the users human-to-human behaviour and interactions with the objects of table. The goal was not to develop a comprehensive assessment model and method, which will be targeted in the near future, after having observed the system in practice first. Matching item The implemented task was to assign the correct name of a planet to the correct image of a planet (according to QTI 6  standard: associate   item). When a user places a card onto a planet, the user gets an immediate feedback in form of a red (false answer) or green (correct answer) circle that is shown around the planet. During the solving of the task, the system counts the number of attempts (i.e., wrong pairing of a card and a planet’s image was counted as one attempt), and the time needed to solve the task. The matching item allows recalling factual knowledge about a particular topic. According to the revised taxonomy of Bloom, such a task can be classified under the level “remember” and the cognitive process “recognise” (Anderson & Krathwohl, 2001). Facts are recalled from long-term memory and associated to the projected images of the planets. Hence, the task consists to match verbal or pictorial representations of the solar system objects. For the experiment, we set up a tangible tabletop system, based on the optical tracking framework reactivision 7 . The worktop has an interactive area of 75x100cm. On the table, we projected an image of the solar system, showing the sun and each of the nine planets. The position of the planets did not reflect the distance to the 6  IMS Question & Test Interoperability Specification, http://www.imsglobal.org/question (accessed 22 June 2012). 7  A toolkit for tangible multi-touch surfaces, http://reactivision.sourceforge.net/  (accessed 22 June 2012).
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks
SAVE OUR EARTH

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...

Sign Now!

We are very appreciated for your Prompt Action!

x