Short-term recall comparison of iconic auditory and visual feedback stimuli in a memory game

Abstract

Multimedia user interfaces incorporate various feedback methods using different modalities. Cognitive processing of audiovisual information requires the ability to recall visual and auditory information, either separately, or in combination. Short-term memory capabilities vary individually and depend on factors such as signal presentation and the number and type of visual and auditory items. In an experiment involving 40 subjects, we aimed to compare short-term auditory and visual capabilities in a serious game application. Subjects played the ‘Pairs’ game at different resolutions, using either visual icons or audio samples, while the total time cost and number of flips were recorded. The results indicate that visual memory is not superior, and female subjects performed better than males at higher levels in the visual task. Additionally, human sound samples, speech and familiar auditory icons were found to be easier to recall than artificial measurement signals.