Análisis de metodologías de evaluación de sistemas de diálogo multimodal

Zusammenfassung

Multimodal dialogue systems represent a very challenging research field nowadays, as they enhance the interaction provided by spoken dialogue systems. This paper presents, in a summarised way, the results of a wide and deep study we have carried out to try to find the cues for the evaluation of these systems, as well as the main differences in comparison with the evaluation of spoken dialogue systems. Our conclusions indicate there are no methods clearly accepted by the scientific community that can be considered as evaluation standards. This fact, together with the diversity of tasks and differences in the system complexity, makes it very difficult to compare systems objectively. Thus, new techniques are necessary to ease the evaluation taking into account the specific features of each system, as well as new methods to compare systems designed for different tasks.