Facilitating Conversational Interaction in Natural Language Interfaces for Visualization
* Equal contribution
IEEE VIS · 2022
Abstract
Natural language (NL) toolkits enable visualization developers, who may not have a background in natural language processing (NLP), to create natural language interfaces (NLIs) for end-users to flexibly specify and interact with visualizations. However, these toolkits currently only support one-off utterances, with minimal capability to facilitate a multi-turn dialog between the user and the system. Developing NLIs with such conversational interaction capabilities remains a challenging task, requiring implementations of low-level NLP techniques to process a new query as an intent to follow-up on an older query. We extend an existing Python-based toolkit, NL4DV, that processes an NL query about a tabular dataset and returns an analytic specification containing data attributes, analytic tasks, and relevant visualizations, modeled as a JSON object. Specifically, NL4DV now enables developers to facilitate multiple simultaneous conversations about a dataset and resolve associated ambiguities, augmenting new conversational information into the output JSON object. We demonstrate these capabilities through three examples: (1) an NLI to learn aspects of the Vega-Lite grammar, (2) a mind mapping application to create free-flowing conversations, and (3) a chatbot to answer questions and resolve ambiguities.
Citation
@article{mitra2022conversational,
author = {Mitra, Rishab and Narechania, Arpit and Endert, Alex and Stasko, John},
booktitle = {2022 IEEE Visualization and Visual Analytics (VIS)},
title = {Facilitating Conversational Interaction in Natural Language Interfaces for Visualization},
year = {2022},
volume = {},
number = {},
pages = {6-10},
doi = {10.1109/VIS54862.2022.00010},
url = {https://doi.org/10.1109/VIS54862.2022.00010}
}