Huang W, Jin R, Zhang W, et al, Joint Training and Decoding for Multilingual End-to-End Simultaneous Speech Translation, ICASSP 2023. (顶级国际会议)

Abstract: Recent studies on end-to-end speech translation(ST) have facilitated the exploration of multilingual end-to-end ST and end-to-end simultaneous ST. In this paper, we investigate end-to-end simultaneous speech translation in a one-to-many multilingual setting which is closer to applications in real scenarios. We explore a separate decoder architecture and a unified architecture for joint synchronous training in this scenario. To further explore knowledge transfer across languages, we propose an asynchronous training strategy on the proposed unified decoder architecture. A multi-way aligned multilingual end-to-end ST dataset was curated as a benchmark testbed to evaluate our methods. Experimental results demonstrate the effectiveness of our models on the collected dataset.

摘要：近期的研究促进了端到端（end-to-end）多语种语音翻译（ST）和端到端同声传译（simultaneous ST）的发展。在本文中，我们研究了一对多多语言设置中的端到端同声传译，这更接近真实场景中的应用程序。在此场景中，我们探索了用于联合同步训练的单独解码器架构和统一架构。为了进一步探索跨语言的知识传递，我们提出了一种关于所提出的统一解码器架构的异步训练策略。我们策划了一个多路对齐的多语言端到端 ST 数据集作为基准测试平台来评估我们的方法。实验结果表明，我们的模型在收集的数据集上的有效性。