Sequence to Sequence Learning with Neural Networks: from Characters to Words and Beyond
Sequence-to-Sequence learning (Sutskever et. al, 2014) equipped with the attention mechanism (Bahdanau et. al, 2015) became the state-of-the-art approach to machine translation, automatic summarization and many other natural language processing tasks, while being much simpler than the previously dominant methods. In this talk we will overview sequence-to-sequence learning with neural networks, followed by two of our recent contributions: the Hard Attention mechanism for linear time inference, and the String-to-Tree model for syntax-aware neural machine translation.