Evgeniia Razumovskaia

Source and Target Contributions to NMT Predictions

This is a post for the paper Analyzing the Source and Target Contributions to Predictions in Neural Machine Translation.

In NMT, the generation of a target token is based on two types of context: the source and the prefix of the target sentence. We show how to evaluate the relative contributions of source and target to NMT predictions and find that:
  • models suffering from exposure bias are more prone to over-relying on target history (and hence to hallucinating) than the ones where the exposure bias is mitigated;
  • models trained with more data rely on the source more and do it more confidently;
  • the training process is non-monotonic with several distinct stages.
October 2020