Hi, I am very interested in this series of work and have a question about the decoder architecture.
I have noticed that the decoder architecture is a little bit different from which in the original Transformer paper.
The differences are circled in red. The K in QKV of second attention block and the skip connection are from encoder in our methods, while from decoder originally.
Is it any improvement of the transformer architecture from some existing papers or just our design？
Hi, I am very interested in this series of work and have a question about the decoder architecture.
I have noticed that the decoder architecture is a little bit different from which in the original Transformer paper.
The differences are circled in red. The K in QKV of second attention block and the skip connection are from encoder in our methods, while from decoder originally.
Is it any improvement of the transformer architecture from some existing papers or just our design？