Are there any plans to provide some performance analysis tools for transformers?

[Hukongtao]

Feature request

pytorch-lightning provides a very useful performance analysis tool that can count the time spent at each stage in the model training process. Does the transformer framework have plans to integrate such a tool?
https://pytorch-lightning.readthedocs.io/en/1.5.10/advanced/profiler.html
Or is there any quick way to integrate the tools provided by pytorch-lighting?

Motivation

It can help analyze some performance bottlenecks encountered during model training

Your contribution

https://pytorch-lightning.readthedocs.io/en/1.5.10/advanced/profiler.html