[Question] Why the forward.shape different with backward.shape before 21.layer

[DengNingyuan]

Question

I use module.register_forward_hook and module.register_backward_hook to check the forward and backward results, then I find the shapes are different in some layers. Here are some examples, I want to know why they are different.

0 model.layers.0.self_attn.q_proj
forward_shape torch.Size([5, 1024, 24, 24])
backward_shape torch.Size([5, 576, 4096])

1 model.layers.0.self_attn.k_proj
forward_shape torch.Size([1, 577, 1024])
backward_shape torch.Size([1, 2691, 4096])

2 model.layers.0.self_attn.v_proj
forward_shape torch.Size([5, 577, 1024])
backward_shape torch.Size([1, 2691, 4096])