TRF050
One rotary module per model: attention classes must not instantiate their own rotary embedding.
| Default | Enabled |
| Scope | All models |
| Source | mlinter/trf050.py |
| Show in terminal | mlinter --rule TRF050 |
What it does
Checks __init__ methods of classes whose name ends in Attention for calls to a *RotaryEmbedding class.
Why is this bad?
The Model owns a single rotary_emb, builds inv_freq once, and passes cos/sin down as position_embeddings. A rotary module per attention layer duplicates buffers, recomputes frequencies per layer, and diverges from the interface contract that attention receives position_embeddings.
Example
class AcmeAttention(nn.Module):
def __init__(self, config, layer_idx):
super().__init__()
- self.rotary_emb = AcmeRotaryEmbedding(config)
class AcmeModel(AcmePreTrainedModel):
def __init__(self, config):
super().__init__(config)
+ self.rotary_emb = AcmeRotaryEmbedding(config)
Suppressing this rule
Add a # trf-ignore: TRF050 comment on the flagged line or the line directly above it. See Suppressing rules for whole-file directives and when a suppression is the wrong answer.
Allowlisted models
4 models are exempt from TRF050 in mlinter/rules.toml, because they predate the convention and cannot be changed without breaking backward compatibility.
Show the 4 allowlisted models
edgetam_videomoshirecurrent_gemmasam2_video