Is there a way for a lexer in mode 'y' to be able to reuse token definitions from mode 'x'?

[Korporal]

When the lexer issues a pushMode(y) operation, it seems that all tokens needed in mode y must be defined, along with unique names across the entire lexer file.

But if lexer mode y also uses some of the tokens defined in mode x then it would make sense to be able to "reuse" them, but we can't do that it seems.

So consider IDENTIFIER for example, id mode x defines that and we transition to mode y, we must copy/paste the rules for it and give it a different name, this is a large duplication of effort and repetition.

So, is there way that I simply don't know about? if not, is this something to consider for a future upgrade?

[kaby76]

Fragment rules are scoped by the entire file, and a fragment can be reused anywhere. In other words, fold the token rule into a fragment rule, then use the fragment symbol on the right-hand side of the production specific to a mode. It may not be exactly what you are looking for, but it allows for the reuse of the majority of the original rule across rules in multiple modes.

[Korporal]

Fragment rules are scoped by the entire file, and a fragment can be reused anywhere. In other words, fold the token rule into a fragment rule, then use the fragment symbol on the right-hand side of the production specific to a mode. It may not be exactly what you are looking for, but it allows for the reuse of the majority of the original rule across rules in multiple modes.

@kaby76

I think this is doable as you explained. We can refer to the same fragment inside multiple mode sections. So we can use some common base token defined as a fragment and then define the per-mode tokens as simply being that. This works and so the actual lexical definition is never repeated.