How to specify arbitrary order

[Korporal]

Look at this rule:

entry_information
    :   parameter_name_commalist?  ((returns_descriptor? coprocedure_specifier?) | (coprocedure_specifier? returns_descriptor?))
    ;

I've written it like this because this did not work:

entry_information
    :   parameter_name_commalist?  (returns_descriptor? | coprocedure_specifier?) 
    ;

The latter rule recognized the returns_descriptor but not the coprocedure_specifier. It reported an error when it was present.

I thought that having (a | b | c) meant one could have one or more of the three and in any order too, is that wrong? have I misunderstood?

In a programming language, in which all of these are valid:

dcl variable static decimal fixed;
dcl variable decimal static fixed;
...
dcl variable fixed static decimal;

The list of possible combinations grows as we add more optional terms. So this is my problem, is there a way to define a grammar rule that says we expect "dcl" then IDENTIFIER then zero_or_more_in_any_order_of("static", "decimal", "fixed") ?

Or must I define a distinct rule for each of all the possible orderings of the terms?

[RossPatterson]

I thought that having (a | b | c) meant one could have one or more of the three and in any order too, is that wrong? have I misunderstood?

Nope. (a | b | c) means exactly one of either a or b or c. To get one or more, you need something like (a | b | c)+. But that would not restrict it to single occurances of a, b, and c (e.g., it would allow things like a b a c b c a b). The Rexx language, influenced in part by the PL/I syntax, has cases of this issue to. Its DO instruction, for example, allows the TO, FOR, and BY clauses to appear in any order. The Rexx ANTLR grammar supports this via:

do_                         :   KWD_DO do_rep? ...
  do_rep                    :   assignment do_cnt?
                            |   ...
                            ;
    do_cnt                  :   dot dob? dof?
                            |   dot dof? dob?
                            |   dob dot? dof?
                            |   dob dof? dot?
                            |   dof dot? dob?
                            |   dof dob? dot?
                            ;
      dot                   :   KWD_TO expression ;
      dob                   :   KWD_BY expression ;
      dof                   :   KWD_FOR expression ;

It ain't pretty, but it works. The formal Rexx syntax specification (ANSI X3J18-199X) looks even uglier:

 do_repetitive := 'DO' dorep | ...
   dorep := assignment [docount] | ...
     docount := dot [dobf] | dob [dotf] | dof [dotb]
       dobf := dob [dof] | dof [dob]
       dotf := dot [dof] | dof [dot]
       dotb := dot [dob] | dob [dot]
       dot := 'TO' toexpr
         toexpr := expression
       dob := 'BY' byexpr
         byexpr := expression
       dof := 'FOR' forexpr
         forexpr := expression

[Korporal]

I thought that having (a | b | c) meant one could have one or more of the three and in any order too, is that wrong? have I misunderstood?

Nope. (a | b | c) means exactly one of either a or b or c. To get one or more, you need something like (a | b | c)+. But that would not restrict it to single occurances of a, b, and c (e.g., it would allow things like a b a c b c a b). The Rexx language, influenced in part by the PL/I syntax, has cases of this issue to. Its DO instruction, for example, allows the TO, FOR, and BY clauses to appear in any order. The Rexx ANTLR grammar supports this via:

do_                         :   KWD_DO do_rep? ...
  do_rep                    :   assignment do_cnt?
                            |   ...
                            ;
    do_cnt                  :   dot dob? dof?
                            |   dot dof? dob?
                            |   dob dot? dof?
                            |   dob dof? dot?
                            |   dof dot? dob?
                            |   dof dob? dot?
                            ;
      dot                   :   KWD_TO expression ;
      dob                   :   KWD_BY expression ;
      dof                   :   KWD_FOR expression ;

It ain't pretty, but it works. The formal Rexx syntax specification (ANSI X3J18-199X) looks even uglier:

 do_repetitive := 'DO' dorep | ...
   dorep := assignment [docount] | ...
     docount := dot [dobf] | dob [dotf] | dof [dotb]
       dobf := dob [dof] | dof [dob]
       dotf := dot [dof] | dof [dot]
       dotb := dot [dob] | dob [dot]
       dot := 'TO' toexpr
         toexpr := expression
       dob := 'BY' byexpr
         byexpr := expression
       dof := 'FOR' forexpr
         forexpr := expression

Many thanks for that clarification! I wonder then if there's a case to be made for enriching the Antlr grammar here? My grammar is in fact derived from PL/I too. In my case I can tolerate multiple occurrences of any term, that is handled at a later phase of processing to its fine if its accepted by Antlr.

[michael-the1]

That would be super nice, I also run into this when I try to parse a Snowflake SQL statement.

-- These two are the same
CREATE TABLE t1 (i INTEGER UNIQUE NOT NULL);
CREATE TABLE t2 (i INTEGER NOT NULL UNIQUE);

-- This is obviously silly :)
CREATE TABLE t2 (i INTEGER UNIQUE UNIQUE);

In my case, there are five options here (UNIQUE, NOT NULL, PRIMARY KEY, COLLATE, DEFAULT) that can go in an arbitrary order so there are !5 combinations, which I'd rather not all type out 😅

For now I just went with something like:

(unique
| not_null
| primary_key
| collate
| default
)*

which is not strictly correct. But it's acceptable since you will probably never see the repetition in real life, because it wouldn't run on your query executor.

[Korporal]

That would be super nice, I also run into this when I try to parse a Snowflake SQL statement.

-- These two are the same
CREATE TABLE t1 (i INTEGER UNIQUE NOT NULL);
CREATE TABLE t2 (i INTEGER NOT NULL UNIQUE);

-- This is obviously silly :)
CREATE TABLE t2 (i INTEGER UNIQUE UNIQUE);

In my case, there are five options here (UNIQUE, NOT NULL, PRIMARY KEY, COLLATE, DEFAULT) that can go in an arbitrary order so there are !5 combinations, which I'd rather not all type out 😅

For now I just went with something like:

(unique
| not_null
| primary_key
| collate
| default
)*

which is not strictly correct. But it's acceptable since you will probably never see the repetition in real life, because it wouldn't run on your query executor.

@michael-the1

Maybe here, in this thread, we can discuss possible options for this. It strikes me as being primarily a notation question, we seek a notation to succinctly represent the kind of stuff presented by @RossPatterson stuff like this:

do_                         :   KWD_DO do_rep? ...
  do_rep                    :   assignment do_cnt?
                            |   ...
                            ;
    do_cnt                  :   dot dob? dof?
                            |   dot dof? dob?
                            |   dob dot? dof?
                            |   dob dof? dot?
                            |   dof dot? dob?
                            |   dof dob? dot?
                            ;
      dot                   :   KWD_TO expression ;
      dob                   :   KWD_BY expression ;
      dof                   :   KWD_FOR expression ;

In his example there are three terms involved (to, by, from) and if more terms get added the complexity grows, there's no way to add another term without restating several of the existing rules.

If we can devise a suitable notation, one that doesn't break existing Antlr syntax, then we might be able to present this to Terence and get his support.

I would find it a huge help and I think without it, today, people are likely imposing order on users not because the order has sematic relevance but because it is too much effort to do otherwise.

[Korporal]

So how about something like this:

W:       (X? ^ Y? ^ Z?);

Perhaps a simple tool could be created that can accept that and generate the expansion? as a proof of concept?