Handle keywords in macro signatures

[MilesCranmer]

It would be great if macros could natively handle keyword parsing, such as

julia> macro foo(arg; kw1, kw2=1)
           #= ... =#
       end
@foo (macro with 1 method)

julia> @foo 1 + 1 kw1=bar

Many macros in the Julia ecosystem and standard library handle keywords, but do so by manually parsing the input expressions and unpacking any matching Expr(:kw, ...). This results in developers handrolling their own parsers, such as @btime:

macro btime(args...)
    _, params = prunekwargs(args...)
    bench, trial, result = gensym(), gensym(), gensym()
    #= ... =#
end

#= ... =#

function prunekwargs(args...)
    @nospecialize
    firstarg = first(args)
    if isa(firstarg, Expr) && firstarg.head == :parameters
        return prunekwargs(drop(args, 1)..., firstarg.args...)
    else
        core = firstarg
        params = collect(drop(args, 1))
        for ex in params
            if isa(ex, Expr) && ex.head == :(=)
                ex.head = :kw
                if ex.args[1] == :evals
                    push!(params, :(evals_set = true))
                end
            end
        end
        if isa(core, Expr) && core.head == :kw
            core.head = :(=)
        end
        return core, params
    end
end

#= several other functions involved too =#

I think it would be nice if Julia itself could automatically handle this, rather than having each macro definition parse keywords in its own way. The dream is for the following syntax to work:

macro btime(ex;
            setup=:(), teardown=:(),
            samples=10000, seconds=5, evals=1, overhead=0, gctrial=true, gcsample=false,
            time_tolerance=0.05, memory_tolerance=0.01) #= no default => required parameter =#
    #= ... =#
end

This would mean that developers wouldn't need to write a keyword parser in each new macro. Overall I think this would result in cleaner, more readable, more robust code.

---

I couldn't find an issue for this but please link if there is one. The original PR to throw errors on keywords is here: #15913. It was also requested in a comment here: #15896 (comment).

[lgoettgens]

I like the idea of adding this to the language. However, this must be attempted very carefully to not break any existing macro in the ecosystem that uses exactly such a custom parsing.

[MilesCranmer]

Indeed. I think what I would do is:

1. If there is a matching keyword, use that.
2. Otherwise, it is treated as an arg (like an assignment expression).

I think this would match all existing semantics for packages that already manually parse keywords.

For example:

macro foo(args...; bar=1)
    return nothing
end

If you were to call:

@foo x bar=2

it would route to the keyword. But if you instead call

@foo x baz=3

it would get passed in as :(x), :(baz=3), and leave the default bar=1 in place. There would be no error for lack of a match, so that keyword macros permit legitimate assignment expressions.

[mikmoore]

For some bikeshedding, it seems the dominant convention in Base is for arguments and keywords to come before the primary expression of a macro. Examples include @code_llvm debuginfo=:none EXPR, @simd ivdep, @time "DESCRIPTION" EXPR, @eval MODULE EXPR. Although this is the opposite of the BenchmarkTools.jl convention. I'm sure the full ecosystem has even more varieties.

But some functionality to handle this ergonomically (or better direction to documentation on existing functionality) seems nice. Although it's not clear there's a one-size-fits-all solution.

[MilesCranmer]

Yeah, this is tricky. Base also isn't entirely consistent on this—for example, @warn "my warning" maxlog=1 treats maxlog as a keyword parameter.

One thing to note is that the old way of manually writing a keyword parser would still continue to work, it is just that this approach allows for optional automatic keyword parsing should you so desire it.

Because of this opt-in behavior, you could choose either:

1. One syntax be chosen—before vs. after?
2. Or should both be allowed?

Either choice wouldn't affect existing macros, since parsing args would still work.

For 2., for example, if I write:

macro code_llvm(ex; debuginfo=:default, raw=true, optimize=true)
    return nothing
end

Could I then call it with either @code_llvm debuginfo=:none ex or @code_llvm ex debuginfo=:none—with both correctly mapping debuginfo to the keyword argument? Or should only one option work?

[MilesCranmer]

I might lean towards handling both cases with the automatic keyword syntax. With the exception that it wouldn't handle both cases in the same macro call, nor would it permit putting any keywords in between args of the argument list.

So, for

macro foo(args...; kw1, kw2)
    return nothing
end

you could allow the following, with two expressions passed:

• @foo ex1 ex2 kw1=1 kw2=2 ✅
• @foo ex1 ex2 kw2=2 kw1=1 ✅
• @foo kw1=1 kw2=2 ex1 ex2 ✅
• @foo kw1=1 ex1 ex2 kw2=2 ❌
• @foo ex1 kw1=1 kw2=2 ex2 ❌