TODO.txt

Batch Parser Issues (TODO)
--------------------------

# DONE & at the end of line is skipped, &&, || or | at the end of line give an error.
  NOT CONFORM behavior: &&, || and | don't give an error at EOL.
  They are processed in the same way as &.

# DONE |&<>^() has no special meaning inside double quotes

# DONE ) treated as matched closing bracket (if unclosed one exists) or as a normal character

# DONE NOT CONFORM: EOF treated as closing of all unclosed brackets.
  It gives ability to extract commands from very corrupted files.

# DONE ^ escapes special chars |&<>() and @,

# DONE before any other char skipped, so ^X => X, ^^ => ^, echo before space just skipped

# DONE combine long lines by ^\n

# DONE strange case with ^\n: \n skipped and ^ acts over next char:
    echo 12 ^
    & echo 14
  treated as 'echo 12 ^& echo 14' (i.e. not 'echo 12 & echo 14')

# DONE: ^ escapes special meaning of " until special handling
  of ^"...^" strings implemented (see below).
  Note it's NOT CONFORM behavior.

# DONE parse labels. Label formats:
  '[any_char](blank)*:(blank)*(label_name)(blank)...'
  - label_name is a char sequence till first blank including quotes etc:
    'name' => 'name'
    '"name"; => '"name"'
    '"long name"' => '"long'
    'spe^&ial' => 'spe&ial'
    '"spe^&ial"' => '"spe&ial'
  - rest of label string is ignored
  - spaces are allowed before and after ':'
  - any single char including @ and special chars is allowed at the 
    first char of label string (before spaces and ':')
    NOT CONFORM: allowed
        - any number of @
        - any number of spaces before @
        - any number of spaces before "char before ':'"
        - both @ and "char before ':'"
  - label name should not be double quoted as normal arg. here, the double quote
    means just an ordinal char. note that 
  - caret is allowed right before or inside a label_name.
    Single caret is just skipped, double caret is translated into single caret
  - caret is allowed as last char of label_name. I can't determine what is label name in such case,
    but it's neither 'name^' nor 'name'
    NOT CONFORM: caret in last char of label_name is added to label_name (i.e. 'name^' produced).
  - caret is not allowed in blanks before and after ':'.

# SKIP "Fuck my brain" case with any_char before label:
  'c:\windows' is a valid label which could be called with 'call :\windows'.
  Now such expression checked to be a command first and then checked to be a label.
  So such label detection could be wrong.

# DONE parse rem in format 'rem comment text till the EOL'

# DONE '^\n' should be skipped before '(' and ')'. It's a valid case:
    (   ^
    (cmd)  ^
    )

# SKIP Strange case with caret and bracket.
    (   ^
    (cmd)  ^
    )
  Such code lead to error unless EOL, space or operator following the last closing bracket.
  NOT CONFORM: we treat such code as valid (And it's really valid!)

# DONE command redirect
    Command with redirect:
    @* ((redirect_op arg) | arg)+
    Where <redirect_op> is '[0-9]?>', '[0-9]?>>', '[0-9]?<'.

# SKIP strange redirect behavior in case it's inside a command and before the last command arg:
    'echo foo >f bar' => f: 'foo ' (should be 'foo  bar')
    'echo foo >f bar wow' => f: 'foo  bar wow'
  NOT CONFORM: not implemented

# DONE redirect before label ('> outfile :label1')
  Expression skipped. Not 100% sure, but can't get any facts of opposide.

# SKIP restriction on argument of handle redirection ('[0-9]>&[0-9]' and '[0-9]<&[0-9]').
  NOT CONFORM: later handle not checked to be a [0-9] but rather checked it's
  a normal redirection argument.

# DONE strange redirect when caret is present:
  FAIL (Prints 'foo > outfile'):
    echo foo ^
    > outfile
  
  OK (any spaces before '>' are OK):
    echo foo ^
     > outfile
  NOT CONFORM: prints extra arg ' ' after 'foo'
  
  OK:
    echo foo >^
    outfile
  
  Works as 'echo foo > " outfile"':
    echo foo >^
     outfile
  
  FAIL (2 or more spaces before 'outfile' not OK):
    echo foo >^
      outfile
  NOT CONFORM: doesn't fail, prints 'foo outfile'

  Works as 'echo foo > outfile'
    echo foo >^
     > outfile
  
  FAIL:
    echo foo >^
    > outfile
  NOT CONFORM: doesn't fail, prints 'foo outfile'

  Works as 'echo foo 1^>outfile':
    echo foo 1^
    >outfile

  Works as 'echo foo 1 >outfile':
    echo foo 1^
     >outfile
  
  Works as 'echo foo 1 >outfile':  
    echo foo ^
    1>outfile
  
  OK:
    echo foo 1>^
    outfile

# DONE group redirect
    (redirect_op arg)* (
    ...
    ) (redirect_op arg)*
  Could be on a single line. Carets could be used.

# DONE strange group redirect when caret is present:
  OK:  
    (echo foo) >^
    outfile
  
  OK:  
    (echo foo) >>^
    outfile
  
  OK:  
    (echo foo) ^
    > outfile
  
  Works as '(echo foo) > " outfile"':
    (echo foo) ^
     > outfile
  
  FAIL:
    (echo foo) ^
     > outfile
  NOT CONFORM: doesn't fail, prints 'foo'
  
  FAIL:  
    (echo foo) ^
    2> outfile
  NOT CONFORM: doesn't fail, prints 'foo'

  FAIL:
    (echo foo) ^
    >> outfile
  NOT CONFORM: doesn't fail, prints 'foo'

  FAIL:
    (echo foo) >^
    > outfile
  NOT CONFORM: doesn't fail, works as 
    (echo foo)
    outfile

  Works as '(echo foo) > outfile' (not '>>'):
    (echo foo) >^
     > outfile
  
  All types of pre redirection are OK:
    > outfile ^
    (echo foo)

# DONE parse IF command and extract commands under it
    IF [/I] [NOT] string1 == string2 command
    IF [/I] [NOT] string1 <compare_op> string2 command
        where compare_op is EQU, NEQ, LSS, LEQ, GTR or GEQ
    IF [/I] [NOT] ERRORLEVEL number command
    IF [/I] [NOT] EXIST filename command    
    IF [/I] [NOT] CMDEXTVERSION number command
    IF [/I] [NOT] DEFINED variable command

    Every if supports:
    IF ... (
        commands
    ) ELSE (
        commands
    )

# DONE working but not specified in IF command help:
    if ERRORLEVEL == number command
    if CMDEXTVERSION == number command

# DONE IF command with caret and group
  FAIL because of unknown command ('(echo', ' ' or '(')
    if ERRORLEVEL 0 ^
    (echo foo)
  or
    if ERRORLEVEL 0 ^
     (echo foo)
  or
    if ERRORLEVEL 0 ^
    ( echo foo)
  NOT CONFORM: doesn't fail, parsed as valid (ant it's really valid)
   
  OK:
    if ERRORLEVEL 0 ^
    echo foo

# DONE Crazy batch files: most commands and their arguments could be prefixed and/or suffixed with any amount of '=' which just skipped.
    Most commands support this. Checked ones:
        if, goto, copy, del
    At least next commands don't support this:
        set
    
    Note that 'echo' command act in slightly different way. First '=' is treated as command name end and skipped. The second '='
    and rest '=' are echoed. (One more reason to parse 'echo' in its own way)
    NOT CONFORM: extra '=' removed from all commands even 'set' and 'echo '

# DONE parse FOR command and extract commands under it
    FOR %variable IN (set) DO command
    FOR /D %variable IN (set) DO command
    FOR /R [[drive:]path] %variable IN (set) DO command
    FOR /L %variable IN (start,step,end) DO command
    FOR /F ["options"] %variable IN (file-set) DO command
    FOR /F ["options"] %variable IN ("string") DO command
    FOR /F ["options"] %variable IN ('command') DO command
    FOR /F ["options"] %variable IN (file-set) DO command
    FOR /F ["options"] %variable IN ('string') DO command
    FOR /F ["options"] %variable IN (`command`) DO command
        where "options" is something like "eol=; tokens=2,3* delims=, "
    
  Note "options" can't include '"' (same as normal argument).
  '(' and ')' can be used inside (set) when placed into double quotes.
    
  NOT CONFORM: no restriction for option (/D,R,L...) letter. No restriction for 
  options, path for /R option and "condition" (expression inside brackets).

# SKIP extract command from FOR command's "condition", i.e. FOR ... ('command') and FOR ... (`command`)
  Would not implement because case is rare so has no effect in overal file analyze.

# TODO fix too high counters (they are incremented on partial match during matching wrong alternative)
  Counter related matches should be moved out from alternatives: (a >> b) | (a >> c) => a >> (b|c).

# TODO: caret could be used in any command name or its argument (even in 'if', 'for' etc)
    ^<eol> should be implemented in lexr, not in parser. Don't forget to leave carets inside double quoted strings untouched.

# TODO label can be the first (and the only) expression at its line
    NOT CONFORM: label parsed as label even it found after another expression at line
    (like '<expr> <operand> <label>')

# TODO? strip ending spaces in quoted params which ends because of EOL/EOF:
  'call "123 456   ' => %1 is '"123 456' but we returns '"123 456   ' (note ending spaces)
  Could be solved by stripping out ending spaces for all params.
  NOT CONFORM: not implemented

# TODO? Wipe out empty params (but not ""). Should be done after stripping ending spaces.
  NOT CONFORM: not implemented

# TODO? parse echo specially 
  1) Merge all args into one without stripping spaces between
  2) Don't strip spaces between args and redirect(s)
  NOT CONFORM: not implemented

# SKIP ^" opens a strange type of quoted arg:
    - spaces allowed inside
    - arg should be closed with ^"
    - arg finished at first special char
    - if double quote found inside - execution stopped - 
    - if no double quote or special char found, line finished at 
      EOL/EOF as normal double quoted arg (i.e. at last non empty char in fact)
    
  call :print ^"12 34           ^
  echo 23 " 24
  =>
  call with %1 = '"12 34           echo 23 "', %2 = '24'
  
  call :print 12 ^" 34       & echo 23
  =>
  call: %1=12, %2:" 34, %3=,
  23
  
  call :print 12 ^" 34   "    & echo 23
  =>
  nothing outputed
  
  NOT CONFORM: not implemented