Stable version 0.3.8. Now GPLv3!

This release is dedicated to Richard Stallman.
Happy hacking, and may we all live in a free GNUtopia!

LICENSE

@@ -1,5 +1,5 @@
-                    GNU AFFERO GENERAL PUBLIC LICENSE
+                    GNU GENERAL PUBLIC LICENSE
-                       Version 3, 19 November 2007
+                       Version 3, 29 June 2007
 
  Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
  Everyone is permitted to copy and distribute verbatim copies
@@ -7,15 +7,17 @@
 
                             Preamble
 
-  The GNU Affero General Public License is a free, copyleft license for
+  The GNU General Public License is a free, copyleft license for
-software and other kinds of works, specifically designed to ensure
+software and other kinds of works.
-cooperation with the community in the case of network server software.
 
   The licenses for most software and other practical works are designed
 to take away your freedom to share and change the works.  By contrast,
-our General Public Licenses are intended to guarantee your freedom to
+the GNU General Public License is intended to guarantee your freedom to
 share and change all versions of a program--to make sure it remains free
-software for all its users.
+software for all its users.  We, the Free Software Foundation, use the
+GNU General Public License for most of our software; it applies also to
+any other work released this way by its authors.  You can apply it to
+your programs, too.
 
   When we speak of free software, we are referring to freedom, not
 price.  Our General Public Licenses are designed to make sure that you
@@ -24,34 +26,44 @@ them if you wish), that you receive source code or can get it if you
 want it, that you can change the software or use pieces of it in new
 free programs, and that you know you can do these things.
 
-  Developers that use our General Public Licenses protect your rights
+  To protect your rights, we need to prevent others from denying you
-with two steps: (1) assert copyright on the software, and (2) offer
+these rights or asking you to surrender the rights.  Therefore, you have
-you this License which gives you legal permission to copy, distribute
+certain responsibilities if you distribute copies of the software, or if
-and/or modify the software.
+you modify it: responsibilities to respect the freedom of others.
 
-  A secondary benefit of defending all users' freedom is that
+  For example, if you distribute copies of such a program, whether
-improvements made in alternate versions of the program, if they
+gratis or for a fee, you must pass on to the recipients the same
-receive widespread use, become available for other developers to
+freedoms that you received.  You must make sure that they, too, receive
-incorporate.  Many developers of free software are heartened and
+or can get the source code.  And you must show them these terms so they
-encouraged by the resulting cooperation.  However, in the case of
+know their rights.
-software used on network servers, this result may fail to come about.
-The GNU General Public License permits making a modified version and
-letting the public access it on a server without ever releasing its
-source code to the public.
 
-  The GNU Affero General Public License is designed specifically to
+  Developers that use the GNU GPL protect your rights with two steps:
-ensure that, in such cases, the modified source code becomes available
+(1) assert copyright on the software, and (2) offer you this License
-to the community.  It requires the operator of a network server to
+giving you legal permission to copy, distribute and/or modify it.
-provide the source code of the modified version running there to the
-users of that server.  Therefore, public use of a modified version, on
-a publicly accessible server, gives the public access to the source
-code of the modified version.
 
-  An older license, called the Affero General Public License and
+  For the developers' and authors' protection, the GPL clearly explains
-published by Affero, was designed to accomplish similar goals.  This is
+that there is no warranty for this free software.  For both users' and
-a different license, not a version of the Affero GPL, but Affero has
+authors' sake, the GPL requires that modified versions be marked as
-released a new version of the Affero GPL which permits relicensing under
+changed, so that their problems will not be attributed erroneously to
-this license.
+authors of previous versions.
+
+  Some devices are designed to deny users access to install or run
+modified versions of the software inside them, although the manufacturer
+can do so.  This is fundamentally incompatible with the aim of
+protecting users' freedom to change the software.  The systematic
+pattern of such abuse occurs in the area of products for individuals to
+use, which is precisely where it is most unacceptable.  Therefore, we
+have designed this version of the GPL to prohibit the practice for those
+products.  If such problems arise substantially in other domains, we
+stand ready to extend this provision to those domains in future versions
+of the GPL, as needed to protect the freedom of users.
+
+  Finally, every program is threatened constantly by software patents.
+States should not allow patents to restrict development and use of
+software on general-purpose computers, but in those that do, we wish to
+avoid the special danger that patents applied to a free program could
+make it effectively proprietary.  To prevent this, the GPL assures that
+patents cannot be used to render the program non-free.
 
   The precise terms and conditions for copying, distribution and
 modification follow.
@@ -60,7 +72,7 @@ modification follow.
 
   0. Definitions.
 
-  "This License" refers to version 3 of the GNU Affero General Public License.
+  "This License" refers to version 3 of the GNU General Public License.
 
   "Copyright" also means copyright-like laws that apply to other kinds of
 works, such as semiconductor masks.
@@ -537,45 +549,35 @@ to collect a royalty for further conveying from those to whom you convey
 the Program, the only way you could satisfy both those terms and this
 License would be to refrain entirely from conveying the Program.
 
-  13. Remote Network Interaction; Use with the GNU General Public License.
+  13. Use with the GNU Affero General Public License.
-
-  Notwithstanding any other provision of this License, if you modify the
-Program, your modified version must prominently offer all users
-interacting with it remotely through a computer network (if your version
-supports such interaction) an opportunity to receive the Corresponding
-Source of your version by providing access to the Corresponding Source
-from a network server at no charge, through some standard or customary
-means of facilitating copying of software.  This Corresponding Source
-shall include the Corresponding Source for any work covered by version 3
-of the GNU General Public License that is incorporated pursuant to the
-following paragraph.
 
   Notwithstanding any other provision of this License, you have
 permission to link or combine any covered work with a work licensed
-under version 3 of the GNU General Public License into a single
+under version 3 of the GNU Affero General Public License into a single
 combined work, and to convey the resulting work.  The terms of this
 License will continue to apply to the part which is the covered work,
-but the work with which it is combined will remain governed by version
+but the special requirements of the GNU Affero General Public License,
-3 of the GNU General Public License.
+section 13, concerning interaction through a network will apply to the
+combination as such.
 
   14. Revised Versions of this License.
 
   The Free Software Foundation may publish revised and/or new versions of
-the GNU Affero General Public License from time to time.  Such new versions
+the GNU General Public License from time to time.  Such new versions will
-will be similar in spirit to the present version, but may differ in detail to
+be similar in spirit to the present version, but may differ in detail to
 address new problems or concerns.
 
   Each version is given a distinguishing version number.  If the
-Program specifies that a certain numbered version of the GNU Affero General
+Program specifies that a certain numbered version of the GNU General
 Public License "or any later version" applies to it, you have the
 option of following the terms and conditions either of that numbered
 version or of any later version published by the Free Software
 Foundation.  If the Program does not specify a version number of the
-GNU Affero General Public License, you may choose any version ever published
+GNU General Public License, you may choose any version ever published
 by the Free Software Foundation.
 
   If the Program specifies that a proxy can decide which future
-versions of the GNU Affero General Public License can be used, that proxy's
+versions of the GNU General Public License can be used, that proxy's
 public statement of acceptance of a version permanently authorizes you
 to choose that version for the Program.
 
@@ -633,29 +635,40 @@ the "copyright" line and a pointer to where the full notice is found.
     Copyright (C) <year>  <name of author>
 
     This program is free software: you can redistribute it and/or modify
-    it under the terms of the GNU Affero General Public License as published by
+    it under the terms of the GNU General Public License as published by
     the Free Software Foundation, either version 3 of the License, or
     (at your option) any later version.
 
     This program is distributed in the hope that it will be useful,
     but WITHOUT ANY WARRANTY; without even the implied warranty of
     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-    GNU Affero General Public License for more details.
+    GNU General Public License for more details.
 
-    You should have received a copy of the GNU Affero General Public License
+    You should have received a copy of the GNU General Public License
     along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 Also add information on how to contact you by electronic and paper mail.
 
-  If your software can interact with users remotely through a computer
+  If the program does terminal interaction, make it output a short
-network, you should also make sure that it provides a way for users to
+notice like this when it starts in an interactive mode:
-get its source.  For example, if your program is a web application, its
+
-interface could display a "Source" link that leads users to an archive
+    <program>  Copyright (C) <year>  <name of author>
-of the code.  There are many ways you could offer source, and different
+    This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
-solutions will be better for different programs; see section 13 for the
+    This is free software, and you are welcome to redistribute it
-specific requirements.
+    under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate
+parts of the General Public License.  Of course, your program's commands
+might be different; for a GUI interface, you would use an "about box".
 
   You should also get your employer (if you work as a programmer) or school,
 if any, to sign a "copyright disclaimer" for the program, if necessary.
-For more information on this, and how to apply and follow the GNU AGPL, see
+For more information on this, and how to apply and follow the GNU GPL, see
 <http://www.gnu.org/licenses/>.
+
+  The GNU General Public License does not permit incorporating your program
+into proprietary programs.  If your program is a subroutine library, you
+may consider it more useful to permit linking proprietary applications with
+the library.  If this is what you want to do, use the GNU Lesser General
+Public License instead of this License.  But first, please read
+<http://www.gnu.org/philosophy/why-not-lgpl.html>.

README.md

@@ -2,8 +2,8 @@
 
 http://www.shellcheck.net
 
-Copyright 2012-2014, Vidar 'koala_man' Holen
+Copyright 2012-2015, Vidar 'koala_man' Holen
-Licensed under the GNU Affero General Public License, v3
+Licensed under the GNU General Public License, v3
 
 The goals of ShellCheck are:
 
@@ -16,7 +16,7 @@ The goals of ShellCheck are:
   - To point out subtle caveats, corner cases and pitfalls, that may cause an
     advanced user's otherwise working script to fail under future circumstances.
 
-ShellCheck is written in Haskell, and requires at least 1 GB of RAM to compile.
+ShellCheck is written in Haskell, and requires 2 GB of memory to compile.
 
 ## Installing
 
@@ -25,9 +25,9 @@ On systems with Cabal:
     cabal update
     cabal install shellcheck
 
-On Arch Linux with community packages enabled:
+On Debian based distros:
 
-    pacman -S shellcheck
+    apt-get install shellcheck
 
 On OS X with homebrew:
 
@@ -57,6 +57,10 @@ On Mac OS X with MacPorts (http://www.macports.org/):
 
     port install hs-cabal-install
 
+On native Windows (https://www.haskell.org/platform/):
+
+    Download and install the latest version of the Haskell Platform.
+
 Let cabal update itself, in case your distro version is outdated:
 
     $ cabal update
@@ -66,17 +70,29 @@ With cabal installed, cd to the ShellCheck source directory and:
 
     $ cabal install
 
-This will install ShellCheck to your ~/.cabal/bin directory.
+This will install ShellCheck to your `~/.cabal/bin` directory.
 
-Add the directory to your PATH (for bash, add this to your ~/.bashrc file):
+Add the directory to your `PATH` (for bash, add this to your `~/.bashrc`):
 
-    export PATH=$HOME/.cabal/bin:$PATH
+    export PATH="$HOME/.cabal/bin:$PATH"
 
 Verify that your PATH is set up correctly:
 
     $ which shellcheck
     ~/.cabal/bin/shellcheck
 
+On native Windows, the `PATH` should already be set up, but the system
+may use a legacy codepage. In `cmd.exe`, `powershell.exe` and Powershell ISE,
+make sure to use a TrueType font, not a Raster font, and set the active
+codepage to UTF-8 (65001) with `chcp`:
+
+    > chcp 65001
+    Active code page: 65001
+
+In Powershell ISE, you may need to additionally update the output encoding:
+
+    > [Console]::OutputEncoding = [System.Text.Encoding]::UTF8
+
 ## Running tests
 
 To run the unit test suite:

Setup.hs

@@ -1,2 +1,43 @@
-import Distribution.Simple
+import Distribution.PackageDescription (
-main = defaultMain
+  HookedBuildInfo,
+  emptyHookedBuildInfo )
+import Distribution.Simple (
+  Args,
+  UserHooks ( preSDist ),
+  defaultMainWithHooks,
+  simpleUserHooks )
+import Distribution.Simple.Setup ( SDistFlags )
+
+-- | This requires the process package from,
+--
+--   https://hackage.haskell.org/package/process
+--
+import System.Process ( callCommand )
+
+
+-- | This will use almost the default implementation, except we switch
+--   out the default pre-sdist hook with our own, 'myPreSDist'.
+--
+main = defaultMainWithHooks myHooks
+  where
+    myHooks = simpleUserHooks { preSDist = myPreSDist }
+
+
+-- | This hook will be executed before e.g. @cabal sdist@. It runs
+--   pandoc to create the man page from shellcheck.1.md. If the pandoc
+--   command is not found, this will fail with an error message:
+--
+--     /bin/sh: pandoc: command not found
+--
+--   Since the man page is listed in the Extra-Source-Files section of
+--   our cabal file, a failure here should result in a failure to
+--   create the distribution tarball (that's a good thing).
+--
+myPreSDist :: Args -> SDistFlags -> IO HookedBuildInfo
+myPreSDist _ _ = do
+  putStrLn "Building the man page..."
+  putStrLn pandoc_cmd
+  callCommand pandoc_cmd
+  return emptyHookedBuildInfo
+  where
+    pandoc_cmd = "pandoc -s -t man shellcheck.1.md -o shellcheck.1"

ShellCheck.cabal

@@ -1,7 +1,7 @@
 Name:             ShellCheck
-Version:          0.3.4
+Version:          0.3.8
 Synopsis:         Shell script analysis tool
-License:          OtherLicense
+License:          GPL-3
 License-file:     LICENSE
 Category:         Static Analysis
 Author:           Vidar Holen
@@ -26,6 +26,8 @@ Extra-Source-Files:
     -- documentation
     README.md
     shellcheck.1.md
+    -- built with a cabal sdist hook
+    shellcheck.1
     -- tests
     test/shellcheck.hs
 
@@ -41,13 +43,15 @@ library
       json,
       mtl,
       parsec,
-      regex-compat,
+      regex-tdfa,
-      QuickCheck >= 2.2
+      QuickCheck >= 2.7.4
     exposed-modules:
       ShellCheck.Analytics
       ShellCheck.AST
       ShellCheck.Data
+      ShellCheck.Options
       ShellCheck.Parser
+      ShellCheck.Regex
       ShellCheck.Simple
     other-modules:
       Paths_ShellCheck
@@ -61,8 +65,9 @@ executable shellcheck
       json,
       mtl,
       parsec,
-      regex-compat,
+      regex-tdfa,
-      QuickCheck >= 2.2
+      transformers,
+      QuickCheck >= 2.7.4
     main-is: shellcheck.hs
 
 test-suite test-shellcheck
@@ -75,7 +80,8 @@ test-suite test-shellcheck
       json,
       mtl,
       parsec,
-      regex-compat,
+      regex-tdfa,
-      QuickCheck >= 2.2
+      transformers,
+      QuickCheck >= 2.7.4
     main-is: test/shellcheck.hs
 

ShellCheck/AST.hs

@@ -3,23 +3,23 @@
     http://www.vidarholen.net/contents/shellcheck
 
     ShellCheck is free software: you can redistribute it and/or modify
-    it under the terms of the GNU Affero General Public License as published by
+    it under the terms of the GNU General Public License as published by
     the Free Software Foundation, either version 3 of the License, or
     (at your option) any later version.
 
     ShellCheck is distributed in the hope that it will be useful,
     but WITHOUT ANY WARRANTY; without even the implied warranty of
     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-    GNU Affero General Public License for more details.
+    GNU General Public License for more details.
 
-    You should have received a copy of the GNU Affero General Public License
+    You should have received a copy of the GNU General Public License
     along with this program.  If not, see <http://www.gnu.org/licenses/>.
 -}
 module ShellCheck.AST where
 
 import Control.Monad
 import Control.Monad.Identity
-import qualified Text.Regex as Re
+import qualified ShellCheck.Regex as Re
 
 data Id = Id Int deriving (Show, Eq, Ord)
 
@@ -28,12 +28,12 @@ data Dashed = Dashed | Undashed deriving (Show, Eq)
 data AssignmentMode = Assign | Append deriving (Show, Eq)
 data FunctionKeyword = FunctionKeyword Bool deriving (Show, Eq)
 data FunctionParentheses = FunctionParentheses Bool deriving (Show, Eq)
-data ForInType = NormalForIn | ShortForIn deriving (Show, Eq)
 data CaseType = CaseBreak | CaseFallThrough | CaseContinue deriving (Show, Eq)
 
 data Token =
     TA_Binary Id String Token Token
     | TA_Expansion Id [Token]
+    | TA_Index Id Token
     | TA_Sequence Id [Token]
     | TA_Trinary Id Token Token Token
     | TA_Unary Id String Token
@@ -48,13 +48,12 @@ data Token =
     | T_Arithmetic Id Token
     | T_Array Id [Token]
     | T_IndexedElement Id Token Token
-    | T_ Id [Token]
     | T_Assignment Id AssignmentMode String (Maybe Token) Token
     | T_Backgrounded Id Token
     | T_Backticked Id [Token]
     | T_Bang Id
     | T_Banged Id Token
-    | T_BraceExpansion Id String
+    | T_BraceExpansion Id [Token]
     | T_BraceGroup Id [Token]
     | T_CLOBBER Id
     | T_Case Id
@@ -82,7 +81,7 @@ data Token =
     | T_Fi Id
     | T_For Id
     | T_ForArithmetic Id Token Token Token [Token]
-    | T_ForIn Id ForInType [String] [Token] [Token]
+    | T_ForIn Id String [Token] [Token]
     | T_Function Id FunctionKeyword FunctionParentheses String Token
     | T_GREATAND Id
     | T_Glob Id String
@@ -122,16 +121,20 @@ data Token =
     | T_WhileExpression Id [Token] [Token]
     | T_Annotation Id [Annotation] Token
     | T_Pipe Id String
+    | T_CoProc Id (Maybe String) Token
+    | T_CoProcBody Id Token
     deriving (Show)
 
 data Annotation = DisableComment Integer deriving (Show, Eq)
 data ConditionType = DoubleBracket | SingleBracket deriving (Show, Eq)
 
--- I apologize for nothing!
+-- This is an abomination.
-lolHax s = Re.subRegex (Re.mkRegex "(Id [0-9]+)") (show s) "(Id 0)"
+tokenEquals :: Token -> Token -> Bool
-instance Eq Token where
+tokenEquals a b = kludge a == kludge b
-    (==) a b = lolHax a == lolHax b
+    where kludge s = Re.subRegex (Re.mkRegex "\\(Id [0-9]+\\)") (show s) "(Id 0)"
 
+instance Eq Token where
+    (==) = tokenEquals
 
 analyze :: Monad m => (Token -> m ()) -> (Token -> m ()) -> (Token -> Token) -> Token -> m Token
 analyze f g i =
@@ -168,6 +171,7 @@ analyze f g i =
     delve (T_DoubleQuoted id list) = dl list $ T_DoubleQuoted id
     delve (T_DollarDoubleQuoted id list) = dl list $ T_DollarDoubleQuoted id
     delve (T_DollarExpansion id list) = dl list $ T_DollarExpansion id
+    delve (T_BraceExpansion id list) = dl list $ T_BraceExpansion id
     delve (T_Backticked id list) = dl list $ T_Backticked id
     delve (T_DollarArithmetic id c) = d1 c $ T_DollarArithmetic id
     delve (T_DollarBracket id c) = d1 c $ T_DollarBracket id
@@ -204,7 +208,7 @@ analyze f g i =
     delve (T_BraceGroup id l) = dl l $ T_BraceGroup id
     delve (T_WhileExpression id c l) = dll c l $ T_WhileExpression id
     delve (T_UntilExpression id c l) = dll c l $ T_UntilExpression id
-    delve (T_ForIn id t v w l) = dll w l $ T_ForIn id t v
+    delve (T_ForIn id v w l) = dll w l $ T_ForIn id v
     delve (T_SelectIn id v w l) = dll w l $ T_SelectIn id v
     delve (T_CaseExpression id word cases) = do
         newWord <- round word
@@ -245,7 +249,10 @@ analyze f g i =
         c <- round t3
         return $ TA_Trinary id a b c
     delve (TA_Expansion id t) = dl t $ TA_Expansion id
+    delve (TA_Index id t) = d1 t $ TA_Index id
     delve (T_Annotation id anns t) = d1 t $ T_Annotation id anns
+    delve (T_CoProc id var body) = d1 body $ T_CoProc id var
+    delve (T_CoProcBody id t) = d1 t $ T_CoProcBody id
     delve t = return t
 
 getId t = case t of
@@ -310,7 +317,7 @@ getId t = case t of
         T_BraceGroup id _  -> id
         T_WhileExpression id _ _  -> id
         T_UntilExpression id _ _  -> id
-        T_ForIn id _ _ _ _  -> id
+        T_ForIn id _ _ _  -> id
         T_SelectIn id _ _ _  -> id
         T_CaseExpression id _ _ -> id
         T_Function id _ _ _ _  -> id
@@ -330,6 +337,7 @@ getId t = case t of
         TA_Sequence id _  -> id
         TA_Trinary id _ _ _  -> id
         TA_Expansion id _  -> id
+        TA_Index id _  -> id
         T_ProcSub id _ _ -> id
         T_Glob id _ -> id
         T_ForArithmetic id _ _ _ _ -> id
@@ -338,6 +346,8 @@ getId t = case t of
         T_DollarBracket id _ -> id
         T_Annotation id _ _ -> id
         T_Pipe id _ -> id
+        T_CoProc id _ _ -> id
+        T_CoProcBody id _ -> id
 
 blank :: Monad m => Token -> m ()
 blank = const $ return ()

ShellCheck/Data.hs

@@ -27,22 +27,10 @@ internalVariables = [
     "LC_MESSAGES", "LC_NUMERIC", "LINES", "MAIL", "MAILCHECK", "MAILPATH",
     "OPTERR", "PATH", "POSIXLY_CORRECT", "PROMPT_COMMAND",
     "PROMPT_DIRTRIM", "PS1", "PS2", "PS3", "PS4", "SHELL", "TIMEFORMAT",
-    "TMOUT", "TMPDIR", "auto_resume", "histchars",
+    "TMOUT", "TMPDIR", "auto_resume", "histchars", "COPROC",
 
-    -- Zsh
+    -- Other
-    "ARGV0", "BAUD", "cdpath", "COLUMNS", "CORRECT_IGNORE",
+    "USER", "TZ", "TERM"
-    "DIRSTACKSIZE", "ENV", "FCEDIT", "fignore", "fpath", "histchars",
-    "HISTCHARS", "HISTFILE", "HISTSIZE", "HOME", "IFS", "KEYBOARD_HACK",
-    "KEYTIMEOUT", "LANG", "LC_ALL", "LC_COLLATE", "LC_CTYPE",
-    "LC_MESSAGES", "LC_NUMERIC", "LC_TIME", "LINES", "LISTMAX",
-    "LOGCHECK", "MAIL", "MAILCHECK", "mailpath", "manpath", "module_path",
-    "NULLCMD", "path", "POSTEDIT", "PROMPT", "PROMPT2", "PROMPT3",
-    "PROMPT4", "prompt", "PROMPT_EOL_MARK", "PS1", "PS2", "PS3", "PS4",
-    "psvar", "READNULLCMD", "REPORTTIME", "REPLY", "reply", "RPROMPT",
-    "RPS1", "RPROMPT2", "RPS2", "SAVEHIST", "SPROMPT", "STTY", "TERM",
-    "TERMINFO", "TIMEFMT", "TMOUT", "TMPPREFIX", "watch", "WATCHFMT",
-    "WORDCHARS", "ZBEEP", "ZDOTDIR", "ZLE_LINE_ABORTED",
-    "ZLE_REMOVE_SUFFIX_CHARS", "ZLE_SPACE_SUFFIX_CHARS"
   ]
 
 variablesWithoutSpaces = [

ShellCheck/Options.hs

@@ -0,0 +1,14 @@
+module ShellCheck.Options where
+
+data Shell = Ksh | Sh | Bash
+    deriving (Show, Eq)
+
+data AnalysisOptions = AnalysisOptions {
+    optionShellType :: Maybe Shell,
+    optionExcludes :: [Integer]
+}
+
+defaultAnalysisOptions = AnalysisOptions {
+    optionShellType = Nothing,
+    optionExcludes = []
+}

ShellCheck/Parser.hs

@@ -3,23 +3,24 @@
     http://www.vidarholen.net/contents/shellcheck
 
     ShellCheck is free software: you can redistribute it and/or modify
-    it under the terms of the GNU Affero General Public License as published by
+    it under the terms of the GNU General Public License as published by
     the Free Software Foundation, either version 3 of the License, or
     (at your option) any later version.
 
     ShellCheck is distributed in the hope that it will be useful,
     but WITHOUT ANY WARRANTY; without even the implied warranty of
     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-    GNU Affero General Public License for more details.
+    GNU General Public License for more details.
 
-    You should have received a copy of the GNU Affero General Public License
+    yOU should have received a copy of the GNU General Public License
     along with this program.  If not, see <http://www.gnu.org/licenses/>.
 -}
-{-# LANGUAGE NoMonomorphismRestriction, TemplateHaskell #-}
+{-# LANGUAGE NoMonomorphismRestriction, TemplateHaskell, FlexibleContexts #-}
-module ShellCheck.Parser (Note(..), Severity(..), parseShell, ParseResult(..), ParseNote(..), sortNotes, noteToParseNote, runTests) where
+module ShellCheck.Parser (Note(..), Severity(..), parseShell, ParseResult(..), ParseNote(..), sortNotes, noteToParseNote, runTests, readScript) where
 
 import ShellCheck.AST
 import ShellCheck.Data
+import ShellCheck.Options
 import Text.Parsec
 import Debug.Trace
 import Control.Monad
@@ -61,13 +62,13 @@ unicodeDoubleQuoteChars = "\x201C\x201D\x2033\x2036"
 
 prop_spacing = isOk spacing "  \\\n # Comment"
 spacing = do
-    x <- many (many1 linewhitespace <|> try (string "\\\n"))
+    x <- many (many1 linewhitespace <|> try (string "\\\n" >> return ""))
     optional readComment
     return $ concat x
 
 spacing1 = do
     spacing <- spacing
-    when (null spacing) $ fail "no spacing"
+    when (null spacing) $ fail "Expected whitespace"
     return spacing
 
 prop_allspacing = isOk allspacing "#foo"
@@ -84,7 +85,7 @@ allspacing = do
 
 allspacingOrFail = do
     s <- allspacing
-    when (null s) $ fail "Expected spaces"
+    when (null s) $ fail "Expected whitespace"
 
 unicodeDoubleQuote = do
     pos <- getPosition
@@ -183,9 +184,9 @@ popContext = do
         then do
             let (a:r) = v
             setCurrentContexts r
-            return [a]
+            return $ Just a
         else
-            return []
+            return Nothing
 
 pushContext c = do
     v <- getCurrentContexts
@@ -233,8 +234,14 @@ reluctantlyTill1 p end = do
 attempting rest branch =
     (try branch >> rest) <|> rest
 
-orFail parser stuff =
+orFail parser errorAction =
-    try (disregard parser) <|> (disregard stuff >> fail "nope")
+    try parser <|> (errorAction >>= fail)
+
+-- Construct a node with a parser, e.g. T_Literal `withParser` (readGenericLiteral ",")
+withParser node parser = do
+    id <- getNextId
+    contents <- parser
+    return $ node id contents
 
 wasIncluded p = option False (p >> return True)
 
@@ -252,7 +259,7 @@ withContext entry p = do
         popContext
         return v
      <|> do -- p failed without consuming input, abort context
-        popContext
+        v <- popContext
         fail ""
 
 called s p = do
@@ -270,12 +277,22 @@ readConditionContents single =
                                     parseProblemAt pos WarningC 1009 "Use 'if cmd; then ..' to check exit code, or 'if [[ $(cmd) == .. ]]' to check output.")
 
   where
+    spacingOrLf = condSpacing True
+    condSpacing required = do
+        pos <- getPosition
+        space <- allspacing
+        when (required && null space) $
+            parseProblemAt pos ErrorC 1035 "You are missing a required space here."
+        when (single && '\n' `elem` space) $
+            parseProblemAt pos ErrorC 1080 "When breaking lines in [ ], you need \\ before the linefeed."
+        return space
+
     typ = if single then SingleBracket else DoubleBracket
     readCondBinaryOp = try $ do
         optional guardArithmetic
         id <- getNextId
         op <- choice (map tryOp ["==", "!=", "<=", ">=", "=~", ">", "<", "=", "\\<=", "\\>=", "\\<", "\\>"]) <|> otherOp
-        hardCondSpacing
+        spacingOrLf
         return op
       where
         tryOp s = try $ do
@@ -285,7 +302,7 @@ readConditionContents single =
         otherOp = try $ do
             id <- getNextId
             s <- readOp
-            when (s == "-a" || s == "-o") $ fail "Wrong operator"
+            when (s == "-a" || s == "-o") $ fail "Unexpected operator"
             return $ TC_Binary id typ s
 
     guardArithmetic = do
@@ -298,17 +315,14 @@ readConditionContents single =
     readCondUnaryExp = do
       op <- readCondUnaryOp
       pos <- getPosition
-      (do
+      (readCondWord >>= return . op) `orFail` do
-          arg <- readCondWord
-          return $ op arg)
-        <|> (do
           parseProblemAt pos ErrorC 1019 "Expected this to be an argument to the unary condition."
-              fail "oops")
+          return "Expected an argument for the unary operator"
 
     readCondUnaryOp = try $ do
         id <- getNextId
         s <- readOp
-        hardCondSpacing
+        spacingOrLf
         return $ TC_Unary id typ s
 
     readOp = try $ do
@@ -337,19 +351,20 @@ readConditionContents single =
 
     readCondAndOp = do
         id <- getNextId
-        x <- try (string "&&" <|> string "-a")
+        x <- try (readAndOrOp "&&" False <|> readAndOrOp "-a" True)
-        softCondSpacing
-        skipLineFeeds
         return $ TC_And id typ x
 
     readCondOrOp = do
         optional guardArithmetic
         id <- getNextId
-        x <- try (string "||" <|> string "-o")
+        x <- try (readAndOrOp "||" False <|> readAndOrOp "-o" True)
-        softCondSpacing
-        skipLineFeeds
         return $ TC_Or id typ x
 
+    readAndOrOp op requiresSpacing = do
+        x <- string op
+        condSpacing requiresSpacing
+        return x
+
     readCondNoaryOrBinary = do
       id <- getNextId
       x <- readCondWord `attempting` (do
@@ -373,16 +388,21 @@ readConditionContents single =
           id <- getNextId
           pos <- getPosition
           lparen <- try $ string "(" <|> string "\\("
-          when (single && lparen == "(") $ parseProblemAt pos ErrorC 1028 "In [..] you have to escape (). Use [[..]] instead."
+          when (single && lparen == "(") $
-          when (not single && lparen == "\\(") $ parseProblemAt pos ErrorC 1029 "In [[..]] you shouldn't escape ()."
+              parseProblemAt pos ErrorC 1028 "In [..] you have to escape (). Use [[..]] instead."
-          if single then hardCondSpacing else disregard spacing
+          when (not single && lparen == "\\(") $
+              parseProblemAt pos ErrorC 1029 "In [[..]] you shouldn't escape ()."
+          condSpacing single
           x <- readCondContents
           cpos <- getPosition
           rparen <- string ")" <|> string "\\)"
-          if single then hardCondSpacing else disregard spacing
+          condSpacing single
-          when (single && rparen == ")") $ parseProblemAt cpos ErrorC 1030 "In [..] you have to escape (). Use [[..]] instead."
+          when (single && rparen == ")") $
-          when (not single && rparen == "\\)") $ parseProblemAt cpos ErrorC 1031 "In [[..]] you shouldn't escape ()."
+              parseProblemAt cpos ErrorC 1030 "In [..] you have to escape (). Use [[..]] instead."
-          when (isEscaped lparen `xor` isEscaped rparen) $ parseProblemAt pos ErrorC 1032 "Did you just escape one half of () but not the other?"
+          when (not single && rparen == "\\)") $
+              parseProblemAt cpos ErrorC 1031 "In [[..]] you shouldn't escape ()."
+          when (isEscaped lparen `xor` isEscaped rparen) $
+              parseProblemAt pos ErrorC 1032 "Did you just escape one half of () but not the other?"
           return $ TC_Group id typ x
       where
         isEscaped ('\\':_) = True
@@ -426,21 +446,15 @@ readConditionContents single =
             str <- string "|"
             return $ T_Literal id str
 
-    skipLineFeeds = do
-        pos <- getPosition
-        spacing <- allspacing
-        when (single  && '\n' `elem` spacing) $
-            parseProblemAt pos ErrorC 1080 "In [ ] you need \\ before line feeds."
-
     readCondTerm = do
         term <- readCondNot <|> readCondExpr
-        skipLineFeeds
+        condSpacing False
         return term
 
     readCondNot = do
         id <- getNextId
         char '!'
-        softCondSpacing
+        spacingOrLf
         expr <- readCondExpr
         return $ TC_Unary id typ "!" expr
 
@@ -452,7 +466,6 @@ readConditionContents single =
     readCondContents = readCondOr
 
 
-
 prop_a1 = isOk readArithmeticContents " n++ + ++c"
 prop_a2 = isOk readArithmeticContents "$N*4-(3,2)"
 prop_a3 = isOk readArithmeticContents "n|=2<<1"
@@ -462,14 +475,15 @@ prop_a6 = isOk readArithmeticContents " 1 | 2 ||3|4"
 prop_a7 = isOk readArithmeticContents "3*2**10"
 prop_a8 = isOk readArithmeticContents "3"
 prop_a9 = isOk readArithmeticContents "a^!-b"
-prop_aA = isOk readArithmeticContents "! $?"
+prop_a10= isOk readArithmeticContents "! $?"
-prop_aB = isOk readArithmeticContents "10#08 * 16#f"
+prop_a11= isOk readArithmeticContents "10#08 * 16#f"
-prop_aC = isOk readArithmeticContents "\"$((3+2))\" + '37'"
+prop_a12= isOk readArithmeticContents "\"$((3+2))\" + '37'"
-prop_aD = isOk readArithmeticContents "foo[9*y+x]++"
+prop_a13= isOk readArithmeticContents "foo[9*y+x]++"
-prop_aE = isOk readArithmeticContents "1+`echo 2`"
+prop_a14= isOk readArithmeticContents "1+`echo 2`"
-prop_aF = isOk readArithmeticContents "foo[`echo foo | sed s/foo/4/g` * 3] + 4"
+prop_a15= isOk readArithmeticContents "foo[`echo foo | sed s/foo/4/g` * 3] + 4"
-prop_a10= isOk readArithmeticContents "$foo$bar"
+prop_a16= isOk readArithmeticContents "$foo$bar"
-prop_a11= isOk readArithmeticContents "i<(0+(1+1))"
+prop_a17= isOk readArithmeticContents "i<(0+(1+1))"
+prop_a18= isOk readArithmeticContents "a?b:c"
 readArithmeticContents =
     readSequence
   where
@@ -489,10 +503,10 @@ readArithmeticContents =
 
     readArrayIndex = do
         id <- getNextId
-        start <- literal "["
+        char '['
         middle <- readArithmeticContents
-        end <- literal "]"
+        char ']'
-        return $ T_NormalWord id [start, middle, end]
+        return $ TA_Index id middle
 
     literal s = do
         id <- getNextId
@@ -511,7 +525,7 @@ readArithmeticContents =
             readNormalDollar,
             readBraced,
             readBackTicked,
-            readNormalLiteral "+-*/=%^,]"
+            readNormalLiteral "+-*/=%^,]?:"
             ]
         spacing
         return $ TA_Expansion id pieces
@@ -596,7 +610,7 @@ readArithmeticContents =
             id <- getNextId
             op <- try $ string "++" <|> string "--"
             spacing
-            return $ TA_Unary id ("|" ++ op) x
+            return $ TA_Unary id ('|':op) x
          <|>
             return x
 
@@ -613,7 +627,9 @@ prop_readCondition6 = isOk readCondition "[[ $c =~ ^[yY]$ ]]"
 prop_readCondition7 = isOk readCondition "[[ ${line} =~ ^[[:space:]]*# ]]"
 prop_readCondition8 = isOk readCondition "[[ $l =~ ogg|flac ]]"
 prop_readCondition9 = isOk readCondition "[ foo -a -f bar ]"
-prop_readCondition10= isOk readCondition "[[ a == b \n || c == d ]]"
+prop_readCondition10= isOk readCondition "[[\na == b\n||\nc == d ]]"
+prop_readCondition10a= isOk readCondition "[[\na == b  ||\nc == d ]]"
+prop_readCondition10b= isOk readCondition "[[ a == b\n||\nc == d ]]"
 prop_readCondition11= isOk readCondition "[[ a == b ||\n c == d ]]"
 prop_readCondition12= isWarning readCondition "[ a == b \n -o c == d ]"
 prop_readCondition13= isOk readCondition "[[ foo =~ ^fo{1,3}$ ]]"
@@ -622,9 +638,17 @@ readCondition = called "test expression" $ do
     id <- getNextId
     open <- try (string "[[") <|> string "["
     let single = open == "["
-    condSpacingMsg False $ if single
+
-          then "You need spaces after the opening [ and before the closing ]."
+    pos <- getPosition
-          else "You need spaces after the opening [[ and before the closing ]]."
+    space <- allspacing
+    when (null space) $
+        parseProblemAt pos ErrorC 1035 $ "You need a space after the " ++
+            if single
+                then "[ and before the ]."
+                else "[[ and before the ]]."
+    when (single && '\n' `elem` space) $
+        parseProblemAt pos ErrorC 1080 "You need \\ before line feeds to break lines in [ ]."
+
     condition <- readConditionContents single
 
     cpos <- getPosition
@@ -635,14 +659,6 @@ readCondition = called "test expression" $ do
     many readCmdWord -- Read and throw away remainders to get then/do warnings. Fixme?
     return $ T_Condition id (if single then SingleBracket else DoubleBracket) condition
 
-
-hardCondSpacing = condSpacingMsg False "You need a space here."
-softCondSpacing = condSpacingMsg True "You need a space here."
-condSpacingMsg soft msg = do
-  pos <- getPosition
-  space <- spacing
-  when (null space) $ (if soft then parseNoteAt else parseProblemAt) pos ErrorC 1035 msg
-
 readAnnotationPrefix = do
     char '#'
     many linewhitespace
@@ -755,15 +771,14 @@ readDollarBracedLiteral = do
 
 prop_readProcSub1 = isOk readProcSub "<(echo test | wc -l)"
 prop_readProcSub2 = isOk readProcSub "<(  if true; then true; fi )"
-prop_readProcSub3 = isOk readProcSub "=(ls)"
+prop_readProcSub3 = isOk readProcSub "<( # nothing here \n)"
 readProcSub = called "process substitution" $ do
     id <- getNextId
     dir <- try $ do
-                    x <- oneOf "<>="
+                    x <- oneOf "<>"
                     char '('
                     return [x]
-    allspacing
+    list <- readCompoundListOrEmpty
-    list <- readCompoundList
     allspacing
     char ')'
     return $ T_ProcSub id dir list
@@ -811,6 +826,8 @@ prop_readBackTicked3 = isWarning readBackTicked "´grep \"\\\"\"´"
 prop_readBackTicked4 = isOk readBackTicked "`echo foo\necho bar`"
 prop_readBackTicked5 = isOk readSimpleCommand "echo `foo`bar"
 prop_readBackTicked6 = isWarning readSimpleCommand "echo `foo\necho `bar"
+prop_readBackTicked7 = isOk readSimpleCommand "`#inline comment`"
+prop_readBackTicked8 = isOk readSimpleCommand "echo `#comment` \\\nbar baz"
 readBackTicked = called "backtick expansion" $ do
     id <- getNextId
     startPos <- getPosition
@@ -826,19 +843,23 @@ readBackTicked = called "backtick expansion" $ do
             suggestForgotClosingQuote startPos endPos "backtick expansion"
 
     -- Result positions may be off due to escapes
-    result <- subParse subStart readCompoundList (unEscape subString)
+    result <- subParse subStart subParser (unEscape subString)
     return $ T_Backticked id result
   where
     unEscape [] = []
     unEscape ('\\':x:rest) | x `elem` "$`\\" = x : unEscape rest
     unEscape ('\\':'\n':rest) = unEscape rest
     unEscape (c:rest) = c : unEscape rest
+    subParser = do
+        cmds <- readCompoundListOrEmpty
+        verifyEof
+        return cmds
     backtick =
       disregard (char '`') <|> do
          pos <- getPosition
          char '´'
          parseProblemAt pos ErrorC 1077
-            "For command expansion, the tick should slant left (` vs ´)."
+            "For command expansion, the tick should slant left (` vs ´). Use $(..) instead."
 
 subParse pos parser input = do
     lastPosition <- getPosition
@@ -939,7 +960,7 @@ readNormalEscaped = called "escaped char" $ do
     pos <- getPosition
     backslash
     do
-        next <- quotable <|> oneOf "?*@!+[]{}.,"
+        next <- quotable <|> oneOf "?*@!+[]{}.,~#"
         return $ if next == '\n' then "" else [next]
       <|>
         do
@@ -1034,16 +1055,29 @@ readGenericEscaped = do
 
 prop_readBraced = isOk readBraced "{1..4}"
 prop_readBraced2 = isOk readBraced "{foo,bar,\"baz lol\"}"
-readBraced = try $ do
+prop_readBraced3 = isOk readBraced "{1,\\},2}"
-    let strip (T_Literal _ s) = return ("\"" ++ s ++ "\"")
+prop_readBraced4 = isOk readBraced "{1,{2,3}}"
-    id <- getNextId
+prop_readBraced5 = isOk readBraced "{JP{,E}G,jp{,e}g}"
+prop_readBraced6 = isOk readBraced "{foo,bar,$((${var}))}"
+readBraced = try braceExpansion
+  where
+    braceExpansion =
+        T_BraceExpansion `withParser` do
             char '{'
-    str <- many1 ((readDoubleQuotedLiteral >>= strip) <|> readGenericLiteral1 (oneOf "}\"" <|> whitespace))
+            elements <- bracedElement `sepBy1` char ','
             char '}'
-    let result = concat str
+            return elements
-    unless (',' `elem` result || ".." `isInfixOf` result) $
+    bracedElement =
-        fail "Not a brace expression"
+        T_NormalWord `withParser` do
-    return $ T_BraceExpansion id result
+            many $ choice [
+                braceExpansion,
+                readDollarExpression,
+                readSingleQuoted,
+                readDoubleQuoted,
+                braceLiteral
+                ]
+    braceLiteral =
+        T_Literal `withParser` readGenericLiteral1 (oneOf "{}\"$'," <|> whitespace)
 
 readNormalDollar = readDollarExpression <|> readDollarDoubleQuote <|> readDollarSingleQuote <|> readDollarLonely
 readDoubleQuotedDollar = readDollarExpression <|> readDollarLonely
@@ -1067,7 +1101,6 @@ readDollarDoubleQuote = do
     doubleQuote <?> "end of translated double quoted string"
     return $ T_DollarDoubleQuoted id x
 
-
 prop_readDollarArithmetic = isOk readDollarArithmetic "$(( 3 * 4 +5))"
 prop_readDollarArithmetic2 = isOk readDollarArithmetic "$(((3*4)+(1*2+(3-1))))"
 readDollarArithmetic = called "$((..)) expression" $ do
@@ -1084,6 +1117,7 @@ readDollarBracket = called "$[..] expression" $ do
     string "]"
     return (T_DollarBracket id c)
 
+prop_readArithmeticExpression = isOk readArithmeticExpression "((a?b:c))"
 readArithmeticExpression = called "((..)) command" $ do
     id <- getNextId
     try (string "((")
@@ -1102,32 +1136,44 @@ readDollarBraced = called "parameter expansion" $ do
     char '}'
     return $ T_DollarBraced id word
 
-prop_readDollarExpansion = isOk readDollarExpansion "$(echo foo; ls\n)"
+prop_readDollarExpansion1= isOk readDollarExpansion "$(echo foo; ls\n)"
+prop_readDollarExpansion2= isOk readDollarExpansion "$(  )"
+prop_readDollarExpansion3= isOk readDollarExpansion "$( command \n#comment \n)"
 readDollarExpansion = called "command expansion" $ do
     id <- getNextId
     try (string "$(")
-    cmds <- readCompoundList
+    cmds <- readCompoundListOrEmpty
     char ')' <?> "end of $(..) expression"
     return $ T_DollarExpansion id cmds
 
 prop_readDollarVariable = isOk readDollarVariable "$@"
+prop_readDollarVariable2 = isOk (readDollarVariable >> anyChar) "$?!"
+prop_readDollarVariable3 = isWarning (readDollarVariable >> anyChar) "$10"
+prop_readDollarVariable4 = isWarning (readDollarVariable >> string "[@]") "$arr[@]"
+
 readDollarVariable = do
     id <- getNextId
+    pos <- getPosition
+
     let singleCharred p = do
         n <- p
         value <- wrap [n]
-        return (T_DollarBraced id value) `attempting` do
+        return (T_DollarBraced id value)
-            pos <- getPosition
+
-            num <- lookAhead $ many1 p
+    let positional = do
-            parseNoteAt pos ErrorC 1037 $ "$" ++ (n:num) ++ " is equivalent to ${" ++ [n] ++ "}"++ num ++"."
+        value <- singleCharred digit
+        return value `attempting` do
+            lookAhead digit
+            parseNoteAt pos ErrorC 1037 "Braces are required for positionals over 9, e.g. ${10}."
 
-    let positional = singleCharred digit
     let special = singleCharred specialVariable
 
     let regular = do
         name <- readVariableName
         value <- wrap name
-        return $ T_DollarBraced id value
+        return (T_DollarBraced id value) `attempting` do
+            lookAhead $ void (string "[@]") <|> void (string "[*]") <|> void readArrayIndex
+            parseNoteAt pos ErrorC 1087 "Braces are required when expanding arrays, as in ${array[idx]}."
 
     try $ char '$' >> (positional <|> special <|> regular)
 
@@ -1335,7 +1381,6 @@ prop_readSimpleCommand3 = isOk readSimpleCommand "export foo=(bar baz)"
 prop_readSimpleCommand4 = isOk readSimpleCommand "typeset -a foo=(lol)"
 prop_readSimpleCommand5 = isOk readSimpleCommand "time if true; then echo foo; fi"
 prop_readSimpleCommand6 = isOk readSimpleCommand "time -p ( ls -l; )"
-prop_readSimpleCommand7 = isOk readSimpleCommand "cat =(ls)"
 readSimpleCommand = called "simple command" $ do
     id1 <- getNextId
     id2 <- getNextId
@@ -1390,6 +1435,12 @@ readAndOr = do
                 then andOr
                 else T_Annotation aid annotations andOr
 
+readTermOrNone = do
+    allspacing
+    readTerm <|> do
+        eof
+        return []
+
 readTerm = do
     allspacing
     m <- readAndOr
@@ -1433,7 +1484,11 @@ readPipe = do
     spacing
     return $ T_Pipe id ('|':qualifier)
 
-readCommand = readCompoundCommand <|> readSimpleCommand
+readCommand = choice [
+    readCompoundCommand,
+    readCoProc,
+    readSimpleCommand
+    ]
 
 readCmdName = do
     f <- readNormalWord
@@ -1460,6 +1515,7 @@ readIfClause = called "if expression" $ do
     g_Fi `orFail` do
         parseProblemAt pos ErrorC 1046 "Couldn't find 'fi' for this 'if'."
         parseProblem ErrorC 1047 "Expected 'fi' matching previously mentioned 'if'."
+        return "Expected 'fi'."
 
     return $ T_IfExpression id ((condition, action):elifs) elses
 
@@ -1475,12 +1531,13 @@ readIfPart = do
     allspacing
     condition <- readTerm
 
-    optional (do
+    ifNextToken (g_Fi <|> g_Elif) $
-        try . lookAhead $ g_Fi
+        parseProblemAt pos ErrorC 1049 "Did you forget the 'then' for this 'if'?"
-        parseProblemAt pos ErrorC 1049 "Did you forget the 'then' for this 'if'?")
 
     called "then clause" $ do
-        g_Then `orFail` parseProblem ErrorC 1050 "Expected 'then'."
+        g_Then `orFail` do
+            parseProblem ErrorC 1050 "Expected 'then'."
+            return "Expected 'then'."
 
         acceptButWarn g_Semi ErrorC 1051 "No semicolons directly after 'then'."
         allspacing
@@ -1496,6 +1553,10 @@ readElifPart = called "elif clause" $ do
         parseProblemAt pos ErrorC 1075 "Use 'elif' instead of 'else if'."
     allspacing
     condition <- readTerm
+
+    ifNextToken (g_Fi <|> g_Elif) $
+        parseProblemAt pos ErrorC 1049 "Did you forget the 'then' for this 'elif'?"
+
     g_Then
     acceptButWarn g_Semi ErrorC 1052 "No semicolons directly after 'then'."
     allspacing
@@ -1514,6 +1575,11 @@ readElsePart = called "else clause" $ do
     verifyNotEmptyIf "else"
     readTerm
 
+ifNextToken parser action =
+    optional $ do
+        try . lookAhead $ parser
+        action
+
 prop_readSubshell = isOk readSubshell "( cd /foo; tar cf stuff.tar * )"
 readSubshell = called "explicit subshell" $ do
     id <- getNextId
@@ -1537,7 +1603,7 @@ readBraceGroup = called "brace group" $ do
     list <- readTerm
     char '}' <|> do
         parseProblem ErrorC 1056 "Expected a '}'. If you have one, try a ; or \\n in front of it."
-        fail "Unable to parse"
+        fail "Missing '}'"
     return $ T_BraceGroup id list
 
 prop_readWhileClause = isOk readWhileClause "while [[ -e foo ]]; do sleep 1; done"
@@ -1562,7 +1628,9 @@ readDoGroup loopPos = do
                 try . lookAhead $ g_Done
                 parseProblemAt loopPos ErrorC 1057 "Did you forget the 'do' for this loop?")
 
-    g_Do `orFail` parseProblem ErrorC 1058 "Expected 'do'."
+    g_Do `orFail` do
+        parseProblem ErrorC 1058 "Expected 'do'."
+        return "Expected 'do'."
 
     acceptButWarn g_Semi ErrorC 1059 "No semicolons directly after 'do'."
     allspacing
@@ -1575,6 +1643,7 @@ readDoGroup loopPos = do
     g_Done `orFail` do
             parseProblemAt pos ErrorC 1061 "Couldn't find 'done' for this 'do'."
             parseProblem ErrorC 1062 "Expected 'done' matching previously mentioned 'do'."
+            return "Expected 'done'."
     return commands
 
 
@@ -1587,12 +1656,12 @@ prop_readForClause7 = isOk readForClause "for ((;;)) do echo $i\ndone"
 prop_readForClause8 = isOk readForClause "for ((;;)) ; do echo $i\ndone"
 prop_readForClause9 = isOk readForClause "for i do true; done"
 prop_readForClause10= isOk readForClause "for ((;;)) { true; }"
-prop_readForClause11= isOk readForClause "for a b in *; do echo $a $b; done"
+prop_readForClause12= isWarning readForClause "for $a in *; do echo \"$a\"; done"
 readForClause = called "for loop" $ do
     pos <- getPosition
     (T_For id) <- g_For
     spacing
-    readRegular id pos <|> readArithmetic id pos
+    readArithmetic id pos <|> readRegular id pos
   where
     readArithmetic id pos = called "arithmetic for condition" $ do
         try $ string "(("
@@ -1613,25 +1682,12 @@ readForClause = called "for loop" $ do
         return list
 
     readRegular id pos = do
-        names <- readNames
+        acceptButWarn (char '$') ErrorC 1086
-        readShort names <|> readLong names
+            "Don't use $ on the iterator name in for loops."
-      where
+        name <- readVariableName `thenSkip` spacing
-        readLong names = do
         values <- readInClause <|> (optional readSequentialSep >> return [])
         group <- readDoGroup pos
-            return $ T_ForIn id NormalForIn names values group
+        return $ T_ForIn id name values group
-        readShort names = do
-            char '('
-            allspacing
-            words <- many (readNormalWord `thenSkip` allspacing)
-            char ')'
-            allspacing
-            command <- readAndOr
-            return $ T_ForIn id ShortForIn names words [command]
-
-    readNames =
-        reluctantlyTill1 (readVariableName `thenSkip` spacing) $
-            disregard g_Do <|> disregard readInClause <|> disregard readSequentialSep
 
 prop_readSelectClause1 = isOk readSelectClause "select foo in *; do echo $foo; done"
 prop_readSelectClause2 = isOk readSelectClause "select foo; do echo $foo; done"
@@ -1687,7 +1743,10 @@ readCaseItem = called "case item" $ do
     optional g_Lparen
     spacing
     pattern <- readPattern
-    g_Rparen
+    void g_Rparen <|> do
+        parseProblem ErrorC 1085
+            "Did you forget to move the ;; after extending this case item?"
+        fail "Expected ) to open a new case item"
     readLineBreak
     list <- (lookAhead readCaseSeparator >> return []) <|> readCompoundList
     separator <- readCaseSeparator `attempting` do
@@ -1749,7 +1808,31 @@ readFunctionDefinition = called "function" $ do
                 g_Rparen
             return ()
 
-        readFunctionName = many functionChars
+        readFunctionName = many1 functionChars
+
+prop_readCoProc1 = isOk readCoProc "coproc foo { echo bar; }"
+prop_readCoProc2 = isOk readCoProc "coproc { echo bar; }"
+prop_readCoProc3 = isOk readCoProc "coproc echo bar"
+readCoProc = called "coproc" $ do
+    id <- getNextId
+    try $ do
+        string "coproc"
+        whitespace
+    choice [ try $ readCompoundCoProc id, readSimpleCoProc id ]
+  where
+    readCompoundCoProc id = do
+        var <- optionMaybe $
+            readVariableName `thenSkip` whitespace
+        body <- readBody readCompoundCommand
+        return $ T_CoProc id var body
+    readSimpleCoProc id = do
+        body <- readBody readSimpleCommand
+        return $ T_CoProc id Nothing body
+    readBody parser = do
+        id <- getNextId
+        body <- parser
+        return $ T_CoProcBody id body
+
 
 readPattern = (readNormalWord `thenSkip` spacing) `sepBy1` (char '|' `thenSkip` spacing)
 
@@ -1768,6 +1851,9 @@ readCompoundCommand = do
 
 
 readCompoundList = readTerm
+readCompoundListOrEmpty = do
+    allspacing
+    readTerm <|> return []
 
 readCmdPrefix = many1 (readIoRedirect <|> readAssignmentWord)
 readCmdSuffix = many1 (readIoRedirect <|> readCmdWord)
@@ -1793,8 +1879,7 @@ readLetSuffix = many1 (readIoRedirect <|> try readLetExpression <|> readCmdWord)
 -- Get whatever a parser would parse as a string
 readStringForParser parser = do
     pos <- lookAhead (parser >> getPosition)
-    s <- readUntil pos
+    readUntil pos
-    return s
   where
     readUntil endPos = anyChar `reluctantlyTill` (getPosition >>= guard . (== endPos))
 
@@ -1807,32 +1892,32 @@ prop_readAssignmentWord6 = isWarning readAssignmentWord "b += (1 2 3)"
 prop_readAssignmentWord7 = isOk readAssignmentWord "a[3$n'']=42"
 prop_readAssignmentWord8 = isOk readAssignmentWord "a[4''$(cat foo)]=42"
 prop_readAssignmentWord9 = isOk readAssignmentWord "IFS= "
+prop_readAssignmentWord9a= isOk readAssignmentWord "foo="
 prop_readAssignmentWord10= isWarning readAssignmentWord "foo$n=42"
 prop_readAssignmentWord11= isOk readAssignmentWord "foo=([a]=b [c] [d]= [e f )"
+prop_readAssignmentWord12= isOk readAssignmentWord "a[b <<= 3 + c]='thing'"
 readAssignmentWord = try $ do
     id <- getNextId
     pos <- getPosition
     optional (char '$' >> parseNote ErrorC 1066 "Don't use $ on the left side of assignments.")
     variable <- readVariableName
-    notFollowedBy2 $ do -- Special case for zsh =(..) syntax
-        spacing1
-        string "=("
     optional (readNormalDollar >> parseNoteAt pos ErrorC
                                 1067 "For indirection, use (associative) arrays or 'read \"var$n\" <<< \"value\"'")
     index <- optionMaybe readArrayIndex
-    space <- spacing
+    hasLeftSpace <- liftM (not . null) spacing
     pos <- getPosition
     op <- readAssignmentOp
-    space2 <- spacing
+    hasRightSpace <- liftM (not . null) spacing
-    if space == "" && space2 /= ""
+    isEndOfCommand <- liftM isJust $ optionMaybe (try . lookAhead $ (disregard (oneOf "\r\n;&|)") <|> eof))
+    if not hasLeftSpace && (hasRightSpace || isEndOfCommand)
       then do
-        when (variable /= "IFS") $
+        when (variable /= "IFS" && hasRightSpace) $
             parseNoteAt pos WarningC 1007
                 "Remove space after = if trying to assign a value (for empty string, use var='' ... )."
         value <- readEmptyLiteral
         return $ T_Assignment id op variable index value
       else do
-        when (space /= "" || space2 /= "") $
+        when (hasLeftSpace || hasRightSpace) $
             parseNoteAt pos ErrorC 1068 "Don't put spaces around the = in assignments."
         value <- readArray <|> readNormalWord
         spacing
@@ -1844,14 +1929,10 @@ readAssignmentWord = try $ do
         id <- getNextId
         return $ T_Literal id ""
 
--- This is only approximate. Fixme?
--- * Bash allows foo[' ' "" $(true) 2 ``]=var
--- * foo[bar] dereferences bar
 readArrayIndex = do
     char '['
     optional space
-    x <- readNormalishWord "]"
+    x <- readArithmeticContents
-    optional space
     char ']'
     return x
 
@@ -1903,8 +1984,8 @@ tryParseWordToken keyword t = try $ do
             "Scripts are case sensitive. Use '" ++ keyword ++ "', not '" ++ str ++ "'."
     return $ t id
 
-anycaseString =
+anycaseString str =
-    mapM anycaseChar
+    mapM anycaseChar str <?> str
   where
     anycaseChar c = char (toLower c) <|> char (toUpper c)
 
@@ -1942,10 +2023,12 @@ g_Rparen = tryToken ")" T_Rparen
 g_Bang = do
     id <- getNextId
     char '!'
-    softCondSpacing
+    void spacing1 <|> do
+        pos <- getPosition
+        parseProblemAt pos ErrorC 1035
+            "You are missing a required space after the !."
     return $ T_Bang id
 
-
 g_Semi = do
     notFollowedBy2 g_DSEMI
     tryToken ";" T_Semi
@@ -1958,17 +2041,41 @@ readKeyword = choice [ g_Then, g_Else, g_Elif, g_Fi, g_Do, g_Done, g_Esac, g_Rbr
 ifParse p t f =
     (lookAhead (try p) >> t) <|> f
 
+prop_readShebang1 = isOk readShebang "#!/bin/sh\n"
+prop_readShebang2 = isWarning readShebang "!# /bin/sh\n"
 readShebang = do
-    try $ string "#!"
+    try readCorrect <|> try readSwapped
     str <- many $ noneOf "\r\n"
     optional carriageReturn
     optional linefeed
     return str
+  where
+    readCorrect = void $ string "#!"
+    readSwapped = do
+        pos <- getPosition
+        string "!#"
+        parseProblemAt pos ErrorC 1084
+            "Use #!, not !#, for the shebang."
+
+verifyEof = eof <|> choice [
+        ifParsable g_Lparen $
+            parseProblem ErrorC 1088 "Parsing stopped here. Invalid use of parentheses?",
+
+        ifParsable readKeyword $
+            parseProblem ErrorC 1089 "Parsing stopped here. Is this keyword correctly matched up?",
+
+        parseProblem ErrorC 1070 "Parsing stopped here. Mismatched keywords or invalid parentheses?"
+    ]
+  where
+    ifParsable p action = do
+        try (lookAhead p)
+        action
 
 prop_readScript1 = isOk readScript "#!/bin/bash\necho hello world\n"
 prop_readScript2 = isWarning readScript "#!/bin/bash\r\necho hello world\n"
 prop_readScript3 = isWarning readScript "#!/bin/bash\necho hello\xA0world"
 prop_readScript4 = isWarning readScript "#!/usr/bin/perl\nfoo=("
+prop_readScript5 = isOk readScript "#!/bin/bash\n#This is an empty script\n\n"
 readScript = do
     id <- getNextId
     pos <- getPosition
@@ -1979,19 +2086,13 @@ readScript = do
     sb <- option "" readShebang
     verifyShell pos (getShell sb)
     if isValidShell (getShell sb) /= Just False
-      then
+      then do
-        do {
+            commands <- readCompoundListOrEmpty
-            allspacing;
+            verifyEof
-            commands <- readTerm;
+            return $ T_Script id sb commands
-            eof <|> parseProblem ErrorC 1070 "Parsing stopped here because of parsing errors.";
-            return $ T_Script id sb commands;
-        } <|> do {
-            parseProblem WarningC 1014 "Couldn't read any commands.";
-            return $ T_Script id sb [T_EOF id];
-        }
         else do
             many anyChar
-        return $ T_Script id sb [T_EOF id];
+            return $ T_Script id sb []
 
   where
     basename s = reverse . takeWhile (/= '/') . reverse $ s
@@ -2007,8 +2108,8 @@ readScript = do
     verifyShell pos s =
         case isValidShell s of
             Just True -> return ()
-            Just False -> parseProblemAt pos ErrorC 1071 "ShellCheck only supports Bourne based shell scripts, sorry!"
+            Just False -> parseProblemAt pos ErrorC 1071 "ShellCheck only supports sh/bash/ksh scripts. Sorry!"
-            Nothing -> parseProblemAt pos InfoC 1008 "This shebang was unrecognized. Note that ShellCheck only handles Bourne based shells."
+            Nothing -> parseProblemAt pos InfoC 1008 "This shebang was unrecognized. Note that ShellCheck only handles sh/bash/ksh."
 
     isValidShell s =
         let good = s == "" || any (`isPrefixOf` s) goodShells
@@ -2022,17 +2123,20 @@ readScript = do
 
     goodShells = [
         "sh",
+        "ash",
+        "dash",
         "bash",
-        "ksh",
+        "ksh"
-        "zsh"
         ]
     badShells = [
         "awk",
         "csh",
+        "expect",
         "perl",
         "python",
         "ruby",
-        "tcsh"
+        "tcsh",
+        "zsh"
         ]
 
     readUtf8Bom = called "Byte Order Mark" $ string "\xFEFF"
@@ -2060,28 +2164,29 @@ sortNotes = sortBy compareNotes
 data ParseResult = ParseResult { parseResult :: Maybe (Token, Map.Map Id SourcePos), parseNotes :: [ParseNote] } deriving (Show)
 
 makeErrorFor parsecError =
-    ParseNote (errorPos parsecError) ErrorC 1072 $ getStringFromParsec $ errorMessages parsecError
+    ParseNote (errorPos parsecError) ErrorC 1072 $
+        getStringFromParsec $ errorMessages parsecError
 
 getStringFromParsec errors =
-        case map snd $ sortWith fst $ map f errors of
+        case map f errors of
-            r -> unwords (take 1 $ nub r)  ++ " Fix any mentioned problems and try again."
+            r -> unwords (take 1 $ catMaybes $ reverse r)  ++
-    where f err =
+                " Fix any mentioned problems and try again."
+    where
+        f err =
             case err of
-                UnExpect s    -> (1, unexpected s)
+                UnExpect s    ->  return $ unexpected s
-                SysUnExpect s -> (2, unexpected s)
+                SysUnExpect s ->  return $ unexpected s
-                Expect s      -> (3, "Expected " ++ s ++ ".")
+                Expect s      ->  return $ "Expected " ++ s ++ "."
-                Message s     -> (4, s ++ ".")
+                Message s     ->  if null s then Nothing else return $ s ++ "."
-          wut "" = "eof"
+        unexpected s = "Unexpected " ++ (if null s then "eof" else s) ++ "."
-          wut x = x
-          unexpected s = "Unexpected " ++ wut s ++ "."
 
-parseShell filename contents =
+parseShell options filename contents =
     case rp (parseWithNotes readScript) filename contents of
         (Right (script, map, notes), (parsenotes, _)) ->
-            ParseResult (Just (script, map)) (nub $ sortNotes $ notes ++ parsenotes)
+            ParseResult (Just (script, map)) (nub . sortNotes . excludeNotes $ notes ++ parsenotes)
         (Left err, (p, context)) ->
             ParseResult Nothing
-                (nub $ sortNotes $ p ++ notesForContext context ++ [makeErrorFor err])
+                (nub . sortNotes . excludeNotes $ p ++ notesForContext context ++ [makeErrorFor err])
   where
     isName (ContextName _ _) = True
     isName _ = False
@@ -2090,6 +2195,7 @@ parseShell filename contents =
         "Couldn't parse this " ++ str ++ "."
     second (ContextName pos str) = ParseNote pos InfoC 1009 $
         "The mentioned parser error was in this " ++ str ++ "."
+    excludeNotes = filter (\c -> codeForParseNote c `notElem` optionExcludes options)
 
 lt x = trace (show x) x
 ltt t = trace (show t)

ShellCheck/Regex.hs

@@ -0,0 +1,71 @@
+{-
+    This file is part of ShellCheck.
+    http://www.vidarholen.net/contents/shellcheck
+
+    ShellCheck is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    ShellCheck is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+-}
+{-# LANGUAGE FlexibleContexts #-}
+
+-- Basically Text.Regex based on regex-tdfa instead of the buggy regex-posix.
+module ShellCheck.Regex where
+
+import Data.List
+import Data.Maybe
+import Control.Monad
+import Text.Regex.TDFA
+
+-- Precompile the regex
+mkRegex :: String -> Regex
+mkRegex str =
+    let make :: RegexMaker Regex CompOption ExecOption String => String -> Regex
+        make = makeRegex
+    in
+        make str
+
+-- Does the regex match?
+matches :: String -> Regex -> Bool
+matches = flip match
+
+-- Get all subgroups of the first match
+matchRegex :: Regex -> String -> Maybe [String]
+matchRegex re str = do
+    (_, _, _, groups) <- matchM re str :: Maybe (String,String,String,[String])
+    return groups
+
+-- Get all full matches
+matchAllStrings :: Regex -> String -> [String]
+matchAllStrings re = unfoldr f
+  where
+    f :: String -> Maybe (String, String)
+    f str = do
+        (_, match, rest, _) <- matchM re str :: Maybe (String, String, String, [String])
+        return (match, rest)
+
+-- Get all subgroups from all matches
+matchAllSubgroups :: Regex -> String -> [[String]]
+matchAllSubgroups re = unfoldr f
+  where
+    f :: String -> Maybe ([String], String)
+    f str = do
+        (_, _, rest, groups) <- matchM re str :: Maybe (String, String, String, [String])
+        return (groups, rest)
+
+-- Replace regex in input with string
+subRegex :: Regex -> String -> String -> String
+subRegex re input replacement = f input
+  where
+    f str = fromMaybe str $ do
+        (before, match, after) <- matchM re str :: Maybe (String, String, String)
+        when (null match) $ error ("Internal error: substituted empty in " ++ str)
+        return $ before ++ replacement ++ f after

ShellCheck/Simple.hs

@@ -3,31 +3,32 @@
     http://www.vidarholen.net/contents/shellcheck
 
     ShellCheck is free software: you can redistribute it and/or modify
-    it under the terms of the GNU Affero General Public License as published by
+    it under the terms of the GNU General Public License as published by
     the Free Software Foundation, either version 3 of the License, or
     (at your option) any later version.
 
     ShellCheck is distributed in the hope that it will be useful,
     but WITHOUT ANY WARRANTY; without even the implied warranty of
     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-    GNU Affero General Public License for more details.
+    GNU General Public License for more details.
 
-    You should have received a copy of the GNU Affero General Public License
+    You should have received a copy of the GNU General Public License
     along with this program.  If not, see <http://www.gnu.org/licenses/>.
 -}
 {-# LANGUAGE TemplateHaskell #-}
 module ShellCheck.Simple (shellCheck, ShellCheckComment, scLine, scColumn, scSeverity, scCode, scMessage, runTests) where
 
-import ShellCheck.Parser hiding (runTests)
-import ShellCheck.Analytics hiding (runTests)
-import Data.Maybe
-import Text.Parsec.Pos
 import Data.List
+import Data.Maybe
+import ShellCheck.Analytics hiding (runTests)
+import ShellCheck.Options
+import ShellCheck.Parser hiding (runTests)
 import Test.QuickCheck.All (quickCheckAll)
+import Text.Parsec.Pos
 
-shellCheck :: String -> [AnalysisOption] -> [ShellCheckComment]
+shellCheck :: AnalysisOptions -> String -> [ShellCheckComment]
-shellCheck script options =
+shellCheck options script =
-    let (ParseResult result notes) = parseShell "-" script in
+    let (ParseResult result notes) = parseShell options "-" script in
         let allNotes = notes ++ concat (maybeToList $ do
             (tree, posMap) <- result
             let list = runAnalytics options tree
@@ -51,21 +52,28 @@ severityToString s =
 formatNote (ParseNote pos severity code text) =
     ShellCheckComment (sourceLine pos) (sourceColumn pos) (severityToString severity) (fromIntegral code) text
 
+testCheck = shellCheck defaultAnalysisOptions { optionExcludes = [2148] } -- Ignore #! warnings
 prop_findsParseIssue =
-    let comments = shellCheck "echo \"$12\"" [] in
+    let comments = testCheck "echo \"$12\"" in
         length comments == 1 && scCode (head comments) == 1037
 prop_commentDisablesParseIssue1 =
-    null $ shellCheck "#shellcheck disable=SC1037\necho \"$12\"" []
+    null $ testCheck "#shellcheck disable=SC1037\necho \"$12\""
 prop_commentDisablesParseIssue2 =
-    null $ shellCheck "#shellcheck disable=SC1037\n#lol\necho \"$12\"" []
+    null $ testCheck "#shellcheck disable=SC1037\n#lol\necho \"$12\""
 
 prop_findsAnalysisIssue =
-    let comments = shellCheck "echo $1" [] in
+    let comments = testCheck "echo $1" in
         length comments == 1 && scCode (head comments) == 2086
 prop_commentDisablesAnalysisIssue1 =
-    null $ shellCheck "#shellcheck disable=SC2086\necho $1" []
+    null $ testCheck "#shellcheck disable=SC2086\necho $1"
 prop_commentDisablesAnalysisIssue2 =
-    null $ shellCheck "#shellcheck disable=SC2086\n#lol\necho $1" []
+    null $ testCheck "#shellcheck disable=SC2086\n#lol\necho $1"
+
+prop_optionDisablesIssue1 =
+    null $ shellCheck (defaultAnalysisOptions { optionExcludes = [2086, 2148] }) "echo $1"
+
+prop_optionDisablesIssue2 =
+    null $ shellCheck (defaultAnalysisOptions { optionExcludes = [2148, 1037] }) "echo \"$10\""
 
 return []
 runTests = $quickCheckAll

shellcheck.1.md

@@ -16,25 +16,43 @@ errors and pitfalls where the shell just gives a cryptic error message or
 strange behavior, but it also reports on a few more advanced issues where
 corner cases can cause delayed failures.
 
+ShellCheck gives shell specific advice. Consider the line:
+
+    (( area = 3.14*r*r ))
+
++ For scripts starting with `#!/bin/sh` (or when using `-s sh`), ShellCheck
+will warn that `(( .. ))` is not POSIX compliant (similar to checkbashisms).
+
++ For scripts starting with `#!/bin/bash` (or using `-s bash`), ShellCheck
+will warn that decimals are not supported.
+
++ For scripts starting with `#!/bin/ksh` (or using `-s ksh`), ShellCheck will
+not warn at all, as `ksh` supports decimals in arithmetic contexts.
+
+
 # OPTIONS
 
-**-f** *FORMAT*, **--format=***FORMAT*
-
-:   Specify the output format of shellcheck, which prints its results in the
-    standard output. Subsequent **-f** options are ignored, see **FORMATS**
-    below for more information.
-
 **-e**\ *CODE1*[,*CODE2*...],\ **--exclude=***CODE1*[,*CODE2*...]
 
 :   Explicitly exclude the specified codes from the report. Subsequent **-e**
     options are cumulative, but all the codes can be specified at once,
     comma-separated as a single argument.
 
+**-f** *FORMAT*, **--format=***FORMAT*
+
+:   Specify the output format of shellcheck, which prints its results in the
+    standard output. Subsequent **-f** options are ignored, see **FORMATS**
+    below for more information.
+
 **-s**\ *shell*,\ **--shell=***shell*
 
-:   Specify Bourne shell dialect. Valid values are *sh*, *bash*, *ksh* and
+:   Specify Bourne shell dialect. Valid values are *sh*, *bash* and *ksh*.
-    *zsh*. The default is to use the file's shebang, or *bash* if the target
+    The default is to use the file's shebang, or *bash* if the target shell
-    shell can't be determined.
+    can't be determined.
+
+**-V**\ *version*,\ **--version**
+
+:   Print version and exit.
 
 # FORMATS
 
@@ -79,11 +97,12 @@ corner cases can cause delayed failures.
 
         [
           {
-            "line": line,
+            "file": "filename",
-            "column": column,
+            "line": lineNumber,
-            "level": level,
+            "column": columnNumber,
-            "code": ####,
+            "level": "severitylevel",
-            "message": message
+            "code": errorCode,
+            "message": "warning message"
           },
           ...
         ]
@@ -100,6 +119,14 @@ For example, to suppress SC2035 about using `./*.jpg`:
     # shellcheck disable=SC2035
     echo "Files: " *.jpg
 
+Here a shell brace group is used to suppress on multiple lines:
+
+    # shellcheck disable=SC2016
+    {
+      echo 'Modifying $PATH'
+      echo 'PATH=foo:$PATH' >> ~/.bashrc
+    }
+
 Valid keys are:
 
 **disable**

shellcheck.hs

@@ -3,57 +3,73 @@
     http://www.vidarholen.net/contents/shellcheck
 
     ShellCheck is free software: you can redistribute it and/or modify
-    it under the terms of the GNU Affero General Public License as published by
+    it under the terms of the GNU General Public License as published by
     the Free Software Foundation, either version 3 of the License, or
     (at your option) any later version.
 
     ShellCheck is distributed in the hope that it will be useful,
     but WITHOUT ANY WARRANTY; without even the implied warranty of
     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-    GNU Affero General Public License for more details.
+    GNU General Public License for more details.
 
-    You should have received a copy of the GNU Affero General Public License
+    You should have received a copy of the GNU General Public License
     along with this program.  If not, see <http://www.gnu.org/licenses/>.
 -}
 import Control.Exception
 import Control.Monad
+import Control.Monad.Trans
+import Control.Monad.Trans.Error
+import Control.Monad.Trans.List
 import Data.Char
+import Data.List
 import Data.Maybe
+import Data.Monoid
 import GHC.Exts
 import GHC.IO.Device
 import Prelude hiding (catch)
 import ShellCheck.Data
+import ShellCheck.Options
 import ShellCheck.Simple
 import ShellCheck.Analytics
 import System.Console.GetOpt
 import System.Directory
 import System.Environment
 import System.Exit
+import System.Info
 import System.IO
 import Text.JSON
 import qualified Data.Map as Map
 
 data Flag = Flag String String
+data Status = NoProblems | SomeProblems | BadInput | SupportFailure | SyntaxFailure | RuntimeException deriving (Ord, Eq)
+
+data JsonComment = JsonComment FilePath ShellCheckComment
+
+instance Error Status where
+    noMsg = RuntimeException
+
+instance Monoid Status where
+    mempty = NoProblems
+    mappend = max
 
 header = "Usage: shellcheck [OPTIONS...] FILES..."
 options = [
-    Option ['f'] ["format"]
+    Option "e" ["exclude"]
-        (ReqArg (Flag "format") "FORMAT") "output format",
-    Option ['e'] ["exclude"]
         (ReqArg (Flag "exclude") "CODE1,CODE2..") "exclude types of warnings",
-    Option ['s'] ["shell"]
+    Option "f" ["format"]
-        (ReqArg (Flag "shell") "SHELLNAME") "Specify dialect (bash,sh,ksh,zsh)",
+        (ReqArg (Flag "format") "FORMAT") "output format",
-    Option ['V'] ["version"]
+    Option "s" ["shell"]
+        (ReqArg (Flag "shell") "SHELLNAME") "Specify dialect (bash,sh,ksh)",
+    Option "V" ["version"]
         (NoArg $ Flag "version" "true") "Print version information"
     ]
 
 printErr = hPutStrLn stderr
 
-syntaxFailure = ExitFailure 3
-supportFailure = ExitFailure 4
 
-instance JSON ShellCheckComment where
+instance JSON (JsonComment) where
-  showJSON c = makeObj [
+  showJSON (JsonComment filename c) = makeObj [
+      ("file", showJSON $ filename),
       ("line", showJSON $ scLine c),
       ("column", showJSON $ scColumn c),
       ("level", showJSON $ scSeverity c),
@@ -62,16 +78,15 @@ instance JSON ShellCheckComment where
       ]
   readJSON = undefined
 
+parseArguments :: [String] -> ErrorT Status IO ([Flag], [FilePath])
 parseArguments argv =
     case getOpt Permute options argv of
-        (opts, files, []) -> do
+        (opts, files, []) -> return (opts, files)
-            verifyOptions opts files
-            return $ Just (opts, files)
-
         (_, _, errors) -> do
-            printErr $ concat errors ++ "\n" ++ usageInfo header options
+            liftIO . printErr $ concat errors ++ "\n" ++ usageInfo header options
-            exitWith syntaxFailure
+            throwError SyntaxFailure
 
+formats :: Map.Map String (AnalysisOptions -> [FilePath] -> IO Status)
 formats = Map.fromList [
     ("json", forJson),
     ("gcc", forGcc),
@@ -79,9 +94,21 @@ formats = Map.fromList [
     ("tty", forTty)
     ]
 
+toStatus = liftM (either id (const NoProblems)) . runErrorT
+
+catchExceptions :: IO Status -> IO Status
+catchExceptions action = action -- action `catch` handler
+  where
+    handler err = do
+        printErr $ show (err :: SomeException)
+        return RuntimeException
+
+checkComments comments = if null comments then NoProblems else SomeProblems
+
+forTty :: AnalysisOptions -> [FilePath] -> IO Status
 forTty options files = do
     output <- mapM doFile files
-    return $ and output
+    return $ mconcat output
   where
     clear = ansi 0
     ansi n = "\x1B[" ++ show n ++ "m"
@@ -97,7 +124,7 @@ forTty options files = do
     colorComment level comment =
         ansi (colorForLevel level) ++ comment ++ clear
 
-    doFile path = do
+    doFile path = catchExceptions $ do
         contents <- readContents path
         doInput path contents
 
@@ -119,34 +146,39 @@ forTty options files = do
             mapM_ (\c -> putStrLn (colorFunc (scSeverity c) $ cuteIndent c)) x
             putStrLn ""
           ) groups
-        return $ null comments
+        return . checkComments $ comments
 
     cuteIndent comment =
         replicate (scColumn comment - 1) ' ' ++
             "^-- " ++ code (scCode comment) ++ ": " ++ scMessage comment
 
-    code code = "SC" ++ (show code)
+    code code = "SC" ++ show code
 
     getColorFunc = do
         term <- hIsTerminalDevice stdout
-        return $ if term then colorComment else const id
+        let windows = "mingw" `isPrefixOf` os
+        return $ if term && not windows then colorComment else const id
 
--- This totally ignores the filenames. Fixme?
+forJson :: AnalysisOptions -> [FilePath] -> IO Status
-forJson options files = do
+forJson options files = catchExceptions $ do
-    comments <- liftM concat $ mapM (commentsFor options) files
+    comments <- runListT $ do
+        file <- ListT $ return files
+        comment <- ListT $ commentsFor options file
+        return $ JsonComment file comment
     putStrLn $ encodeStrict comments
-    return . null $ comments
+    return $ checkComments comments
 
 -- Mimic GCC "file:line:col: (error|warning|note): message" format
+forGcc :: AnalysisOptions -> [FilePath] -> IO Status
 forGcc options files = do
     files <- mapM process files
-    return $ and files
+    return $ mconcat files
   where
-    process file = do
+    process file = catchExceptions $ do
         contents <- readContents file
         let comments = makeNonVirtual (getComments options contents) contents
         mapM_ (putStrLn . format file) comments
-        return $ null comments
+        return $ checkComments comments
 
     format filename c = concat [
             filename, ":",
@@ -162,20 +194,18 @@ forGcc options files = do
       ]
 
 -- Checkstyle compatible output. A bit of a hack to avoid XML dependencies
+forCheckstyle :: AnalysisOptions -> [FilePath] -> IO Status
 forCheckstyle options files = do
     putStrLn "<?xml version='1.0' encoding='UTF-8'?>"
     putStrLn "<checkstyle version='4.3'>"
-    statuses <- mapM (\x -> process x `catch` report) files
+    statuses <- mapM process files
     putStrLn "</checkstyle>"
-    return $ and statuses
+    return $ mconcat statuses
   where
-    process file = do
+    process file = catchExceptions $ do
         comments <- commentsFor options file
         putStrLn (formatFile file comments)
-        return $ null comments
+        return $ checkComments comments
-    report error = do
-        printErr $ show (error :: SomeException)
-        return False
 
     severity "error" = "error"
     severity "warning" = "warning"
@@ -197,31 +227,31 @@ forCheckstyle options files = do
         attr "column" $ show . scColumn $ c,
         attr "severity" $ severity . scSeverity $ c,
         attr "message" $ scMessage c,
-        attr "source" $ "ShellCheck.SC" ++ (show $ scCode c),
+        attr "source" $ "ShellCheck.SC" ++ show (scCode c),
         "/>\n"
         ]
 
-commentsFor options file =
+commentsFor options file = liftM (getComments options) $ readContents file
-    liftM (getComments options) $ readContents file
 
-getComments options contents =
+getComments = shellCheck
-    excludeCodes (getExclusions options) $ shellCheck contents analysisOptions
-  where
-    analysisOptions = catMaybes [ shellOption ]
-    shellOption = do
-        option <- getOption options "shell"
-        sh <- shellForExecutable option
-        return $ ForceShell sh
 
-
+readContents :: FilePath -> IO String
-readContents file = if file == "-" then getContents else readFile file
+readContents file =
+    if file == "-"
+    then getContents
+    else readFile file
 
 -- Realign comments from a tabstop of 8 to 1
 makeNonVirtual comments contents =
     map fix comments
   where
     ls = lines contents
-    fix c = c { scColumn = real (ls !! (scLine c - 1)) 0 0 (scColumn c) }
+    fix c = c {
+        scColumn =
+            if scLine c > 0 && scLine c <= length ls
+            then real (ls !! (scLine c - 1)) 0 0 (scColumn c)
+            else scColumn c
+    }
     real _ r v target | target <= v = r
     real [] r v _ = r -- should never happen
     real ('\t':rest) r v target =
@@ -240,7 +270,7 @@ split char str =
   where
     split' (a:rest) element =
         if a == char
-        then (reverse element) : split' rest []
+        then reverse element : split' rest []
         else split' rest (a:element)
     split' [] element = [reverse element]
 
@@ -257,45 +287,71 @@ excludeCodes codes =
 
 main = do
     args <- getArgs
-    parsedArgs <- parseArguments args
+    status <- toStatus $ do
-    code <- do
+        (flags, files) <- parseArguments args
-        status <- process parsedArgs
+        process flags files
-        return $ if status then ExitSuccess else ExitFailure 1
+    exitWith $ statusToCode status
-     `catch` return
-     `catch` \err -> do
-        printErr $ show (err :: SomeException)
-        return $ ExitFailure 2
-    exitWith code
 
-process Nothing = return False
+statusToCode status =
-process (Just (options, files)) =
+    case status of
-  let format = fromMaybe "tty" $ getOption options "format" in
+        NoProblems -> ExitSuccess
+        SomeProblems -> ExitFailure 1
+        BadInput -> ExitFailure 5
+        SyntaxFailure -> ExitFailure 3
+        SupportFailure -> ExitFailure 4
+        RuntimeException -> ExitFailure 2
+
+process :: [Flag] -> [FilePath] -> ErrorT Status IO ()
+process flags files = do
+    options <- foldM (flip parseOption) defaultAnalysisOptions flags
+    verifyFiles files
+    let format = fromMaybe "tty" $ getOption flags "format"
     case Map.lookup format formats of
         Nothing -> do
+            liftIO $ do
                 printErr $ "Unknown format " ++ format
-            printErr $ "Supported formats:"
+                printErr "Supported formats:"
                 mapM_ (printErr . write) $ Map.keys formats
-            exitWith supportFailure
+            throwError SupportFailure
           where write s = "  " ++ s
-        Just f -> do
+        Just f -> ErrorT $ liftM Left $ f options files
-            f options files
 
-verifyOptions opts files = do
+parseOption flag options =
-    when (isJust $ getOption opts "version") printVersionAndExit
+    case flag of
+        Flag "shell" str ->
+                fromMaybe (die $ "Unknown shell: " ++ str) $ do
+                    shell <- shellForExecutable str
+                    return $ return options { optionShellType = Just shell }
 
-    let shell = getOption opts "shell" in
+        Flag "exclude" str -> do
-        when (isJust shell && isNothing (shell >>= shellForExecutable)) $ do
+            new <- mapM parseNum $ split ',' str
-            printErr $ "Unknown shell: " ++ (fromJust shell)
+            let old = optionExcludes options
-            exitWith supportFailure
+            return options { optionExcludes = new ++ old }
 
+        Flag "version" _ -> do
+            liftIO printVersion
+            throwError NoProblems
+
+        _ -> return options
+  where
+    die s = do
+        liftIO $ printErr s
+        throwError SupportFailure
+    parseNum ('S':'C':str) = parseNum str
+    parseNum num = do
+        unless (all isDigit num) $ do
+            liftIO . printErr $ "Bad exclusion: " ++ num
+            throwError SyntaxFailure
+        return (Prelude.read num :: Integer)
+
+verifyFiles files =
     when (null files) $ do
-        printErr "No files specified.\n"
+        liftIO $ printErr "No files specified.\n"
-        printErr $ usageInfo header options
+        liftIO $ printErr $ usageInfo header options
-        exitWith syntaxFailure
+        throwError SyntaxFailure
 
-printVersionAndExit = do
+printVersion = do
-    putStrLn $ "ShellCheck - shell script analysis tool"
+    putStrLn   "ShellCheck - shell script analysis tool"
     putStrLn $ "version: " ++ shellcheckVersion
-    putStrLn $ "license: GNU Affero General Public License, version 3"
+    putStrLn   "license: GNU General Public License, version 3"
-    putStrLn $ "website: http://www.shellcheck.net"
+    putStrLn   "website: http://www.shellcheck.net"
-    exitWith ExitSuccess