11 Commits

Author SHA1 Message Date
Vidar Holen
c9b8ad3439 Drop attoparsec/text dependencies 2023-10-08 18:16:09 -07:00
Vidar Holen
e59fbfebda Re-add other Portage functionality 2023-10-08 16:09:58 -07:00
Vidar Holen
ce3414eeea Move from Parameters to SystemInterface for Portage variables 2023-08-27 17:53:14 -07:00
Vidar Holen
feebbbb096 Merge branch 'kangie' into ebuild 2023-08-27 15:20:00 -07:00
Vidar Holen
87ef5ae18a Merge branch 'portage' of https://github.com/Kangie/shellcheck into kangie 2023-08-27 15:18:32 -07:00
Vidar Holen
0138a6fafc Example plumbing for Portage variables 2023-08-13 17:49:36 -07:00
hololeap
dfa920c5d2 Switch to attoparsec for gentoo scan
Signed-off-by: hololeap <hololeap@users.noreply.github.com>
2023-08-05 17:38:01 -06:00
hololeap
fc9b63fb5e Remove PortageAutoInternalVariables and python
The Gentoo eclass list is now populated using pure Haskell. The old
python generators and generated module are no longer needed.

Signed-off-by: hololeap <hololeap@users.noreply.github.com>
2023-08-05 15:31:15 -06:00
hololeap
272ef819b9 Scan for Gentoo eclass variables
Creates a Map of eclass names to eclass variables by scanning the
system for repositories and their respective eclasses. Runs `portageq`
to determine repository names and locations. Emits a warning if an
IOException is caught when attempting to run `portageq`.

This Map is passed via CheckSpec to AnalysisSpec and finally to
Parameters, where it is read by `checkUnusedAssignments` in order to
determine which variables can be safely ignored by this check.

Signed-off-by: hololeap <hololeap@users.noreply.github.com>
2023-08-05 15:31:15 -06:00
hololeap
08ae7ef836 New IO interface to scan for Gentoo eclass vars
Uses the `portageq` command to scan for repositories, which in turn are
scanned for eclasses, which are then scanned for eclass variables.

The variables are scanned using a heuristic which looks for

    "# @ECLASS_VARIABLE: "

at the start of each line, which means only properly documented
variables will be found.

Signed-off-by: hololeap <hololeap@users.noreply.github.com>
2023-08-04 17:19:05 -06:00
Matt Jolly
e3d8483e49 Rebase of chromiumos fork
https://chromium.googlesource.com/chromiumos/third_party/shellcheck/
2023-08-04 15:56:48 -06:00
34 changed files with 755 additions and 655 deletions

View File

@@ -47,7 +47,7 @@ jobs:
needs: package_source
strategy:
matrix:
build: [linux.x86_64, linux.aarch64, linux.armv6hf, darwin.x86_64, darwin.aarch64, windows.x86_64]
build: [linux.x86_64, linux.aarch64, linux.armv6hf, darwin.x86_64, windows.x86_64]
runs-on: ubuntu-latest
steps:
- name: Checkout repository
@@ -103,11 +103,6 @@ jobs:
runs-on: ubuntu-latest
environment: Deploy
steps:
- name: Install Dependencies
run: |
sudo apt-get update
sudo apt-get install hub
- name: Checkout repository
uses: actions/checkout@v3

1
.gitignore vendored
View File

@@ -20,4 +20,3 @@ cabal.config
/parts/
/prime/
*.snap
/dist-newstyle/

14
.snapsquid.conf Normal file
View File

@@ -0,0 +1,14 @@
# In 2015, cabal-install had a http bug triggered when proxies didn't keep
# the connection open. This version made it into Ubuntu Xenial as used by
# Snapcraft. In June 2018, Snapcraft's proxy started triggering this bug.
#
# https://bugs.launchpad.net/launchpad-buildd/+bug/1797809
#
# Workaround: add more proxy
visible_hostname localhost
http_port 8888
cache_peer 10.10.10.1 parent 8222 0 no-query default
cache_peer_domain localhost !.internal
http_access allow all

View File

@@ -1,23 +1,13 @@
## v0.10.0 - 2024-03-07
## Git
### Added
- Precompiled binaries for macOS ARM64 (darwin.aarch64)
- Added support for busybox sh
- Added flag --rcfile to specify an rc file by name.
- Added `extended-analysis=true` directive to enable/disable dataflow analysis
(with a corresponding --extended-analysis flag).
- SC2324: Warn when x+=1 appends instead of increments
- SC2325: Warn about multiple `!`s in dash/sh.
- SC2326: Warn about `foo | ! bar` in bash/dash/sh.
- SC3012: Warn about lexicographic-compare bashism in test like in [ ]
- SC3013: Warn bashism `test _ -op/-nt/-ef _` like in [ ]
- SC3014: Warn bashism `test _ == _` like in [ ]
- SC3015: Warn bashism `test _ =~ _` like in [ ]
- SC3016: Warn bashism `test -v _` like in [ ]
- SC3017: Warn bashism `test -a _` like in [ ]
### Fixed
- source statements with here docs now work correctly
- "(Array.!): undefined array element" error should no longer occur
### Changed
## v0.9.0 - 2022-12-12

View File

@@ -77,7 +77,7 @@ You can see ShellCheck suggestions directly in a variety of editors.
* Sublime, through [SublimeLinter](https://github.com/SublimeLinter/SublimeLinter-shellcheck).
* Pulsar Edit (former Atom), through [linter-shellcheck-pulsar](https://github.com/pulsar-cooperative/linter-shellcheck-pulsar).
* Atom, through [Linter](https://github.com/AtomLinter/linter-shellcheck).
* VSCode, through [vscode-shellcheck](https://github.com/timonwong/vscode-shellcheck).
@@ -194,12 +194,6 @@ On Windows (via [chocolatey](https://chocolatey.org/packages/shellcheck)):
C:\> choco install shellcheck
```
Or Windows (via [winget](https://github.com/microsoft/winget-pkgs)):
```cmd
C:\> winget install --id koalaman.shellcheck
```
Or Windows (via [scoop](http://scoop.sh)):
```cmd
@@ -309,6 +303,10 @@ Verify that `cabal` is installed and update its dependency list with
$ cabal install
Or if you intend to run the tests:
$ cabal install --enable-tests
This will compile ShellCheck and install it to your `~/.cabal/bin` directory.
Add this directory to your `PATH` (for bash, add this to your `~/.bashrc`):
@@ -554,3 +552,4 @@ Happy ShellChecking!
* The wiki has [long form descriptions](https://github.com/koalaman/shellcheck/wiki/Checks) for each warning, e.g. [SC2221](https://github.com/koalaman/shellcheck/wiki/SC2221).
* ShellCheck does not attempt to enforce any kind of formatting or indenting style, so also check out [shfmt](https://github.com/mvdan/sh)!

View File

@@ -1,5 +1,5 @@
Name: ShellCheck
Version: 0.10.0
Version: 0.9.0
Synopsis: Shell script analysis tool
License: GPL-3
License-file: LICENSE
@@ -46,14 +46,14 @@ library
semigroups
build-depends:
-- The lower bounds are based on GHC 7.10.3
-- The upper bounds are based on GHC 9.8.1
aeson >= 1.4.0 && < 2.3,
-- The upper bounds are based on GHC 9.6.1
aeson >= 1.4.0 && < 2.2,
array >= 0.5.1 && < 0.6,
base >= 4.8.0.0 && < 5,
bytestring >= 0.10.6 && < 0.13,
containers >= 0.5.6 && < 0.8,
deepseq >= 1.4.1 && < 1.6,
Diff >= 0.4.0 && < 0.6,
bytestring >= 0.10.6 && < 0.12,
containers >= 0.5.6 && < 0.7,
deepseq >= 1.4.1 && < 1.5,
Diff >= 0.4.0 && < 0.5,
fgl (>= 5.7.0 && < 5.8.1.0) || (>= 5.8.1.1 && < 5.9),
filepath >= 1.4.0 && < 1.5,
mtl >= 2.2.2 && < 2.4,
@@ -93,6 +93,7 @@ library
ShellCheck.Formatter.Quiet
ShellCheck.Interface
ShellCheck.Parser
ShellCheck.PortageVariables
ShellCheck.Prelude
ShellCheck.Regex
other-modules:

View File

@@ -1,40 +0,0 @@
FROM ghcr.io/shepherdjerred/macos-cross-compiler:latest
ENV TARGET aarch64-apple-darwin22
ENV TARGETNAME darwin.aarch64
# Build dependencies
USER root
ENV DEBIAN_FRONTEND noninteractive
ENV LC_ALL C.utf8
# Install basic deps
RUN apt-get update && apt-get install -y automake autoconf build-essential curl xz-utils qemu-user-static
# Install a more suitable host compiler
WORKDIR /host-ghc
RUN curl -L "https://downloads.haskell.org/~cabal/cabal-install-3.9.0.0/cabal-install-3.9-x86_64-linux-alpine.tar.xz" | tar xJv -C /usr/local/bin
RUN curl -L 'https://downloads.haskell.org/~ghc/8.10.7/ghc-8.10.7-x86_64-deb10-linux.tar.xz' | tar xJ --strip-components=1
RUN ./configure && make install
# Build GHC. We have to use an old version because cross-compilation across OS has since broken.
WORKDIR /ghc
RUN curl -L "https://downloads.haskell.org/~ghc/8.10.7/ghc-8.10.7-src.tar.xz" | tar xJ --strip-components=1
RUN apt-get install -y llvm-12
RUN ./boot && ./configure --host x86_64-linux-gnu --build x86_64-linux-gnu --target "$TARGET"
RUN cp mk/flavours/quick-cross.mk mk/build.mk && make -j "$(nproc)"
RUN make install
# Due to an apparent cabal bug, we specify our options directly to cabal
# It won't reuse caches if ghc-options are specified in ~/.cabal/config
ENV CABALOPTS "--ghc-options;-optc-Os -optc-fPIC;--with-ghc=$TARGET-ghc;--with-hc-pkg=$TARGET-ghc-pkg;--constraint=hashable==1.3.5.0"
# Prebuild the dependencies
RUN cabal update
RUN IFS=';' && cabal install --dependencies-only $CABALOPTS ShellCheck
# Copy the build script
COPY build /usr/bin
WORKDIR /scratch
ENTRYPOINT ["/usr/bin/build"]

View File

@@ -1,17 +0,0 @@
#!/bin/sh
set -xe
{
tar xzv --strip-components=1
chmod +x striptests && ./striptests
mkdir "$TARGETNAME"
cabal update
( IFS=';'; cabal build $CABALOPTS )
find . -name shellcheck -type f -exec mv {} "$TARGETNAME/" \;
ls -l "$TARGETNAME"
# Stripping invalidates the code signature and the build image does
# not appear to have anything similar to the 'codesign' tool.
# "$TARGET-strip" "$TARGETNAME/shellcheck"
ls -l "$TARGETNAME"
file "$TARGETNAME/shellcheck" | grep "Mach-O 64-bit arm64 executable"
} >&2
tar czv "$TARGETNAME"

View File

@@ -1 +0,0 @@
koalaman/scbuilder-darwin-aarch64

View File

@@ -56,13 +56,6 @@ not warn at all, as `ksh` supports decimals in arithmetic contexts.
options are cumulative, but all the codes can be specified at once,
comma-separated as a single argument.
**--extended-analysis=true/false**
: Enable/disable Dataflow Analysis to identify more issues (default true). If
ShellCheck uses too much CPU/RAM when checking scripts with several
thousand lines of code, extended analysis can be disabled with this flag
or a directive. This flag overrides directives and rc files.
**-f** *FORMAT*, **--format=***FORMAT*
: Specify the output format of shellcheck, which prints its results in the
@@ -78,11 +71,6 @@ not warn at all, as `ksh` supports decimals in arithmetic contexts.
: Don't try to look for .shellcheckrc configuration files.
--rcfile\ RCFILE
: Prefer the specified configuration file over searching for one
in the default locations.
**-o**\ *NAME1*[,*NAME2*...],\ **--enable=***NAME1*[,*NAME2*...]
: Enable optional checks. The special name *all* enables all of them.
@@ -97,11 +85,11 @@ not warn at all, as `ksh` supports decimals in arithmetic contexts.
**-s**\ *shell*,\ **--shell=***shell*
: Specify Bourne shell dialect. Valid values are *sh*, *bash*, *dash*, *ksh*,
and *busybox*.
: Specify Bourne shell dialect. Valid values are *sh*, *bash*, *dash* and *ksh*.
The default is to deduce the shell from the file's `shell` directive,
shebang, or `.bash/.bats/.dash/.ksh` extension, in that order. *sh* refers to
POSIX `sh` (not the system's), and will warn of portability issues.
shebang, or `.bash/.bats/.dash/.ksh/.ebuild/.eclass` extension, in that
order. *sh* refers to POSIX `sh` (not the system's), and will warn of
portability issues.
**-S**\ *SEVERITY*,\ **--severity=***severity*
@@ -256,12 +244,6 @@ Valid keys are:
: Enable an optional check by name, as listed with **--list-optional**.
Only file-wide `enable` directives are considered.
**extended-analysis**
: Set to true/false to enable/disable dataflow analysis. Specifying
`# shellcheck extended-analysis=false` in particularly large (2000+ line)
auto-generated scripts will reduce ShellCheck's resource usage at the
expense of certain checks. Extended analysis is enabled by default.
**external-sources**
: Set to `true` in `.shellcheckrc` to always allow ShellCheck to open
arbitrary files from 'source' statements (the way most tools do).
@@ -397,7 +379,7 @@ long list of wonderful contributors.
# COPYRIGHT
Copyright 2012-2024, Vidar Holen and contributors.
Copyright 2012-2022, Vidar Holen and contributors.
Licensed under the GNU General Public License version 3 or later,
see https://gnu.org/licenses/gpl.html

View File

@@ -21,6 +21,7 @@ import qualified ShellCheck.Analyzer
import ShellCheck.Checker
import ShellCheck.Data
import ShellCheck.Interface
import ShellCheck.PortageVariables
import ShellCheck.Regex
import qualified ShellCheck.Formatter.CheckStyle
@@ -76,8 +77,7 @@ data Options = Options {
externalSources :: Bool,
sourcePaths :: [FilePath],
formatterOptions :: FormatterOptions,
minSeverity :: Severity,
rcfile :: Maybe FilePath
minSeverity :: Severity
}
defaultOptions = Options {
@@ -87,8 +87,7 @@ defaultOptions = Options {
formatterOptions = newFormatterOptions {
foColorOption = ColorAuto
},
minSeverity = StyleC,
rcfile = Nothing
minSeverity = StyleC
}
usageHeader = "Usage: shellcheck [OPTIONS...] FILES..."
@@ -102,8 +101,6 @@ options = [
(ReqArg (Flag "include") "CODE1,CODE2..") "Consider only given types of warnings",
Option "e" ["exclude"]
(ReqArg (Flag "exclude") "CODE1,CODE2..") "Exclude types of warnings",
Option "" ["extended-analysis"]
(ReqArg (Flag "extended-analysis") "bool") "Perform dataflow analysis (default true)",
Option "f" ["format"]
(ReqArg (Flag "format") "FORMAT") $
"Output format (" ++ formatList ++ ")",
@@ -111,9 +108,6 @@ options = [
(NoArg $ Flag "list-optional" "true") "List checks disabled by default",
Option "" ["norc"]
(NoArg $ Flag "norc" "true") "Don't look for .shellcheckrc files",
Option "" ["rcfile"]
(ReqArg (Flag "rcfile") "RCFILE")
"Prefer the specified configuration file over searching for one",
Option "o" ["enable"]
(ReqArg (Flag "enable") "check1,check2..")
"List of optional checks to enable (or 'all')",
@@ -122,7 +116,7 @@ options = [
"Specify path when looking for sourced files (\"SCRIPTDIR\" for script's dir)",
Option "s" ["shell"]
(ReqArg (Flag "shell") "SHELLNAME")
"Specify dialect (sh, bash, dash, ksh, busybox)",
"Specify dialect (sh, bash, dash, ksh)",
Option "S" ["severity"]
(ReqArg (Flag "severity") "SEVERITY")
"Minimum severity of errors to consider (error, warning, info, style)",
@@ -259,9 +253,9 @@ runFormatter sys format options files = do
else SomeProblems
parseEnum name value list =
case lookup value list of
Just value -> return value
Nothing -> do
case filter ((== value) . fst) list of
[(name, value)] -> return value
[] -> do
printErr $ "Unknown value for --" ++ name ++ ". " ++
"Valid options are: " ++ (intercalate ", " $ map fst list)
throwError SupportFailure
@@ -374,11 +368,6 @@ parseOption flag options =
}
}
Flag "rcfile" str -> do
return options {
rcfile = Just str
}
Flag "enable" value ->
let cs = checkSpec options in return options {
checkSpec = cs {
@@ -386,14 +375,6 @@ parseOption flag options =
}
}
Flag "extended-analysis" str -> do
value <- parseBool str
return options {
checkSpec = (checkSpec options) {
csExtendedAnalysis = Just value
}
}
-- This flag is handled specially in 'process'
Flag "format" _ -> return options
@@ -411,23 +392,17 @@ parseOption flag options =
throwError SyntaxFailure
return (Prelude.read num :: Integer)
parseBool str = do
case str of
"true" -> return True
"false" -> return False
_ -> do
printErr $ "Invalid boolean, expected true/false: " ++ str
throwError SyntaxFailure
ioInterface :: Options -> [FilePath] -> IO (SystemInterface IO)
ioInterface options files = do
inputs <- mapM normalize files
cache <- newIORef emptyCache
configCache <- newIORef ("", Nothing)
portageVars <- newIORef Nothing
return (newSystemInterface :: SystemInterface IO) {
siReadFile = get cache inputs,
siFindSource = findSourceFile inputs (sourcePaths options),
siGetConfig = getConfig configCache
siGetConfig = getConfig configCache,
siGetPortageVariables = getOrLoadPortage portageVars
}
where
emptyCache :: Map.Map FilePath String
@@ -469,33 +444,18 @@ ioInterface options files = do
fallback :: FilePath -> IOException -> IO FilePath
fallback path _ = return path
-- Returns the name and contents of .shellcheckrc for the given file
getConfig cache filename =
case rcfile options of
Just file -> do
-- We have a specified rcfile. Ignore normal rcfile resolution.
(path, result) <- readIORef cache
if path == "/"
then return result
else do
result <- readConfig file
when (isNothing result) $
hPutStrLn stderr $ "Warning: unable to read --rcfile " ++ file
writeIORef cache ("/", result)
return result
Nothing -> do
path <- normalize filename
let dir = takeDirectory path
(previousPath, result) <- readIORef cache
if dir == previousPath
then return result
else do
paths <- getConfigPaths dir
result <- findConfig paths
writeIORef cache (dir, result)
return result
getConfig cache filename = do
path <- normalize filename
let dir = takeDirectory path
(previousPath, result) <- readIORef cache
if dir == previousPath
then return result
else do
paths <- getConfigPaths dir
result <- findConfig paths
writeIORef cache (dir, result)
return result
findConfig paths =
case paths of
@@ -533,7 +493,7 @@ ioInterface options files = do
where
handler :: FilePath -> IOException -> IO (String, Bool)
handler file err = do
hPutStrLn stderr $ file ++ ": " ++ show err
putStrLn $ file ++ ": " ++ show err
return ("", True)
andM a b arg = do
@@ -566,6 +526,21 @@ ioInterface options files = do
("SCRIPTDIR":rest) -> joinPath (scriptdir:rest)
_ -> str
getOrLoadPortage cache = do
x <- readIORef cache
case x of
Just m -> do
return m
Nothing -> do
vars <- readPortageVariables `catch` handler
writeIORef cache $ Just vars
return vars
where
handler :: IOException -> IO (Map.Map String [String])
handler e = do
hPutStrLn stderr $ "Error finding portage repos, eclass definitions will be ignored: " ++ show e
return $ Map.empty
inputFile file = do
(handle, shouldCache) <-
if file == "-"

View File

@@ -23,7 +23,7 @@ description: |
# snap connect shellcheck:removable-media
version: git
base: core20
base: core18
grade: stable
confinement: strict
@@ -40,16 +40,16 @@ parts:
source: .
build-packages:
- cabal-install
stage-packages:
- libatomic1
- squid
override-build: |
# Give ourselves enough memory to build
dd if=/dev/zero of=/tmp/swap bs=1M count=2000
mkswap /tmp/swap
swapon /tmp/swap
# See comments in .snapsquid.conf
[ "$http_proxy" ] && {
squid3 -f .snapsquid.conf
export http_proxy="http://localhost:8888"
sleep 3
}
cabal sandbox init
cabal update
cabal update || cat /var/log/squid/*
cabal install -j
install -d $SNAPCRAFT_PART_INSTALL/usr/bin

View File

@@ -152,7 +152,6 @@ data Annotation =
| ShellOverride String
| SourcePath String
| ExternalSources Bool
| ExtendedAnalysis Bool
deriving (Show, Eq)
data ConditionType = DoubleBracket | SingleBracket deriving (Show, Eq)

View File

@@ -31,14 +31,11 @@ import Data.Functor
import Data.Functor.Identity
import Data.List
import Data.Maybe
import qualified Data.List.NonEmpty as NE
import qualified Data.Map as Map
import Numeric (showHex)
import Test.QuickCheck
arguments (T_SimpleCommand _ _ (cmd:args)) = args
-- Is this a type of loop?
isLoop t = case t of
T_WhileExpression {} -> True
@@ -158,10 +155,9 @@ isFlag token =
_ -> False
-- Is this token a flag where the - is unquoted?
isUnquotedFlag token =
case getLeadingUnquotedString token of
Just ('-':_) -> True
_ -> False
isUnquotedFlag token = fromMaybe False $ do
str <- getLeadingUnquotedString token
return $ "-" `isPrefixOf` str
-- getGnuOpts "erd:u:" will parse a list of arguments tokens like `read`
-- -re -d : -u 3 bar
@@ -548,17 +544,16 @@ getCommandNameAndToken direct t = fromMaybe (Nothing, t) $ do
return t
_ -> fail ""
-- If a command substitution is a single command, get its name.
-- $(date +%s) = Just "date"
getCommandNameFromExpansion :: Token -> Maybe String
getCommandNameFromExpansion t =
-- If a command substitution is a single SimpleCommand, return it.
getSimpleCommandFromExpansion :: Token -> Maybe Token
getSimpleCommandFromExpansion t =
case t of
T_DollarExpansion _ [c] -> extract c
T_Backticked _ [c] -> extract c
T_DollarBraceCommandExpansion _ [c] -> extract c
_ -> Nothing
where
extract (T_Pipeline _ _ [cmd]) = getCommandName cmd
extract (T_Pipeline _ _ [c]) = getCommand c
extract _ = Nothing
-- Get the basename of a token representing a command
@@ -566,6 +561,10 @@ getCommandBasename = fmap basename . getCommandName
basename = reverse . takeWhile (/= '/') . reverse
-- Get the arguments to a command
arguments (T_SimpleCommand _ _ (cmd:args)) = args
arguments t = maybe [] arguments (getCommand t)
isAssignment t =
case t of
T_Redirecting _ _ w -> isAssignment w
@@ -760,8 +759,8 @@ prop_executableFromShebang6 = executableFromShebang "/usr/bin/env --split-string
prop_executableFromShebang7 = executableFromShebang "/usr/bin/env --split-string bash -x" == "bash"
prop_executableFromShebang8 = executableFromShebang "/usr/bin/env --split-string foo=bar bash -x" == "bash"
prop_executableFromShebang9 = executableFromShebang "/usr/bin/env foo=bar dash" == "dash"
prop_executableFromShebang10 = executableFromShebang "/bin/busybox sh" == "busybox sh"
prop_executableFromShebang11 = executableFromShebang "/bin/busybox ash" == "busybox ash"
prop_executableFromShebang10 = executableFromShebang "/bin/busybox sh" == "ash"
prop_executableFromShebang11 = executableFromShebang "/bin/busybox ash" == "ash"
-- Get the shell executable from a string like '/usr/bin/env bash'
executableFromShebang :: String -> String
@@ -778,8 +777,7 @@ executableFromShebang = shellFor
[x] -> basename x
(first:second:args) | basename first == "busybox" ->
case basename second of
"sh" -> "busybox sh"
"ash" -> "busybox ash"
"sh" -> "ash" -- busybox sh is ash
x -> x
(first:args) | basename first == "env" ->
fromEnvArgs args
@@ -859,7 +857,8 @@ getBracedModifier s = headOrDefault "" $ do
-- Get the variables from indices like ["x", "y"] in ${var[x+y+1]}
prop_getIndexReferences1 = getIndexReferences "var[x+y+1]" == ["x", "y"]
getIndexReferences s = fromMaybe [] $ do
index:_ <- matchRegex re s
match <- matchRegex re s
index <- match !!! 0
return $ matchAllStrings variableNameRegex index
where
re = mkRegex "(\\[.*\\])"
@@ -870,7 +869,8 @@ prop_getOffsetReferences3 = getOffsetReferences "[foo]:bar" == ["bar"]
prop_getOffsetReferences4 = getOffsetReferences "[foo]:bar:baz" == ["bar", "baz"]
getOffsetReferences mods = fromMaybe [] $ do
-- if mods start with [, then drop until ]
_:offsets:_ <- matchRegex re mods
match <- matchRegex re mods
offsets <- match !!! 1
return $ matchAllStrings variableNameRegex offsets
where
re = mkRegex "^(\\[.+\\])? *:([^-=?+].*)"
@@ -897,7 +897,10 @@ getUnmodifiedParameterExpansion t =
_ -> Nothing
--- A list of the element and all its parents up to the root node.
getPath tree = NE.unfoldr $ \t -> (t, Map.lookup (getId t) tree)
getPath tree t = t :
case Map.lookup (getId t) tree of
Nothing -> []
Just parent -> getPath tree parent
isClosingFileOp op =
case op of
@@ -910,11 +913,20 @@ getEnableDirectives root =
T_Annotation _ list _ -> [s | EnableComment s <- list]
_ -> []
getExtendedAnalysisDirective :: Token -> Maybe Bool
getExtendedAnalysisDirective root =
case root of
T_Annotation _ list _ -> listToMaybe $ [s | ExtendedAnalysis s <- list]
commandExpansionShouldBeSplit t = do
cmd <- getSimpleCommandFromExpansion t
name <- getCommandName cmd
case () of
-- Should probably be split
_ | name `elem` ["seq", "pgrep"] -> return True
-- Portage macros that return a single word or nothing
_ | name `elem` ["usev", "use_with", "use_enable"] -> return True
-- Portage macros that are fine as long as the arguments have no spaces
_ | name `elem` ["usex", "meson_use", "meson_feature"] -> do
return . not $ any (' ' `elem`) $ map (getLiteralStringDef " ") $ arguments cmd
_ -> Nothing
return []
runTests = $quickCheckAll

View File

@@ -1,5 +1,5 @@
{-
Copyright 2012-2024 Vidar Holen
Copyright 2012-2022 Vidar Holen
This file is part of ShellCheck.
https://www.shellcheck.net
@@ -19,7 +19,6 @@
-}
{-# LANGUAGE TemplateHaskell #-}
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE PatternGuards #-}
module ShellCheck.Analytics (checker, optionalChecks, ShellCheck.Analytics.runTests) where
import ShellCheck.AST
@@ -47,7 +46,6 @@ import Data.Maybe
import Data.Ord
import Data.Semigroup
import Debug.Trace -- STRIP
import qualified Data.List.NonEmpty as NE
import qualified Data.Map.Strict as Map
import qualified Data.Set as S
import Test.QuickCheck.All (forAllProperties)
@@ -316,7 +314,7 @@ runAndGetComments f s = do
let pr = pScript s
root <- prRoot pr
let spec = defaultSpec pr
let params = makeParameters spec
let params = runIdentity $ makeParameters (mockedSystemInterface []) spec
return $
filterByAnnotation spec params $
f params root
@@ -346,11 +344,13 @@ dist a b
hasFloatingPoint params = shellType params == Ksh
-- Checks whether the current parent path is part of a condition
isCondition (x NE.:| xs) = foldr go (const False) xs x
isCondition [] = False
isCondition [_] = False
isCondition (child:parent:rest) =
case child of
T_BatsTest {} -> True -- count anything in a @test as conditional
_ -> getId child `elem` map getId (getConditionChildren parent) || isCondition (parent:rest)
where
go _ _ T_BatsTest{} = True -- count anything in a @test as conditional
go parent go_rest child =
getId child `elem` map getId (getConditionChildren parent) || go_rest parent
getConditionChildren t =
case t of
T_AndIf _ left right -> [left]
@@ -467,8 +467,9 @@ checkAssignAteCommand _ (T_SimpleCommand id [T_Assignment _ _ _ _ assignmentTerm
where
isCommonCommand (Just s) = s `elem` commonCommands
isCommonCommand _ = False
firstWordIsArg (head:_) = isGlob head || isUnquotedFlag head
firstWordIsArg [] = False
firstWordIsArg list = fromMaybe False $ do
head <- list !!! 0
return $ isGlob head || isUnquotedFlag head
checkAssignAteCommand _ _ = return ()
@@ -489,7 +490,9 @@ prop_checkWrongArit2 = verify checkWrongArithmeticAssignment "n=2; i=n*2"
checkWrongArithmeticAssignment params (T_SimpleCommand id [T_Assignment _ _ _ _ val] []) =
sequence_ $ do
str <- getNormalString val
var:op:_ <- matchRegex regex str
match <- matchRegex regex str
var <- match !!! 0
op <- match !!! 1
Map.lookup var references
return . warn (getId val) 2100 $
"Use $((..)) for arithmetics, e.g. i=$((i " ++ op ++ " 2))"
@@ -643,10 +646,10 @@ prop_checkShebang9 = verifyNotTree checkShebang "# shellcheck shell=sh\ntrue"
prop_checkShebang10 = verifyNotTree checkShebang "#!foo\n# shellcheck shell=sh ignore=SC2239\ntrue"
prop_checkShebang11 = verifyTree checkShebang "#!/bin/sh/\ntrue"
prop_checkShebang12 = verifyTree checkShebang "#!/bin/sh/ -xe\ntrue"
prop_checkShebang13 = verifyNotTree checkShebang "#!/bin/busybox sh"
prop_checkShebang13 = verifyTree checkShebang "#!/bin/busybox sh"
prop_checkShebang14 = verifyNotTree checkShebang "#!/bin/busybox sh\n# shellcheck shell=sh\n"
prop_checkShebang15 = verifyNotTree checkShebang "#!/bin/busybox sh\n# shellcheck shell=dash\n"
prop_checkShebang16 = verifyNotTree checkShebang "#!/bin/busybox ash"
prop_checkShebang16 = verifyTree checkShebang "#!/bin/busybox ash"
prop_checkShebang17 = verifyNotTree checkShebang "#!/bin/busybox ash\n# shellcheck shell=dash\n"
prop_checkShebang18 = verifyNotTree checkShebang "#!/bin/busybox ash\n# shellcheck shell=sh\n"
checkShebang params (T_Annotation _ list t) =
@@ -789,6 +792,11 @@ prop_checkUnquotedExpansions8 = verifyNot checkUnquotedExpansions "set -- $(seq
prop_checkUnquotedExpansions9 = verifyNot checkUnquotedExpansions "echo foo `# inline comment`"
prop_checkUnquotedExpansions10 = verify checkUnquotedExpansions "#!/bin/sh\nexport var=$(val)"
prop_checkUnquotedExpansions11 = verifyNot checkUnquotedExpansions "ps -p $(pgrep foo)"
prop_checkUnquotedExpansions12 = verify checkUnquotedExpansions "#!/bin/sh\nexport var=$(val)"
prop_checkUnquotedExpansions13 = verifyNot checkUnquotedExpansions "echo $(usev X)"
prop_checkUnquotedExpansions14 = verifyNot checkUnquotedExpansions "echo $(usex X \"\" Y)"
prop_checkUnquotedExpansions15 = verify checkUnquotedExpansions "echo $(usex X \"Y Z\" W)"
checkUnquotedExpansions params =
check
where
@@ -798,12 +806,9 @@ checkUnquotedExpansions params =
check _ = return ()
tree = parentMap params
examine t contents =
unless (null contents || shouldBeSplit t || isQuoteFree (shellType params) tree t || usedAsCommandName tree t) $
unless (null contents || commandExpansionShouldBeSplit t == Just True || isQuoteFree (shellType params) tree t || usedAsCommandName tree t) $
warn (getId t) 2046 "Quote this to prevent word splitting."
shouldBeSplit t =
getCommandNameFromExpansion t `elem` [Just "seq", Just "pgrep"]
prop_checkRedirectToSame = verify checkRedirectToSame "cat foo > foo"
prop_checkRedirectToSame2 = verify checkRedirectToSame "cat lol | sed -e 's/a/b/g' > lol"
@@ -843,14 +848,14 @@ checkRedirectToSame params s@(T_Pipeline _ _ list) =
getRedirs _ = []
special x = "/dev/" `isPrefixOf` concat (oversimplify x)
isInput t =
case NE.tail $ getPath (parentMap params) t of
case drop 1 $ getPath (parentMap params) t of
T_IoFile _ op _:_ ->
case op of
T_Less _ -> True
_ -> False
_ -> False
isOutput t =
case NE.tail $ getPath (parentMap params) t of
case drop 1 $ getPath (parentMap params) t of
T_IoFile _ op _:_ ->
case op of
T_Greater _ -> True
@@ -1034,9 +1039,6 @@ checkStderrRedirect params redir@(T_Redirecting _ [
checkStderrRedirect _ _ = return ()
lt x = trace ("Tracing " ++ show x) x -- STRIP
ltt t = trace ("Tracing " ++ show t) -- STRIP
prop_checkSingleQuotedVariables = verify checkSingleQuotedVariables "echo '$foo'"
prop_checkSingleQuotedVariables2 = verify checkSingleQuotedVariables "echo 'lol$1.jpg'"
@@ -1067,6 +1069,10 @@ prop_checkSingleQuotedVariables22 = verifyNot checkSingleQuotedVariables "jq '$_
prop_checkSingleQuotedVariables23 = verifyNot checkSingleQuotedVariables "command jq '$__loc__'"
prop_checkSingleQuotedVariables24 = verifyNot checkSingleQuotedVariables "exec jq '$__loc__'"
prop_checkSingleQuotedVariables25 = verifyNot checkSingleQuotedVariables "exec -c -a foo jq '$__loc__'"
prop_checkSingleQuotedVariablesCros1 = verifyNot checkSingleQuotedVariables "python_gen_any_dep 'dev-python/pyyaml[${PYTHON_USEDEP}]'"
prop_checkSingleQuotedVariablesCros2 = verifyNot checkSingleQuotedVariables "python_gen_cond_dep 'dev-python/unittest2[${PYTHON_USEDEP}]' python2_7 pypy"
prop_checkSingleQuotedVariablesCros3 = verifyNot checkSingleQuotedVariables "version_format_string '${PN}_source_$1_$2-$3_$4'"
checkSingleQuotedVariables params t@(T_SingleQuoted id s) =
@@ -1084,7 +1090,7 @@ checkSingleQuotedVariables params t@(T_SingleQuoted id s) =
return $ if name == "find" then getFindCommand cmd else if name == "git" then getGitCommand cmd else if name == "mumps" then getMumpsCommand cmd else name
isProbablyOk =
any isOkAssignment (NE.take 3 $ getPath parents t)
any isOkAssignment (take 3 $ getPath parents t)
|| commandName `elem` [
"trap"
,"sh"
@@ -1106,6 +1112,9 @@ checkSingleQuotedVariables params t@(T_SingleQuoted id s) =
,"git filter-branch"
,"mumps -run %XCMD"
,"mumps -run LOOP%XCMD"
,"python_gen_any_dep"
,"python_gen_cond_dep"
,"version_format_string"
]
|| "awk" `isSuffixOf` commandName
|| "perl" `isPrefixOf` commandName
@@ -1201,7 +1210,6 @@ checkNumberComparisons params (TC_Binary id typ op lhs rhs) = do
case shellType params of
Sh -> return () -- These are unsupported and will be caught by bashism checks.
Dash -> err id 2073 $ "Escape \\" ++ op ++ " to prevent it redirecting."
BusyboxSh -> err id 2073 $ "Escape \\" ++ op ++ " to prevent it redirecting."
_ -> err id 2073 $ "Escape \\" ++ op ++ " to prevent it redirecting (or switch to [[ .. ]])."
when (op `elem` arithmeticBinaryTestOps) $ do
@@ -1262,8 +1270,7 @@ checkNumberComparisons params (TC_Binary id typ op lhs rhs) = do
str = concat $ oversimplify c
var = getBracedReference str
in fromMaybe False $ do
cfga <- cfgAnalysis params
state <- CF.getIncomingState cfga id
state <- CF.getIncomingState (cfgAnalysis params) id
value <- Map.lookup var $ CF.variablesInScope state
return $ CF.numericalStatus (CF.variableValue value) >= CF.NumericalStatusMaybe
_ ->
@@ -1441,14 +1448,14 @@ prop_checkConstantNullary5 = verify checkConstantNullary "[[ true ]]"
prop_checkConstantNullary6 = verify checkConstantNullary "[ 1 ]"
prop_checkConstantNullary7 = verify checkConstantNullary "[ false ]"
checkConstantNullary _ (TC_Nullary _ _ t) | isConstant t =
case onlyLiteralString t of
case fromMaybe "" $ getLiteralString t of
"false" -> err (getId t) 2158 "[ false ] is true. Remove the brackets."
"0" -> err (getId t) 2159 "[ 0 ] is true. Use 'false' instead."
"true" -> style (getId t) 2160 "Instead of '[ true ]', just use 'true'."
"1" -> style (getId t) 2161 "Instead of '[ 1 ]', use 'true'."
_ -> err (getId t) 2078 "This expression is constant. Did you forget a $ somewhere?"
where
string = onlyLiteralString t
string = fromMaybe "" $ getLiteralString t
checkConstantNullary _ _ = return ()
@@ -1457,8 +1464,9 @@ prop_checkForDecimals2 = verify checkForDecimals "foo[1.2]=bar"
prop_checkForDecimals3 = verifyNot checkForDecimals "declare -A foo; foo[1.2]=bar"
checkForDecimals params t@(TA_Expansion id _) = sequence_ $ do
guard $ not (hasFloatingPoint params)
first:rest <- getLiteralString t
guard $ isDigit first && '.' `elem` rest
str <- getLiteralString t
first <- str !!! 0
guard $ isDigit first && '.' `elem` str
return $ err id 2079 "(( )) doesn't support decimals. Use bc or awk."
checkForDecimals _ _ = return ()
@@ -1492,7 +1500,7 @@ checkArithmeticDeref params t@(TA_Expansion _ [T_DollarBraced id _ l]) =
where
isException [] = True
isException s@(h:_) = any (`elem` "/.:#%?*@$-!+=^,") s || isDigit h
getWarning = fromMaybe noWarning . msum . NE.map warningFor $ parents params t
getWarning = fromMaybe noWarning . msum . map warningFor $ parents params t
warningFor t =
case t of
T_Arithmetic {} -> return normalWarning
@@ -1820,7 +1828,7 @@ checkInexplicablyUnquoted params (T_NormalWord id tokens) = mapM_ check (tails t
T_Literal id s
| not (quotesSingleThing a && quotesSingleThing b
|| s `elem` ["=", ":", "/"]
|| isSpecial (NE.toList $ getPath (parentMap params) trapped)
|| isSpecial (getPath (parentMap params) trapped)
) ->
warnAboutLiteral id
_ -> return ()
@@ -2038,7 +2046,7 @@ doVariableFlowAnalysis readFunc writeFunc empty flow = evalState (
-- from $foo=bar to foo=bar. This is not pretty but ok.
quotesMayConflictWithSC2281 params t =
case getPath (parentMap params) t of
_ NE.:| T_NormalWord parentId (me:T_Literal _ ('=':_):_) : T_SimpleCommand _ _ (cmd:_) : _ ->
_ : T_NormalWord parentId (me:T_Literal _ ('=':_):_) : T_SimpleCommand _ _ (cmd:_) : _ ->
(getId t) == (getId me) && (parentId == getId cmd)
_ -> False
@@ -2144,8 +2152,7 @@ checkSpacefulnessCfg' dirtyPass params token@(T_DollarBraced id _ list) =
&& not (usedAsCommandName parents token)
isClean = fromMaybe False $ do
cfga <- cfgAnalysis params
state <- CF.getIncomingState cfga id
state <- CF.getIncomingState (cfgAnalysis params) id
value <- Map.lookup name $ CF.variablesInScope state
return $ isCleanState value
@@ -2274,7 +2281,7 @@ checkFunctionsUsedExternally params t =
(Just str, t) -> do
let name = basename str
let args = skipOver t argv
let argStrings = map (\x -> (onlyLiteralString x, x)) args
let argStrings = map (\x -> (fromMaybe "" $ getLiteralString x, x)) args
let candidates = getPotentialCommands name argStrings
mapM_ (checkArg name (getId t)) candidates
_ -> return ()
@@ -2391,7 +2398,7 @@ checkUnusedAssignments params t = execWriter (mapM_ warnFor unused)
name ++ " appears unused. Verify use (or export if used externally)."
stripSuffix = takeWhile isVariableChar
defaultMap = Map.fromList $ zip internalVariables $ repeat ()
defaultMap = Map.fromList $ zip (internalVariables ++ additionalKnownVariables params) $ repeat ()
prop_checkUnassignedReferences1 = verifyTree checkUnassignedReferences "echo $foo"
prop_checkUnassignedReferences2 = verifyNotTree checkUnassignedReferences "foo=hello; echo $foo"
@@ -2450,7 +2457,7 @@ checkUnassignedReferences = checkUnassignedReferences' False
checkUnassignedReferences' includeGlobals params t = warnings
where
(readMap, writeMap) = execState (mapM tally $ variableFlow params) (Map.empty, Map.empty)
defaultAssigned = Map.fromList $ map (\a -> (a, ())) $ filter (not . null) internalVariables
defaultAssigned = Map.fromList $ map (\a -> (a, ())) $ filter (not . null) (internalVariables ++ additionalKnownVariables params)
tally (Assignment (_, _, name, _)) =
modify (\(read, written) -> (read, Map.insert name () written))
@@ -2650,7 +2657,7 @@ checkPrefixAssignmentReference params t@(T_DollarBraced id _ value) =
check path
where
name = getBracedReference $ concat $ oversimplify value
path = NE.toList $ getPath (parentMap params) t
path = getPath (parentMap params) t
idPath = map getId path
check [] = return ()
@@ -2699,7 +2706,7 @@ checkCharRangeGlob p t@(T_Glob id str) |
return $ isCommandMatch cmd (`elem` ["tr", "read"])
-- Check if this is a dereferencing context like [[ -v array[operandhere] ]]
isDereferenced = fromMaybe False . msum . NE.map isDereferencingOp . getPath (parentMap p)
isDereferenced = fromMaybe False . msum . map isDereferencingOp . getPath (parentMap p)
isDereferencingOp t =
case t of
TC_Binary _ DoubleBracket str _ _ -> return $ isDereferencingBinaryOp str
@@ -2750,18 +2757,19 @@ prop_checkLoopKeywordScope5 = verify checkLoopKeywordScope "if true; then break;
prop_checkLoopKeywordScope6 = verify checkLoopKeywordScope "while true; do true | { break; }; done"
prop_checkLoopKeywordScope7 = verifyNot checkLoopKeywordScope "#!/bin/ksh\nwhile true; do true | { break; }; done"
checkLoopKeywordScope params t |
Just name <- getCommandName t, name `elem` ["continue", "break"] =
if any isLoop path
then case map subshellType $ filter (not . isFunction) path of
name `elem` map Just ["continue", "break"] =
if not $ any isLoop path
then if any isFunction $ take 1 path
-- breaking at a source/function invocation is an abomination. Let's ignore it.
then err (getId t) 2104 $ "In functions, use return instead of " ++ fromJust name ++ "."
else err (getId t) 2105 $ fromJust name ++ " is only valid in loops."
else case map subshellType $ filter (not . isFunction) path of
Just str:_ -> warn (getId t) 2106 $
"This only exits the subshell caused by the " ++ str ++ "."
_ -> return ()
else case path of
-- breaking at a source/function invocation is an abomination. Let's ignore it.
h:_ | isFunction h -> err (getId t) 2104 $ "In functions, use return instead of " ++ name ++ "."
_ -> err (getId t) 2105 $ name ++ " is only valid in loops."
where
path = let p = getPath (parentMap params) t in NE.filter relevant p
name = getCommandName t
path = let p = getPath (parentMap params) t in filter relevant p
subshellType t = case leadType params t of
NoneScope -> Nothing
SubshellScope str -> return str
@@ -2780,7 +2788,6 @@ checkFunctionDeclarations params
when (hasKeyword && hasParens) $
err id 2111 "ksh does not allow 'function' keyword and '()' at the same time."
Dash -> forSh
BusyboxSh -> forSh
Sh -> forSh
where
@@ -2822,11 +2829,13 @@ checkUnpassedInFunctions params root =
execWriter $ mapM_ warnForGroup referenceGroups
where
functionMap :: Map.Map String Token
functionMap = Map.fromList $ execWriter $ doAnalysis (tell . maybeToList . findFunction) root
functionMap = Map.fromList $
map (\t@(T_Function _ _ _ name _) -> (name,t)) functions
functions = execWriter $ doAnalysis (tell . maybeToList . findFunction) root
findFunction t@(T_Function id _ _ name body)
| any (isPositionalReference t) flow && not (any isPositionalAssignment flow)
= return (name,t)
= return t
where flow = getVariableFlow params body
findFunction _ = Nothing
@@ -3214,7 +3223,7 @@ checkLoopVariableReassignment params token =
return $ do
warn (getId token) 2165 "This nested loop overrides the index variable of its parent."
warn (getId next) 2167 "This parent loop has its index variable overridden."
path = NE.tail $ getPath (parentMap params) token
path = drop 1 $ getPath (parentMap params) token
loopVariable :: Token -> Maybe String
loopVariable t =
case t of
@@ -3287,17 +3296,17 @@ checkReturnAgainstZero params token =
-- We don't want to warn about composite expressions like
-- [[ $? -eq 0 || $? -eq 4 ]] since these can be annoying to rewrite.
isOnlyTestInCommand t =
case NE.tail $ getPath (parentMap params) t of
(T_Condition {}):_ -> True
(T_Arithmetic {}):_ -> True
(TA_Sequence _ [_]):(T_Arithmetic {}):_ -> True
case getPath (parentMap params) t of
_:(T_Condition {}):_ -> True
_:(T_Arithmetic {}):_ -> True
_:(TA_Sequence _ [_]):(T_Arithmetic {}):_ -> True
-- Some negations and groupings are also fine
next@(TC_Unary _ _ "!" _):_ -> isOnlyTestInCommand next
next@(TA_Unary _ "!" _):_ -> isOnlyTestInCommand next
next@(TC_Group {}):_ -> isOnlyTestInCommand next
next@(TA_Sequence _ [_]):_ -> isOnlyTestInCommand next
next@(TA_Parentesis _ _):_ -> isOnlyTestInCommand next
_:next@(TC_Unary _ _ "!" _):_ -> isOnlyTestInCommand next
_:next@(TA_Unary _ "!" _):_ -> isOnlyTestInCommand next
_:next@(TC_Group {}):_ -> isOnlyTestInCommand next
_:next@(TA_Sequence _ [_]):_ -> isOnlyTestInCommand next
_:next@(TA_Parentesis _ _):_ -> isOnlyTestInCommand next
_ -> False
-- TODO: Do better $? tracking and filter on whether
@@ -3317,7 +3326,7 @@ checkReturnAgainstZero params token =
isFirstCommandInFunction = fromMaybe False $ do
let path = getPath (parentMap params) token
func <- find isFunction path
func <- listToMaybe $ filter isFunction path
cmd <- getClosestCommand (parentMap params) token
return $ getId cmd == getId (getFirstCommandInFunction func)
@@ -3362,7 +3371,7 @@ checkRedirectedNowhere params token =
_ -> return ()
where
isInExpansion t =
case NE.tail $ getPath (parentMap params) t of
case drop 1 $ getPath (parentMap params) t of
T_DollarExpansion _ [_] : _ -> True
T_Backticked _ [_] : _ -> True
t@T_Annotation {} : _ -> isInExpansion t
@@ -3506,6 +3515,7 @@ prop_checkSplittingInArrays5 = verifyNot checkSplittingInArrays "a=( $! $$ $# )"
prop_checkSplittingInArrays6 = verifyNot checkSplittingInArrays "a=( ${#arr[@]} )"
prop_checkSplittingInArrays7 = verifyNot checkSplittingInArrays "a=( foo{1,2} )"
prop_checkSplittingInArrays8 = verifyNot checkSplittingInArrays "a=( * )"
prop_checkSplittingInArrays9 = verifyNot checkSplittingInArrays "a=( $(use_enable foo) )"
checkSplittingInArrays params t =
case t of
T_Array _ elements -> mapM_ check elements
@@ -3515,6 +3525,7 @@ checkSplittingInArrays params t =
T_NormalWord _ parts -> mapM_ checkPart parts
_ -> return ()
checkPart part = case part of
_ | commandExpansionShouldBeSplit part == Just True -> return ()
T_DollarExpansion id _ -> forCommand id
T_DollarBraceCommandExpansion id _ -> forCommand id
T_Backticked id _ -> forCommand id
@@ -3836,7 +3847,7 @@ checkSubshelledTests params t =
isFunctionBody path =
case path of
(_ NE.:| f:_) -> isFunction f
(_:f:_) -> isFunction f
_ -> False
isTestStructure t =
@@ -3863,7 +3874,7 @@ checkSubshelledTests params t =
-- This technically also triggers for `if true; then ( test ); fi`
-- but it's still a valid suggestion.
isCompoundCondition chain =
case dropWhile skippable (NE.tail chain) of
case dropWhile skippable (drop 1 chain) of
T_IfExpression {} : _ -> True
T_WhileExpression {} : _ -> True
T_UntilExpression {} : _ -> True
@@ -4026,7 +4037,7 @@ checkUselessBang params t = when (hasSetE params) $ mapM_ check (getNonReturning
isFunctionBody t =
case getPath (parentMap params) t of
_ NE.:| T_Function {}:_-> True
_:T_Function {}:_-> True
_ -> False
dropLast t =
@@ -4041,8 +4052,7 @@ prop_checkModifiedArithmeticInRedirection3 = verifyNot checkModifiedArithmeticIn
prop_checkModifiedArithmeticInRedirection4 = verify checkModifiedArithmeticInRedirection "cat <<< $((i++))"
prop_checkModifiedArithmeticInRedirection5 = verify checkModifiedArithmeticInRedirection "cat << foo\n$((i++))\nfoo\n"
prop_checkModifiedArithmeticInRedirection6 = verifyNot checkModifiedArithmeticInRedirection "#!/bin/dash\nls > $((i=i+1))"
prop_checkModifiedArithmeticInRedirection7 = verifyNot checkModifiedArithmeticInRedirection "#!/bin/busybox sh\ncat << foo\n$((i++))\nfoo\n"
checkModifiedArithmeticInRedirection params t = unless (shellType params == Dash || shellType params == BusyboxSh) $
checkModifiedArithmeticInRedirection params t = unless (shellType params == Dash) $
case t of
T_Redirecting _ redirs (T_SimpleCommand _ _ (_:_)) -> mapM_ checkRedirs redirs
_ -> return ()
@@ -4354,7 +4364,6 @@ checkEqualsInCommand params originalToken =
Bash -> errWithFix id 2277 "Use BASH_ARGV0 to assign to $0 in bash (or use [ ] to compare)." bashfix
Ksh -> err id 2278 "$0 can't be assigned in Ksh (but it does reflect the current function)."
Dash -> err id 2279 "$0 can't be assigned in Dash. This becomes a command name."
BusyboxSh -> err id 2279 "$0 can't be assigned in Busybox Ash. This becomes a command name."
_ -> err id 2280 "$0 can't be assigned this way, and there is no portable alternative."
leadingNumberMsg id =
err id 2282 "Variable names can't start with numbers, so this is interpreted as a command."
@@ -4377,9 +4386,9 @@ checkEqualsInCommand params originalToken =
return $ isVariableName str
isLeadingNumberVar s =
case takeWhile (/= '=') s of
lead@(x:_) -> isDigit x && all isVariableChar lead && not (all isDigit lead)
[] -> False
let lead = takeWhile (/= '=') s
in not (null lead) && isDigit (head lead)
&& all isVariableChar lead && not (all isDigit lead)
msg cmd leading (T_Literal litId s) = do
-- There are many different cases, and the order of the branches matter.
@@ -4509,7 +4518,7 @@ prop_checkCommandWithTrailingSymbol9 = verifyNot checkCommandWithTrailingSymbol
checkCommandWithTrailingSymbol _ t =
case t of
T_SimpleCommand _ _ (cmd:_) ->
let str = getLiteralStringDef "x" cmd
let str = fromJust $ getLiteralStringExt (\_ -> Just "x") cmd
last = lastOrDefault 'x' str
in
case str of
@@ -4624,8 +4633,7 @@ checkArrayValueUsedAsIndex params _ =
-- Is this one of the 'for' arrays?
(loopWord, _) <- find ((==arrayName) . snd) arrays
-- Are we still in this loop?
let loopId = getId loop
guard $ any (\t -> loopId == getId t) (getPath parents t)
guard $ getId loop `elem` map getId (getPath parents t)
return [
makeComment WarningC (getId loopWord) 2302 "This loops over values. To loop over keys, use \"${!array[@]}\".",
makeComment WarningC (getId arrayRef) 2303 $ (e4m name) ++ " is an array value, not a key. Use directly or loop over keys instead."
@@ -4707,7 +4715,7 @@ checkSetESuppressed params t =
literalArg <- getUnquotedLiteral cmd
Map.lookup literalArg functions_
checkCmd cmd = go $ NE.toList $ getPath (parentMap params) cmd
checkCmd cmd = go $ getPath (parentMap params) cmd
where
go (child:parent:rest) = do
case parent of
@@ -4821,15 +4829,15 @@ checkExtraMaskedReturns params t =
++ "separately to avoid masking its return value (or use '|| true' "
++ "to ignore).")
isMaskDeliberate t = any isOrIf $ NE.init $ parents params t
isMaskDeliberate t = hasParent isOrIf t
where
isOrIf (T_OrIf _ _ (T_Pipeline _ _ [T_Redirecting _ _ cmd]))
isOrIf _ (T_OrIf _ _ (T_Pipeline _ _ [T_Redirecting _ _ cmd]))
= getCommandBasename cmd `elem` [Just "true", Just ":"]
isOrIf _ = False
isOrIf _ _ = False
isCheckedElsewhere t = any isDeclaringCommand $ NE.tail $ parents params t
isCheckedElsewhere t = hasParent isDeclaringCommand t
where
isDeclaringCommand t = fromMaybe False $ do
isDeclaringCommand t _ = fromMaybe False $ do
cmd <- getCommand t
basename <- getCommandBasename cmd
return $
@@ -4849,7 +4857,16 @@ checkExtraMaskedReturns params t =
,"shopt"
]
isTransparentCommand t = getCommandBasename t == Just "time"
isTransparentCommand t = fromMaybe False $ do
basename <- getCommandBasename t
return $ basename == "time"
parentChildPairs t = go $ parents params t
where
go (child:parent:rest) = (parent, child):go (parent:rest)
go _ = []
hasParent pred t = any (uncurry pred) (parentChildPairs t)
-- hard error on negated command that is not last
@@ -4898,8 +4915,7 @@ prop_checkCommandIsUnreachable3 = verifyNot checkCommandIsUnreachable "foo; bar
checkCommandIsUnreachable params t =
case t of
T_Pipeline {} -> sequence_ $ do
cfga <- cfgAnalysis params
state <- CF.getIncomingState cfga id
state <- CF.getIncomingState (cfgAnalysis params) id
guard . not $ CF.stateIsReachable state
guard . not $ isSourced params t
return $ info id 2317 "Command appears to be unreachable. Check usage (or ignore if invoked indirectly)."
@@ -4921,15 +4937,14 @@ checkOverwrittenExitCode params t =
_ -> return ()
where
check id = sequence_ $ do
cfga <- cfgAnalysis params
state <- CF.getIncomingState cfga id
state <- CF.getIncomingState (cfgAnalysis params) id
let exitCodeIds = CF.exitCodes state
guard . not $ S.null exitCodeIds
let idToToken = idMap params
exitCodeTokens <- traverse (\k -> Map.lookup k idToToken) $ S.toList exitCodeIds
exitCodeTokens <- sequence $ map (\k -> Map.lookup k idToToken) $ S.toList exitCodeIds
return $ do
when (all isCondition exitCodeTokens && not (usedUnconditionally cfga t exitCodeIds)) $
when (all isCondition exitCodeTokens && not (usedUnconditionally t exitCodeIds)) $
warn id 2319 "This $? refers to a condition, not a command. Assign to a variable to avoid it being overwritten."
when (all isPrinting exitCodeTokens) $
warn id 2320 "This $? refers to echo/printf, not a previous command. Assign to variable to avoid it being overwritten."
@@ -4942,8 +4957,8 @@ checkOverwrittenExitCode params t =
-- If we don't do anything based on the condition, assume we wanted the condition itself
-- This helps differentiate `x; [ $? -gt 0 ] && exit $?` vs `[ cond ]; exit $?`
usedUnconditionally cfga t testIds =
all (\c -> CF.doesPostDominate cfga (getId t) c) testIds
usedUnconditionally t testIds =
all (\c -> CF.doesPostDominate (cfgAnalysis params) (getId t) c) testIds
isPrinting t =
case getCommandBasename t of
@@ -5013,8 +5028,7 @@ prop_checkPlusEqualsNumber9 = verifyNot checkPlusEqualsNumber "declare -ia var;
checkPlusEqualsNumber params t =
case t of
T_Assignment id Append var _ word -> sequence_ $ do
cfga <- cfgAnalysis params
state <- CF.getIncomingState cfga id
state <- CF.getIncomingState (cfgAnalysis params) id
guard $ isNumber state word
guard . not $ fromMaybe False $ CF.variableMayBeDeclaredInteger state var
return $ warn id 2324 "var+=1 will append, not increment. Use (( var += 1 )), declare -i var, or quote number to silence."
@@ -5025,7 +5039,7 @@ checkPlusEqualsNumber params t =
let
unquotedLiteral = getUnquotedLiteral word
isEmpty = unquotedLiteral == Just ""
isUnquotedNumber = not isEmpty && maybe False (all isDigit) unquotedLiteral
isUnquotedNumber = not isEmpty && fromMaybe False (all isDigit <$> unquotedLiteral)
isNumericalVariableName = fromMaybe False $ do
str <- unquotedLiteral
CF.variableMayBeAssignedInteger state str

View File

@@ -31,14 +31,14 @@ import qualified ShellCheck.Checks.ShellSupport
-- TODO: Clean up the cruft this is layered on
analyzeScript :: AnalysisSpec -> AnalysisResult
analyzeScript spec = newAnalysisResult {
arComments =
filterByAnnotation spec params . nub $
runChecker params (checkers spec params)
}
where
params = makeParameters spec
analyzeScript :: Monad m => SystemInterface m -> AnalysisSpec -> m AnalysisResult
analyzeScript sys spec = do
params <- makeParameters sys spec
return $ newAnalysisResult {
arComments =
filterByAnnotation spec params . nub $
runChecker params (checkers spec params)
}
checkers spec params = mconcat $ map ($ params) [
ShellCheck.Analytics.checker spec,

View File

@@ -41,7 +41,6 @@ import Data.Char
import Data.List
import Data.Maybe
import Data.Semigroup
import qualified Data.List.NonEmpty as NE
import qualified Data.Map as Map
import Test.QuickCheck.All (forAllProperties)
@@ -89,6 +88,8 @@ data Parameters = Parameters {
hasSetE :: Bool,
-- Whether this script has 'set -o pipefail' anywhere.
hasPipefail :: Bool,
-- Whether this script is an Ebuild file.
isPortage :: Bool,
-- A linear (bad) analysis of data flow
variableFlow :: [StackData],
-- A map from Id to Token
@@ -104,9 +105,12 @@ data Parameters = Parameters {
-- map from token id to start and end position
tokenPositions :: Map.Map Id (Position, Position),
-- Result from Control Flow Graph analysis (including data flow analysis)
cfgAnalysis :: Maybe CF.CFGAnalysis
cfgAnalysis :: CF.CFGAnalysis,
-- A set of additional variables known to be set (TODO: make this more general?)
additionalKnownVariables :: [String]
} deriving (Show)
-- TODO: Cache results of common AST ops here
data Cache = Cache {}
@@ -153,7 +157,7 @@ producesComments c s = do
let pr = pScript s
prRoot pr
let spec = defaultSpec pr
let params = makeParameters spec
let params = runIdentity $ makeParameters (mockedSystemInterface []) spec
return . not . null $ filterByAnnotation spec params $ runChecker params c
makeComment :: Severity -> Id -> Code -> String -> TokenComment
@@ -197,48 +201,58 @@ makeCommentWithFix severity id code str fix =
}
in force withFix
-- makeParameters :: CheckSpec -> Parameters
makeParameters spec = params
makeParameters :: Monad m => SystemInterface m -> AnalysisSpec -> m Parameters
makeParameters sys spec = do
extraVars <-
if asIsPortage spec
then do
vars <- siGetPortageVariables sys
let classes = getInheritedEclasses root
return $ concatMap (\c -> Map.findWithDefault [] c vars) classes
else
return []
return $ makeParams extraVars
where
extendedAnalysis = fromMaybe True $ msum [asExtendedAnalysis spec, getExtendedAnalysisDirective root]
params = Parameters {
rootNode = root,
shellType = fromMaybe (determineShell (asFallbackShell spec) root) $ asShellType spec,
hasSetE = containsSetE root,
hasLastpipe =
case shellType params of
Bash -> isOptionSet "lastpipe" root
Dash -> False
BusyboxSh -> False
Sh -> False
Ksh -> True,
hasInheritErrexit =
case shellType params of
Bash -> isOptionSet "inherit_errexit" root
Dash -> True
BusyboxSh -> True
Sh -> True
Ksh -> False,
hasPipefail =
case shellType params of
Bash -> isOptionSet "pipefail" root
Dash -> True
BusyboxSh -> isOptionSet "pipefail" root
Sh -> True
Ksh -> isOptionSet "pipefail" root,
shellTypeSpecified = isJust (asShellType spec) || isJust (asFallbackShell spec),
idMap = getTokenMap root,
parentMap = getParentTree root,
variableFlow = getVariableFlow params root,
tokenPositions = asTokenPositions spec,
cfgAnalysis = do
guard extendedAnalysis
return $ CF.analyzeControlFlow cfParams root
}
cfParams = CF.CFGParameters {
CF.cfLastpipe = hasLastpipe params,
CF.cfPipefail = hasPipefail params
}
shell = fromMaybe (determineShell (asFallbackShell spec) root) $ asShellType spec
makeParams extraVars = params
where
params = Parameters {
rootNode = root,
shellType = shell,
hasSetE = containsSetE root,
hasLastpipe =
case shellType params of
Bash -> isOptionSet "lastpipe" root
Dash -> False
Sh -> False
Ksh -> True,
hasInheritErrexit =
case shellType params of
Bash -> isOptionSet "inherit_errexit" root
Dash -> True
Sh -> True
Ksh -> False,
hasPipefail =
case shellType params of
Bash -> isOptionSet "pipefail" root
Dash -> True
Sh -> True
Ksh -> isOptionSet "pipefail" root,
shellTypeSpecified = isJust (asShellType spec) || isJust (asFallbackShell spec),
isPortage = asIsPortage spec,
idMap = getTokenMap root,
parentMap = getParentTree root,
variableFlow = getVariableFlow params root,
tokenPositions = asTokenPositions spec,
cfgAnalysis = CF.analyzeControlFlow cfParams root,
additionalKnownVariables = extraVars
}
cfParams = CF.CFGParameters {
CF.cfLastpipe = hasLastpipe params,
CF.cfPipefail = hasPipefail params,
CF.cfAdditionalInitialVariables = additionalKnownVariables params
}
root = asScript spec
@@ -292,8 +306,8 @@ prop_determineShell7 = determineShellTest "#! /bin/ash" == Dash
prop_determineShell8 = determineShellTest' (Just Ksh) "#!/bin/sh" == Sh
prop_determineShell9 = determineShellTest "#!/bin/env -S dash -x" == Dash
prop_determineShell10 = determineShellTest "#!/bin/env --split-string= dash -x" == Dash
prop_determineShell11 = determineShellTest "#!/bin/busybox sh" == BusyboxSh -- busybox sh is a specific shell, not posix sh
prop_determineShell12 = determineShellTest "#!/bin/busybox ash" == BusyboxSh
prop_determineShell11 = determineShellTest "#!/bin/busybox sh" == Dash -- busybox sh is a specific shell, not posix sh
prop_determineShell12 = determineShellTest "#!/bin/busybox ash" == Dash
determineShellTest = determineShellTest' Nothing
determineShellTest' fallbackShell = determineShell fallbackShell . fromJust . prRoot . pScript
@@ -341,14 +355,14 @@ isQuoteFree = isQuoteFreeNode False
isQuoteFreeNode strict shell tree t =
isQuoteFreeElement t ||
(fromMaybe False $ msum $ map isQuoteFreeContext $ NE.tail $ getPath tree t)
(fromMaybe False $ msum $ map isQuoteFreeContext $ drop 1 $ getPath tree t)
where
-- Is this node self-quoting in itself?
isQuoteFreeElement t =
case t of
T_Assignment id _ _ _ _ -> assignmentIsQuoting id
T_FdRedirect {} -> True
_ -> False
T_Assignment {} -> assignmentIsQuoting t
T_FdRedirect {} -> True
_ -> False
-- Are any subnodes inherently self-quoting?
isQuoteFreeContext t =
@@ -358,7 +372,7 @@ isQuoteFreeNode strict shell tree t =
TC_Binary _ DoubleBracket _ _ _ -> return True
TA_Sequence {} -> return True
T_Arithmetic {} -> return True
T_Assignment id _ _ _ _ -> return $ assignmentIsQuoting id
T_Assignment {} -> return $ assignmentIsQuoting t
T_Redirecting {} -> return False
T_DoubleQuoted _ _ -> return True
T_DollarDoubleQuoted _ _ -> return True
@@ -373,11 +387,11 @@ isQuoteFreeNode strict shell tree t =
-- Check whether this assignment is self-quoting due to being a recognized
-- assignment passed to a Declaration Utility. This will soon be required
-- by POSIX: https://austingroupbugs.net/view.php?id=351
assignmentIsQuoting id = shellParsesParamsAsAssignments || not (isAssignmentParamToCommand id)
assignmentIsQuoting t = shellParsesParamsAsAssignments || not (isAssignmentParamToCommand t)
shellParsesParamsAsAssignments = shell /= Sh
-- Is this assignment a parameter to a command like export/typeset/etc?
isAssignmentParamToCommand id =
isAssignmentParamToCommand (T_Assignment id _ _ _ _) =
case Map.lookup id tree of
Just (T_SimpleCommand _ _ (_:args)) -> id `elem` (map getId args)
_ -> False
@@ -403,7 +417,7 @@ isParamTo tree cmd =
-- Get the parent command (T_Redirecting) of a Token, if any.
getClosestCommand :: Map.Map Id Token -> Token -> Maybe Token
getClosestCommand tree t =
findFirst findCommand $ NE.toList $ getPath tree t
findFirst findCommand $ getPath tree t
where
findCommand t =
case t of
@@ -417,7 +431,7 @@ getClosestCommandM t = do
return $ getClosestCommand (parentMap params) t
-- Is the token used as a command name (the first word in a T_SimpleCommand)?
usedAsCommandName tree token = go (getId token) (NE.tail $ getPath tree token)
usedAsCommandName tree token = go (getId token) (tail $ getPath tree token)
where
go currentId (T_NormalWord id [word]:rest)
| currentId == getId word = go id rest
@@ -434,9 +448,7 @@ getPathM t = do
return $ getPath (parentMap params) t
isParentOf tree parent child =
any (\t -> parentId == getId t) (getPath tree child)
where
parentId = getId parent
elem (getId parent) . map getId $ getPath tree child
parents params = getPath (parentMap params)
@@ -592,6 +604,17 @@ getReferencedVariableCommand base@(T_SimpleCommand _ _ (T_NormalWord _ (T_Litera
head:_ -> map (\x -> (base, head, x)) $ getVariablesFromLiteralToken head
_ -> []
"alias" -> [(base, token, name) | token <- rest, name <- getVariablesFromLiteralToken token]
-- tc-export makes a list of toolchain variables available, similar to export.
-- Usage tc-export CC CXX
"tc-export" -> concatMap getReference rest
-- tc-export_build_env exports the listed variables plus a bunch of BUILD_XX variables.
-- Usage tc-export_build_env BUILD_CC
"tc-export_build_env" ->
concatMap getReference rest
++ [ (base, base, v) | v <- portageBuildEnvVariables ]
_ -> []
where
forDeclare =
@@ -663,6 +686,16 @@ getModifiedVariableCommand base@(T_SimpleCommand id cmdPrefix (T_NormalWord _ (T
"DEFINE_integer" -> maybeToList $ getFlagVariable rest
"DEFINE_string" -> maybeToList $ getFlagVariable rest
"tc-export" -> concatMap getModifierParamString rest
-- tc-export_build_env exports the listed variables plus a bunch of BUILD_XX variables.
-- Usage tc-export_build_env BUILD_CC
"tc-export_build_env" ->
concatMap getModifierParamString rest
++ [ (base, base, var, DataString $ SourceExternal) |
var <- ["BUILD_" ++ x, x ++ "_FOR_BUILD" ],
x <- portageBuildEnvVariables ]
_ -> []
where
flags = map snd $ getAllFlags base
@@ -820,7 +853,7 @@ getReferencedVariables parents t =
return (context, token, getBracedReference str)
isArithmeticAssignment t = case getPath parents t of
this NE.:| TA_Assignment _ "=" lhs _ :_ -> lhs == t
this: TA_Assignment _ "=" lhs _ :_ -> lhs == t
_ -> False
isDereferencingBinaryOp = (`elem` ["-eq", "-ne", "-lt", "-le", "-gt", "-ge"])
@@ -909,7 +942,6 @@ isBashLike params =
Bash -> True
Ksh -> True
Dash -> False
BusyboxSh -> False
Sh -> False
isTrueAssignmentSource c =
@@ -929,6 +961,13 @@ modifiesVariable params token name =
Assignment (_, _, n, source) -> isTrueAssignmentSource source && n == name
_ -> False
-- Ebuild files inherit eclasses using 'inherit myclass1 myclass2'
getInheritedEclasses :: Token -> [String]
getInheritedEclasses root = execWriter $ doAnalysis findInheritedEclasses root
where
findInheritedEclasses cmd
| cmd `isCommand` "inherit" = tell $ catMaybes $ getLiteralString <$> (arguments cmd)
findInheritedEclasses _ = return ()
return []
runTests = $( [| $(forAllProperties) (quickCheckWithResult (stdArgs { maxSuccess = 1 }) ) |])

View File

@@ -51,7 +51,6 @@ import Control.Monad.Identity
import Data.Array.Unboxed
import Data.Array.ST
import Data.List hiding (map)
import qualified Data.List.NonEmpty as NE
import Data.Maybe
import qualified Data.Map as M
import qualified Data.Set as S
@@ -112,8 +111,8 @@ data CFEdge =
-- Actions we track
data CFEffect =
CFSetProps (Maybe Scope) String (S.Set CFVariableProp)
| CFUnsetProps (Maybe Scope) String (S.Set CFVariableProp)
CFSetProps Scope String (S.Set CFVariableProp)
| CFUnsetProps Scope String (S.Set CFVariableProp)
| CFReadVariable String
| CFWriteVariable String CFValue
| CFWriteGlobal String CFValue
@@ -168,7 +167,9 @@ data CFGParameters = CFGParameters {
-- Whether the last element in a pipeline runs in the current shell
cfLastpipe :: Bool,
-- Whether all elements in a pipeline count towards the exit status
cfPipefail :: Bool
cfPipefail :: Bool,
-- Additional variables to consider defined
cfAdditionalInitialVariables :: [String]
}
data CFGResult = CFGResult {
@@ -579,7 +580,7 @@ build t = do
T_Array _ list -> sequentially list
T_Assignment {} -> buildAssignment Nothing t
T_Assignment {} -> buildAssignment DefaultScope t
T_Backgrounded id body -> do
start <- newStructuralNode
@@ -615,15 +616,15 @@ build t = do
T_CaseExpression id t [] -> build t
T_CaseExpression id t list@(hd:tl) -> do
T_CaseExpression id t list -> do
start <- newStructuralNode
token <- build t
branches <- mapM buildBranch (hd NE.:| tl)
branches <- mapM buildBranch list
end <- newStructuralNode
let neighbors = zip (NE.toList branches) $ NE.tail branches
let (_, firstCond, _) = NE.head branches
let (_, lastCond, lastBody) = NE.last branches
let neighbors = zip branches $ tail branches
let (_, firstCond, _) = head branches
let (_, lastCond, lastBody) = last branches
linkRange start token
linkRange token firstCond
@@ -858,8 +859,8 @@ build t = do
status <- newNodeRange (CFSetExitCode id)
linkRange assignments status
T_SimpleCommand id vars (cmd:args) ->
handleCommand t vars (cmd NE.:| args) $ getUnquotedLiteral cmd
T_SimpleCommand id vars list@(cmd:_) ->
handleCommand t vars list $ getUnquotedLiteral cmd
T_SingleQuoted _ _ -> none
@@ -888,9 +889,7 @@ build t = do
T_Less _ -> none
T_ParamSubSpecialChar _ _ -> none
x -> do
error ("Unimplemented: " ++ show x) -- STRIP
none
x -> error ("Unimplemented: " ++ show x)
-- Still in `where` clause
forInHelper id name words body = do
@@ -926,8 +925,8 @@ handleCommand cmd vars args literalCmd = do
-- TODO: Handle assignments in declaring commands
case literalCmd of
Just "exit" -> regularExpansion vars (NE.toList args) $ handleExit
Just "return" -> regularExpansion vars (NE.toList args) $ handleReturn
Just "exit" -> regularExpansion vars args $ handleExit
Just "return" -> regularExpansion vars args $ handleReturn
Just "unset" -> regularExpansionWithStatus vars args $ handleUnset args
Just "declare" -> handleDeclare args
@@ -950,14 +949,14 @@ handleCommand cmd vars args literalCmd = do
-- This will mostly behave like 'command' but ok
Just "builtin" ->
case args of
_ NE.:| [] -> regular
(_ NE.:| newcmd:newargs) ->
handleCommand newcmd vars (newcmd NE.:| newargs) $ getLiteralString newcmd
[_] -> regular
(_:newargs@(newcmd:_)) ->
handleCommand newcmd vars newargs $ getLiteralString newcmd
Just "command" ->
case args of
_ NE.:| [] -> regular
(_ NE.:| newcmd:newargs) ->
handleOthers (getId newcmd) vars (newcmd NE.:| newargs) $ getLiteralString newcmd
[_] -> regular
(_:newargs@(newcmd:_)) ->
handleOthers (getId newcmd) vars newargs $ getLiteralString newcmd
_ -> regular
where
@@ -985,7 +984,7 @@ handleCommand cmd vars args literalCmd = do
unreachable <- newNode CFUnreachable
return $ Range ret unreachable
handleUnset (cmd NE.:| args) = do
handleUnset (cmd:args) = do
case () of
_ | "n" `elem` flagNames -> unsetWith CFUndefineNameref
_ | "v" `elem` flagNames -> unsetWith CFUndefineVariable
@@ -997,14 +996,14 @@ handleCommand cmd vars args literalCmd = do
(names, flags) = partition (null . fst) pairs
flagNames = map fst flags
literalNames :: [(Token, String)] -- Literal names to unset, e.g. [(myfuncToken, "myfunc")]
literalNames = mapMaybe (\(_, t) -> (,) t <$> getLiteralString t) names
literalNames = mapMaybe (\(_, t) -> getLiteralString t >>= (return . (,) t)) names
-- Apply a constructor like CFUndefineVariable to each literalName, and tag with its id
unsetWith c = newNodeRange $ CFApplyEffects $ map (\(token, name) -> IdTagged (getId token) $ c name) literalNames
variableAssignRegex = mkRegex "^([_a-zA-Z][_a-zA-Z0-9]*)="
handleDeclare (cmd NE.:| args) = do
handleDeclare (cmd:args) = do
isFunc <- asks cfIsFunction
-- This is a bit of a kludge: we don't have great support for things like
-- 'declare -i x=$x' so do one round with declare x=$x, followed by declare -i x
@@ -1031,9 +1030,9 @@ handleCommand cmd vars args literalCmd = do
scope isFunc =
case () of
_ | global -> Just GlobalScope
_ | isFunc -> Just LocalScope
_ -> Nothing
_ | global -> GlobalScope
_ | isFunc -> LocalScope
_ -> DefaultScope
addedProps = S.fromList $ concat $ [
[ CFVPArray | array ],
@@ -1061,7 +1060,7 @@ handleCommand cmd vars args literalCmd = do
let
id = getId t
pre = [t]
literal = getLiteralStringDef "\0" t
literal = fromJust $ getLiteralStringExt (const $ Just "\0") t
isKnown = '\0' `notElem` literal
match = fmap head $ variableAssignRegex `matchRegex` literal
name = fromMaybe literal match
@@ -1093,7 +1092,7 @@ handleCommand cmd vars args literalCmd = do
in
concatMap (drop 1) plusses
handlePrintf (cmd NE.:| args) =
handlePrintf (cmd:args) =
newNodeRange $ CFApplyEffects $ maybeToList findVar
where
findVar = do
@@ -1102,7 +1101,7 @@ handleCommand cmd vars args literalCmd = do
name <- getLiteralString arg
return $ IdTagged (getId arg) $ CFWriteVariable name CFValueString
handleWait (cmd NE.:| args) =
handleWait (cmd:args) =
newNodeRange $ CFApplyEffects $ maybeToList findVar
where
findVar = do
@@ -1111,7 +1110,7 @@ handleCommand cmd vars args literalCmd = do
name <- getLiteralString arg
return $ IdTagged (getId arg) $ CFWriteVariable name CFValueInteger
handleMapfile (cmd NE.:| args) =
handleMapfile (cmd:args) =
newNodeRange $ CFApplyEffects [findVar]
where
findVar =
@@ -1131,7 +1130,7 @@ handleCommand cmd vars args literalCmd = do
guard $ isVariableName name
return (getId c, name)
handleRead (cmd NE.:| args) = newNodeRange $ CFApplyEffects main
handleRead (cmd:args) = newNodeRange $ CFApplyEffects main
where
main = fromMaybe fallback $ do
flags <- getGnuOpts flagsForRead args
@@ -1161,7 +1160,7 @@ handleCommand cmd vars args literalCmd = do
in
map (\(id, name) -> IdTagged id $ CFWriteVariable name value) namesOrDefault
handleDEFINE (cmd NE.:| args) =
handleDEFINE (cmd:args) =
newNodeRange $ CFApplyEffects $ maybeToList findVar
where
findVar = do
@@ -1171,14 +1170,14 @@ handleCommand cmd vars args literalCmd = do
return $ IdTagged (getId name) $ CFWriteVariable str CFValueString
handleOthers id vars args cmd =
regularExpansion vars (NE.toList args) $ do
regularExpansion vars args $ do
exe <- newNodeRange $ CFExecuteCommand cmd
status <- newNodeRange $ CFSetExitCode id
linkRange exe status
regularExpansion vars args p = do
args <- sequentially args
assignments <- mapM (buildAssignment (Just PrefixScope)) vars
assignments <- mapM (buildAssignment PrefixScope) vars
exe <- p
dropAssignments <-
if null vars
@@ -1190,15 +1189,15 @@ handleCommand cmd vars args literalCmd = do
linkRanges $ [args] ++ assignments ++ [exe] ++ dropAssignments
regularExpansionWithStatus vars args@(cmd NE.:| _) p = do
initial <- regularExpansion vars (NE.toList args) p
regularExpansionWithStatus vars args@(cmd:_) p = do
initial <- regularExpansion vars args p
status <- newNodeRange $ CFSetExitCode (getId cmd)
linkRange initial status
none = newStructuralNode
data Scope = GlobalScope | LocalScope | PrefixScope
data Scope = DefaultScope | GlobalScope | LocalScope | PrefixScope
deriving (Eq, Ord, Show, Generic, NFData)
buildAssignment scope t = do
@@ -1212,10 +1211,10 @@ buildAssignment scope t = do
let valueType = if null indices then f id value else CFValueArray
let scoper =
case scope of
Just PrefixScope -> CFWritePrefix
Just LocalScope -> CFWriteLocal
Just GlobalScope -> CFWriteGlobal
Nothing -> CFWriteVariable
PrefixScope -> CFWritePrefix
LocalScope -> CFWriteLocal
GlobalScope -> CFWriteGlobal
DefaultScope -> CFWriteVariable
write <- newNodeRange $ applySingle $ IdTagged id $ scoper var valueType
linkRanges [expand, index, read, write]
where

View File

@@ -197,12 +197,13 @@ unreachableState = modified newInternalState {
}
-- The default state we assume we get from the environment
createEnvironmentState :: InternalState
createEnvironmentState = do
createEnvironmentState :: CFGParameters -> InternalState
createEnvironmentState params = do
foldl' (flip ($)) newInternalState $ concat [
addVars Data.internalVariables unknownVariableState,
addVars Data.variablesWithoutSpaces spacelessVariableState,
addVars Data.specialIntegerVariables integerVariableState
addVars Data.specialIntegerVariables integerVariableState,
addVars (cfAdditionalInitialVariables params) unknownVariableState
]
where
addVars names val = map (\name -> insertGlobal name val) names
@@ -299,6 +300,7 @@ depsToState set = foldl insert newInternalState $ S.toList set
PrefixScope -> (sPrefixValues, insertPrefix)
LocalScope -> (sLocalValues, insertLocal)
GlobalScope -> (sGlobalValues, insertGlobal)
DefaultScope -> error $ pleaseReport "Unresolved scope in dependency"
alreadyExists = isJust $ vmLookup name $ mapToCheck state
in
@@ -829,7 +831,7 @@ lookupStack' functionOnly get dep def ctx key = do
f (s:rest) = do
-- Go up the stack until we find the value, and add
-- a dependency on each state (including where it was found)
res <- maybe (f rest) return (get (stackState s) key)
res <- fromMaybe (f rest) (return <$> get (stackState s) key)
modifySTRef (dependencies s) $ S.insert $ dep key res
return res
@@ -1119,34 +1121,34 @@ transferEffect ctx effect =
CFSetProps scope name props ->
case scope of
Nothing -> do
DefaultScope -> do
state <- readVariable ctx name
writeVariable ctx name $ addProperties props state
Just GlobalScope -> do
GlobalScope -> do
state <- readGlobal ctx name
writeGlobal ctx name $ addProperties props state
Just LocalScope -> do
LocalScope -> do
out <- readSTRef (cOutput ctx)
state <- readLocal ctx name
writeLocal ctx name $ addProperties props state
Just PrefixScope -> do
PrefixScope -> do
-- Prefix values become local
state <- readLocal ctx name
writeLocal ctx name $ addProperties props state
CFUnsetProps scope name props ->
case scope of
Nothing -> do
DefaultScope -> do
state <- readVariable ctx name
writeVariable ctx name $ removeProperties props state
Just GlobalScope -> do
GlobalScope -> do
state <- readGlobal ctx name
writeGlobal ctx name $ removeProperties props state
Just LocalScope -> do
LocalScope -> do
out <- readSTRef (cOutput ctx)
state <- readLocal ctx name
writeLocal ctx name $ removeProperties props state
Just PrefixScope -> do
PrefixScope -> do
-- Prefix values become local
state <- readLocal ctx name
writeLocal ctx name $ removeProperties props state
@@ -1343,7 +1345,7 @@ analyzeControlFlow params t =
runST $ f cfg entry exit
where
f cfg entry exit = do
let env = createEnvironmentState
let env = createEnvironmentState params
ctx <- newCtx $ cfGraph cfg
-- Do a dataflow analysis starting on the root node
exitState <- runRoot ctx env entry exit

View File

@@ -25,7 +25,7 @@ import ShellCheck.ASTLib
import ShellCheck.Interface
import ShellCheck.Parser
import Debug.Trace -- DO NOT SUBMIT
import Data.Char
import Data.Either
import Data.Functor
import Data.List
@@ -55,6 +55,8 @@ shellFromFilename filename = listToMaybe candidates
shellExtensions = [(".ksh", Ksh)
,(".bash", Bash)
,(".bats", Bash)
,(".ebuild", Bash)
,(".eclass", Bash)
,(".dash", Dash)]
-- The `.sh` is too generic to determine the shell:
-- We fallback to Bash in this case and emit SC2148 if there is no shebang
@@ -85,19 +87,24 @@ checkScript sys spec = do
asShellType = csShellTypeOverride spec,
asFallbackShell = shellFromFilename $ csFilename spec,
asCheckSourced = csCheckSourced spec,
asIsPortage = isPortage $ csFilename spec,
asExecutionMode = Executed,
asTokenPositions = tokenPositions,
asExtendedAnalysis = csExtendedAnalysis spec,
asOptionalChecks = getEnableDirectives root ++ csOptionalChecks spec
} where as = newAnalysisSpec root
let analysisMessages =
maybe []
(arComments . analyzeScript . analysisSpec)
$ prRoot result
let getAnalysisMessages =
case prRoot result of
Just root -> arComments <$> (analyzeScript sys $ analysisSpec root)
Nothing -> return []
let translator = tokenToPosition tokenPositions
analysisMessages <- getAnalysisMessages
return . nub . sortMessages . filter shouldInclude $
(parseMessages ++ map translator analysisMessages)
isPortage filename =
let f = map toLower filename in
".ebuild" `isSuffixOf` f || ".eclass" `isSuffixOf` f
shouldInclude pc =
severity <= csMinSeverity spec &&
case csIncludedWarnings spec of
@@ -518,47 +525,6 @@ prop_hereDocsAreParsedWithoutTrailingLinefeed = 1044 `elem` result
where
result = check "cat << eof"
prop_hereDocsWillHaveParsedIndices = null result
where
result = check "#!/bin/bash\nmy_array=(a b)\ncat <<EOF >> ./test\n $(( 1 + my_array[1] ))\nEOF"
prop_rcCanSuppressDfa = null result
where
result = checkWithRc "extended-analysis=false" emptyCheckSpec {
csScript = "#!/bin/sh\nexit; foo;"
}
prop_fileCanSuppressDfa = null $ traceShowId result
where
result = checkWithRc "" emptyCheckSpec {
csScript = "#!/bin/sh\n# shellcheck extended-analysis=false\nexit; foo;"
}
prop_fileWinsWhenSuppressingDfa1 = null result
where
result = checkWithRc "extended-analysis=true" emptyCheckSpec {
csScript = "#!/bin/sh\n# shellcheck extended-analysis=false\nexit; foo;"
}
prop_fileWinsWhenSuppressingDfa2 = result == [2317]
where
result = checkWithRc "extended-analysis=false" emptyCheckSpec {
csScript = "#!/bin/sh\n# shellcheck extended-analysis=true\nexit; foo;"
}
prop_flagWinsWhenSuppressingDfa1 = result == [2317]
where
result = checkWithRc "extended-analysis=false" emptyCheckSpec {
csScript = "#!/bin/sh\n# shellcheck extended-analysis=false\nexit; foo;",
csExtendedAnalysis = Just True
}
prop_flagWinsWhenSuppressingDfa2 = null result
where
result = checkWithRc "extended-analysis=true" emptyCheckSpec {
csScript = "#!/bin/sh\n# shellcheck extended-analysis=true\nexit; foo;",
csExtendedAnalysis = Just False
}
return []
runTests = $quickCheckAll

View File

@@ -20,7 +20,6 @@
{-# LANGUAGE TemplateHaskell #-}
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE MultiWayIf #-}
{-# LANGUAGE PatternGuards #-}
-- This module contains checks that examine specific commands by name.
module ShellCheck.Checks.Commands (checker, optionalChecks, ShellCheck.Checks.Commands.runTests) where
@@ -43,7 +42,6 @@ import Data.Functor.Identity
import qualified Data.Graph.Inductive.Graph as G
import Data.List
import Data.Maybe
import qualified Data.List.NonEmpty as NE
import qualified Data.Map.Strict as M
import qualified Data.Set as S
import Test.QuickCheck.All (forAllProperties)
@@ -183,15 +181,16 @@ checkCommand :: M.Map CommandName (Token -> Analysis) -> Token -> Analysis
checkCommand map t@(T_SimpleCommand id cmdPrefix (cmd:rest)) = sequence_ $ do
name <- getLiteralString cmd
return $
if | '/' `elem` name ->
M.findWithDefault nullCheck (Basename $ basename name) map t
| name == "builtin", (h:_) <- rest ->
let t' = T_SimpleCommand id cmdPrefix rest
selectedBuiltin = onlyLiteralString h
in M.findWithDefault nullCheck (Exactly selectedBuiltin) map t'
| otherwise -> do
M.findWithDefault nullCheck (Exactly name) map t
M.findWithDefault nullCheck (Basename name) map t
if '/' `elem` name
then
M.findWithDefault nullCheck (Basename $ basename name) map t
else if name == "builtin" && not (null rest) then
let t' = T_SimpleCommand id cmdPrefix rest
selectedBuiltin = fromMaybe "" $ getLiteralString . head $ rest
in M.findWithDefault nullCheck (Exactly selectedBuiltin) map t'
else do
M.findWithDefault nullCheck (Exactly name) map t
M.findWithDefault nullCheck (Basename name) map t
where
basename = reverse . takeWhile (/= '/') . reverse
@@ -300,7 +299,7 @@ checkExpr = CommandCheck (Basename "expr") f where
"'expr' expects 3+ arguments but sees 1. Make sure each operator/operand is a separate argument, and escape <>&|."
[first, second] |
onlyLiteralString first /= "length"
(fromMaybe "" $ getLiteralString first) /= "length"
&& not (willSplit first || willSplit second) -> do
checkOp first
warn (getId t) 2307
@@ -931,7 +930,7 @@ prop_checkTimedCommand2 = verify checkTimedCommand "#!/bin/dash\ntime ( foo; bar
prop_checkTimedCommand3 = verifyNot checkTimedCommand "#!/bin/sh\ntime sleep 1"
checkTimedCommand = CommandCheck (Exactly "time") f where
f (T_SimpleCommand _ _ (c:args@(_:_))) =
whenShell [Sh, Dash, BusyboxSh] $ do
whenShell [Sh, Dash] $ do
let cmd = last args -- "time" is parsed with a command as argument
when (isPiped cmd) $
warn (getId c) 2176 "'time' is undefined for pipelines. time single stage or bash -c instead."
@@ -955,7 +954,7 @@ checkTimedCommand = CommandCheck (Exactly "time") f where
prop_checkLocalScope1 = verify checkLocalScope "local foo=3"
prop_checkLocalScope2 = verifyNot checkLocalScope "f() { local foo=3; }"
checkLocalScope = CommandCheck (Exactly "local") $ \t ->
whenShell [Bash, Dash, BusyboxSh] $ do -- Ksh allows it, Sh doesn't support local
whenShell [Bash, Dash] $ do -- Ksh allows it, Sh doesn't support local
path <- getPathM t
unless (any isFunctionLike path) $
err (getId $ getCommandTokenOrThis t) 2168 "'local' is only valid in functions."
@@ -1006,8 +1005,8 @@ checkWhileGetoptsCase = CommandCheck (Exactly "getopts") f
sequence_ $ do
options <- getLiteralString arg1
getoptsVar <- getLiteralString name
(T_WhileExpression _ _ body) <- findFirst whileLoop (NE.toList path)
T_CaseExpression id var list <- mapMaybe findCase body !!! 0
(T_WhileExpression _ _ body) <- findFirst whileLoop path
caseCmd@(T_CaseExpression _ var _) <- mapMaybe findCase body !!! 0
-- Make sure getopts name and case variable matches
[T_DollarBraced _ _ bracedWord] <- return $ getWordParts var
@@ -1017,11 +1016,11 @@ checkWhileGetoptsCase = CommandCheck (Exactly "getopts") f
-- Make sure the variable isn't modified
guard . not $ modifiesVariable params (T_BraceGroup (Id 0) body) getoptsVar
return $ check (getId arg1) (map (:[]) $ filter (/= ':') options) id list
return $ check (getId arg1) (map (:[]) $ filter (/= ':') options) caseCmd
f _ = return ()
check :: Id -> [String] -> Id -> [(CaseType, [Token], [Token])] -> Analysis
check optId opts id list = do
check :: Id -> [String] -> Token -> Analysis
check optId opts (T_CaseExpression id _ list) = do
unless (Nothing `M.member` handledMap) $ do
mapM_ (warnUnhandled optId id) $ catMaybes $ M.keys notHandled
@@ -1237,7 +1236,8 @@ checkSudoArgs = CommandCheck (Basename "sudo") f
where
f t = sequence_ $ do
opts <- parseOpts $ arguments t
(_,(commandArg, _)) <- find (null . fst) opts
let nonFlags = [x | ("",(x, _)) <- opts]
commandArg <- nonFlags !!! 0
command <- getLiteralString commandArg
guard $ command `elem` builtins
return $ warn (getId t) 2232 $ "Can't use sudo with builtins like " ++ command ++ ". Did you want sudo sh -c .. instead?"
@@ -1430,28 +1430,26 @@ prop_checkBackreferencingDeclaration6 = verify (checkBackreferencingDeclaration
prop_checkBackreferencingDeclaration7 = verify (checkBackreferencingDeclaration "declare") "declare x=var $k=$x"
checkBackreferencingDeclaration cmd = CommandCheck (Exactly cmd) check
where
check t = do
cfga <- asks cfgAnalysis
when (isJust cfga) $
foldM_ (perArg $ fromJust cfga) M.empty $ arguments t
check t = foldM_ perArg M.empty $ arguments t
perArg cfga leftArgs t =
perArg leftArgs t =
case t of
T_Assignment id _ name idx t -> do
warnIfBackreferencing cfga leftArgs $ t:idx
warnIfBackreferencing leftArgs $ t:idx
return $ M.insert name id leftArgs
t -> do
warnIfBackreferencing cfga leftArgs [t]
warnIfBackreferencing leftArgs [t]
return leftArgs
warnIfBackreferencing cfga backrefs l = do
references <- findReferences cfga l
warnIfBackreferencing backrefs l = do
references <- findReferences l
let reused = M.intersection backrefs references
mapM msg $ M.toList reused
msg (name, id) = warn id 2318 $ "This assignment is used again in this '" ++ cmd ++ "', but won't have taken effect. Use two '" ++ cmd ++ "'s."
findReferences cfga list = do
findReferences list = do
cfga <- asks cfgAnalysis
let graph = CF.graph cfga
let nodesMap = CF.tokenToNodes cfga
let nodes = S.unions $ map (\id -> M.findWithDefault S.empty id nodesMap) $ map getId $ list

View File

@@ -78,7 +78,7 @@ controlFlowEffectChecks = [
runNodeChecks :: [ControlFlowNodeCheck] -> ControlFlowCheck
runNodeChecks perNode = do
cfg <- asks cfgAnalysis
sequence_ $ runOnAll <$> cfg
runOnAll cfg
where
getData datas n@(node, label) = do
(pre, post) <- M.lookup node datas

View File

@@ -19,7 +19,6 @@
-}
{-# LANGUAGE TemplateHaskell #-}
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE ViewPatterns #-}
module ShellCheck.Checks.ShellSupport (checker , ShellCheck.Checks.ShellSupport.runTests) where
import ShellCheck.AST
@@ -76,11 +75,12 @@ verifyNot c s = producesComments (testChecker c) s == Just False
prop_checkForDecimals1 = verify checkForDecimals "((3.14*c))"
prop_checkForDecimals2 = verify checkForDecimals "foo[1.2]=bar"
prop_checkForDecimals3 = verifyNot checkForDecimals "declare -A foo; foo[1.2]=bar"
checkForDecimals = ForShell [Sh, Dash, BusyboxSh, Bash] f
checkForDecimals = ForShell [Sh, Dash, Bash] f
where
f t@(TA_Expansion id _) = sequence_ $ do
first:rest <- getLiteralString t
guard $ isDigit first && '.' `elem` rest
str <- getLiteralString t
first <- str !!! 0
guard $ isDigit first && '.' `elem` str
return $ err id 2079 "(( )) doesn't support decimals. Use bc or awk."
f _ = return ()
@@ -91,9 +91,6 @@ prop_checkBashisms3 = verify checkBashisms "echo $((i++))"
prop_checkBashisms4 = verify checkBashisms "rm !(*.hs)"
prop_checkBashisms5 = verify checkBashisms "source file"
prop_checkBashisms6 = verify checkBashisms "[ \"$a\" == 42 ]"
prop_checkBashisms6b = verify checkBashisms "test \"$a\" == 42"
prop_checkBashisms6c = verify checkBashisms "[ foo =~ bar ]"
prop_checkBashisms6d = verify checkBashisms "test foo =~ bar"
prop_checkBashisms7 = verify checkBashisms "echo ${var[1]}"
prop_checkBashisms8 = verify checkBashisms "echo ${!var[@]}"
prop_checkBashisms9 = verify checkBashisms "echo ${!var*}"
@@ -109,7 +106,6 @@ prop_checkBashisms18 = verify checkBashisms "foo &> /dev/null"
prop_checkBashisms19 = verify checkBashisms "foo > file*.txt"
prop_checkBashisms20 = verify checkBashisms "read -ra foo"
prop_checkBashisms21 = verify checkBashisms "[ -a foo ]"
prop_checkBashisms21b = verify checkBashisms "test -a foo"
prop_checkBashisms22 = verifyNot checkBashisms "[ foo -a bar ]"
prop_checkBashisms23 = verify checkBashisms "trap mything ERR INT"
prop_checkBashisms24 = verifyNot checkBashisms "trap mything INT TERM"
@@ -195,37 +191,18 @@ prop_checkBashisms101 = verify checkBashisms "read"
prop_checkBashisms102 = verifyNot checkBashisms "read -r foo"
prop_checkBashisms103 = verifyNot checkBashisms "read foo"
prop_checkBashisms104 = verifyNot checkBashisms "read ''"
prop_checkBashisms105 = verifyNot checkBashisms "#!/bin/busybox sh\nset -o pipefail"
prop_checkBashisms106 = verifyNot checkBashisms "#!/bin/busybox sh\nx=x\n[[ \"$x\" = \"$x\" ]]"
prop_checkBashisms107 = verifyNot checkBashisms "#!/bin/busybox sh\nx=x\n[ \"$x\" == \"$x\" ]"
prop_checkBashisms108 = verifyNot checkBashisms "#!/bin/busybox sh\necho magic &> /dev/null"
prop_checkBashisms109 = verifyNot checkBashisms "#!/bin/busybox sh\ntrap stop EXIT SIGTERM"
prop_checkBashisms110 = verifyNot checkBashisms "#!/bin/busybox sh\nsource /dev/null"
prop_checkBashisms111 = verify checkBashisms "#!/bin/dash\nx='test'\n${x:0:3}" -- SC3057
prop_checkBashisms112 = verifyNot checkBashisms "#!/bin/busybox sh\nx='test'\n${x:0:3}" -- SC3057
prop_checkBashisms113 = verify checkBashisms "#!/bin/dash\nx='test'\n${x/st/xt}" -- SC3060
prop_checkBashisms114 = verifyNot checkBashisms "#!/bin/busybox sh\nx='test'\n${x/st/xt}" -- SC3060
prop_checkBashisms115 = verify checkBashisms "#!/bin/busybox sh\nx='test'\n${!x}" -- SC3053
prop_checkBashisms116 = verify checkBashisms "#!/bin/busybox sh\nx='test'\n${x[1]}" -- SC3054
prop_checkBashisms117 = verify checkBashisms "#!/bin/busybox sh\nx='test'\n${!x[@]}" -- SC3055
prop_checkBashisms118 = verify checkBashisms "#!/bin/busybox sh\nxyz=1\n${!x*}" -- SC3056
prop_checkBashisms119 = verify checkBashisms "#!/bin/busybox sh\nx='test'\n${x^^[t]}" -- SC3059
prop_checkBashisms120 = verify checkBashisms "#!/bin/sh\n[ x == y ]"
prop_checkBashisms121 = verifyNot checkBashisms "#!/bin/sh\n# shellcheck shell=busybox\n[ x == y ]"
checkBashisms = ForShell [Sh, Dash, BusyboxSh] $ \t -> do
checkBashisms = ForShell [Sh, Dash] $ \t -> do
params <- ask
kludge params t
where
-- This code was copy-pasted from Analytics where params was a variable
kludge params = bashism
where
isBusyboxSh = shellType params == BusyboxSh
isDash = shellType params == Dash || isBusyboxSh
isDash = shellType params == Dash
warnMsg id code s =
if isDash
then err id code $ "In dash, " ++ s ++ " not supported."
else warn id code $ "In POSIX sh, " ++ s ++ " undefined."
asStr = getLiteralString
bashism (T_ProcSub id _ _) = warnMsg id 3001 "process substitution is"
bashism (T_Extglob id _ _) = warnMsg id 3002 "extglob is"
@@ -236,43 +213,27 @@ checkBashisms = ForShell [Sh, Dash, BusyboxSh] $ \t -> do
bashism (T_DollarBracket id _) = warnMsg id 3007 "$[..] in place of $((..)) is"
bashism (T_SelectIn id _ _ _) = warnMsg id 3008 "select loops are"
bashism (T_BraceExpansion id _) = warnMsg id 3009 "brace expansion is"
bashism (T_Condition id DoubleBracket _) =
unless isBusyboxSh $ warnMsg id 3010 "[[ ]] is"
bashism (T_Condition id DoubleBracket _) = warnMsg id 3010 "[[ ]] is"
bashism (T_HereString id _) = warnMsg id 3011 "here-strings are"
bashism (TC_Binary id SingleBracket op _ _)
| op `elem` [ "<", ">", "\\<", "\\>", "<=", ">=", "\\<=", "\\>="] =
unless isDash $ warnMsg id 3012 $ "lexicographical " ++ op ++ " is"
bashism (T_SimpleCommand id _ [asStr -> Just "test", lhs, asStr -> Just op, rhs])
| op `elem` [ "<", ">", "\\<", "\\>", "<=", ">=", "\\<=", "\\>="] =
unless isDash $ warnMsg id 3012 $ "lexicographical " ++ op ++ " is"
bashism (TC_Binary id SingleBracket op _ _)
| op `elem` [ "-ot", "-nt", "-ef" ] =
unless isDash $ warnMsg id 3013 $ op ++ " is"
bashism (T_SimpleCommand id _ [asStr -> Just "test", lhs, asStr -> Just op, rhs])
| op `elem` [ "-ot", "-nt", "-ef" ] =
unless isDash $ warnMsg id 3013 $ op ++ " is"
bashism (TC_Binary id SingleBracket "==" _ _) =
unless isBusyboxSh $ warnMsg id 3014 "== in place of = is"
bashism (T_SimpleCommand id _ [asStr -> Just "test", lhs, asStr -> Just "==", rhs]) =
unless isBusyboxSh $ warnMsg id 3014 "== in place of = is"
warnMsg id 3014 "== in place of = is"
bashism (TC_Binary id SingleBracket "=~" _ _) =
warnMsg id 3015 "=~ regex matching is"
bashism (T_SimpleCommand id _ [asStr -> Just "test", lhs, asStr -> Just "=~", rhs]) =
warnMsg id 3015 "=~ regex matching is"
bashism (TC_Unary id SingleBracket "-v" _) =
warnMsg id 3016 "unary -v (in place of [ -n \"${var+x}\" ]) is"
bashism (T_SimpleCommand id _ [asStr -> Just "test", asStr -> Just "-v", _]) =
warnMsg id 3016 "unary -v (in place of [ -n \"${var+x}\" ]) is"
bashism (TC_Unary id _ "-a" _) =
warnMsg id 3017 "unary -a in place of -e is"
bashism (T_SimpleCommand id _ [asStr -> Just "test", asStr -> Just "-a", _]) =
warnMsg id 3017 "unary -a in place of -e is"
bashism (TA_Unary id op _)
| op `elem` [ "|++", "|--", "++|", "--|"] =
warnMsg id 3018 $ filter (/= '|') op ++ " is"
bashism (TA_Binary id "**" _ _) = warnMsg id 3019 "exponentials are"
bashism (T_FdRedirect id "&" (T_IoFile _ (T_Greater _) _)) =
unless isBusyboxSh $ warnMsg id 3020 "&> is"
bashism (T_FdRedirect id "&" (T_IoFile _ (T_Greater _) _)) = warnMsg id 3020 "&> is"
bashism (T_FdRedirect id "" (T_IoFile _ (T_GREATAND _) file)) =
unless (all isDigit $ onlyLiteralString file) $ warnMsg id 3021 ">& filename (as opposed to >& fd) is"
bashism (T_FdRedirect id ('{':_) _) = warnMsg id 3022 "named file descriptors are"
@@ -292,8 +253,7 @@ checkBashisms = ForShell [Sh, Dash, BusyboxSh] $ \t -> do
warnMsg id 3028 $ str ++ " is"
bashism t@(T_DollarBraced id _ token) = do
unless isBusyboxSh $ mapM_ check simpleExpansions
mapM_ check advancedExpansions
mapM_ check expansion
when (isBashVariable var) $
warnMsg id 3028 $ var ++ " is"
where
@@ -403,8 +363,7 @@ checkBashisms = ForShell [Sh, Dash, BusyboxSh] $ \t -> do
(\x -> (not . null . snd $ x) && snd x `notElem` allowed) flags
return . warnMsg (getId word) 3045 $ name ++ " -" ++ flag ++ " is"
when (name == "source" && not isBusyboxSh) $
warnMsg id 3046 "'source' in place of '.' is"
when (name == "source") $ warnMsg id 3046 "'source' in place of '.' is"
when (name == "trap") $
let
check token = sequence_ $ do
@@ -413,7 +372,7 @@ checkBashisms = ForShell [Sh, Dash, BusyboxSh] $ \t -> do
return $ do
when (upper `elem` ["ERR", "DEBUG", "RETURN"]) $
warnMsg (getId token) 3047 $ "trapping " ++ str ++ " is"
when (not isBusyboxSh && "SIG" `isPrefixOf` upper) $
when ("SIG" `isPrefixOf` upper) $
warnMsg (getId token) 3048
"prefixing signal names with 'SIG' is"
when (not isDash && upper /= str) $
@@ -453,9 +412,7 @@ checkBashisms = ForShell [Sh, Dash, BusyboxSh] $ \t -> do
("wait", Just [])
]
bashism t@(T_SourceCommand id src _)
| getCommandName src == Just "source" =
unless isBusyboxSh $
warnMsg id 3051 "'source' in place of '.' is"
| getCommandName src == Just "source" = warnMsg id 3051 "'source' in place of '.' is"
bashism (TA_Expansion _ (T_Literal id str : _))
| str `matches` radix = warnMsg id 3052 "arithmetic base conversion is"
where
@@ -463,16 +420,14 @@ checkBashisms = ForShell [Sh, Dash, BusyboxSh] $ \t -> do
bashism _ = return ()
varChars="_0-9a-zA-Z"
advancedExpansions = let re = mkRegex in [
expansion = let re = mkRegex in [
(re $ "^![" ++ varChars ++ "]", 3053, "indirect expansion is"),
(re $ "^[" ++ varChars ++ "]+\\[.*\\]$", 3054, "array references are"),
(re $ "^![" ++ varChars ++ "]+\\[[*@]]$", 3055, "array key expansion is"),
(re $ "^![" ++ varChars ++ "]+[*@]$", 3056, "name matching prefixes are"),
(re $ "^[" ++ varChars ++ "*@]+(\\[.*\\])?[,^]", 3059, "case modification is")
]
simpleExpansions = let re = mkRegex in [
(re $ "^[" ++ varChars ++ "*@]+:[^-=?+]", 3057, "string indexing is"),
(re $ "^([*@][%#]|#[@*])", 3058, "string operations on $@/$* are"),
(re $ "^[" ++ varChars ++ "*@]+(\\[.*\\])?[,^]", 3059, "case modification is"),
(re $ "^[" ++ varChars ++ "*@]+(\\[.*\\])?/", 3060, "string replacement is")
]
bashVars = [
@@ -615,7 +570,7 @@ checkPS1Assignments = ForShell [Bash] f
prop_checkMultipleBangs1 = verify checkMultipleBangs "! ! true"
prop_checkMultipleBangs2 = verifyNot checkMultipleBangs "! true"
checkMultipleBangs = ForShell [Dash, BusyboxSh, Sh] f
checkMultipleBangs = ForShell [Dash, Sh] f
where
f token = case token of
T_Banged id (T_Banged _ _) ->
@@ -626,7 +581,7 @@ checkMultipleBangs = ForShell [Dash, BusyboxSh, Sh] f
prop_checkBangAfterPipe1 = verify checkBangAfterPipe "true | ! true"
prop_checkBangAfterPipe2 = verifyNot checkBangAfterPipe "true | ( ! true )"
prop_checkBangAfterPipe3 = verifyNot checkBangAfterPipe "! ! true | true"
checkBangAfterPipe = ForShell [Dash, BusyboxSh, Sh, Bash] f
checkBangAfterPipe = ForShell [Dash, Sh, Bash] f
where
f token = case token of
T_Pipeline _ _ cmds -> mapM_ check cmds

View File

@@ -62,8 +62,92 @@ internalVariables = [
, "FLAGS_ARGC", "FLAGS_ARGV", "FLAGS_ERROR", "FLAGS_FALSE", "FLAGS_HELP",
"FLAGS_PARENT", "FLAGS_RESERVED", "FLAGS_TRUE", "FLAGS_VERSION",
"flags_error", "flags_return"
] ++ portageManualInternalVariables
portageManualInternalVariables = [
-- toolchain settings
"CFLAGS", "CXXFLAGS", "CPPFLAGS", "LDFLAGS", "FFLAGS", "FCFLAGS",
"CBUILD", "CHOST", "MAKEOPTS",
-- TODO: Delete these if we can handle `tc-export CC` implicit export.
"CC", "CPP", "CXX",
-- portage internals
"EBUILD_PHASE", "EBUILD_SH_ARGS", "EMERGE_FROM", "FILESDIR",
"MERGE_TYPE", "PM_EBUILD_HOOK_DIR", "PORTAGE_ACTUAL_DISTDIR",
"PORTAGE_ARCHLIST", "PORTAGE_BASHRC", "PORTAGE_BINPKG_FILE",
"PORTAGE_BINPKG_TAR_OPTS", "PORTAGE_BINPKG_TMPFILE", "PORTAGE_BIN_PATH",
"PORTAGE_BUILDDIR", "PORTAGE_BUILD_GROUP", "PORTAGE_BUILD_USER",
"PORTAGE_BUNZIP2_COMMAND", "PORTAGE_BZIP2_COMMAND", "PORTAGE_COLORMAP",
"PORTAGE_CONFIGROOT", "PORTAGE_DEBUG", "PORTAGE_DEPCACHEDIR",
"PORTAGE_EBUILD_EXIT_FILE", "PORTAGE_ECLASS_LOCATIONS", "PORTAGE_GID",
"PORTAGE_GRPNAME", "PORTAGE_INST_GID", "PORTAGE_INST_UID",
"PORTAGE_INTERNAL_CALLER", "PORTAGE_IPC_DAEMON", "PORTAGE_IUSE",
"PORTAGE_LOG_FILE", "PORTAGE_MUTABLE_FILTERED_VARS",
"PORTAGE_OVERRIDE_EPREFIX", "PORTAGE_PYM_PATH", "PORTAGE_PYTHON",
"PORTAGE_PYTHONPATH", "PORTAGE_READONLY_METADATA", "PORTAGE_READONLY_VARS",
"PORTAGE_REPO_NAME", "PORTAGE_REPOSITORIES", "PORTAGE_RESTRICT",
"PORTAGE_SAVED_READONLY_VARS", "PORTAGE_SIGPIPE_STATUS", "PORTAGE_TMPDIR",
"PORTAGE_UPDATE_ENV", "PORTAGE_USERNAME", "PORTAGE_VERBOSE",
"PORTAGE_WORKDIR_MODE", "PORTAGE_XATTR_EXCLUDE", "REPLACING_VERSIONS",
"REPLACED_BY_VERSION", "__PORTAGE_HELPER", "__PORTAGE_TEST_HARDLINK_LOCKS",
-- generic ebuilds
"A", "ARCH", "BDEPEND", "BOARD_USE", "BROOT", "CATEGORY", "D",
"DEFINED_PHASES", "DEPEND", "DESCRIPTION", "DISTDIR", "DOCS", "EAPI",
"ECLASS", "ED", "EPREFIX", "EROOT", "ESYSROOT", "EXTRA_ECONF",
"EXTRA_EINSTALL", "EXTRA_MAKE", "FEATURES", "FILESDIR", "HOME", "HOMEPAGE",
"HTML_DOCS", "INHERITED", "IUSE", "KEYWORDS", "LICENSE", "P", "PATCHES",
"PDEPEND", "PF", "PKG_INSTALL_MASK", "PKGUSE", "PN", "PR", "PROPERTIES",
"PROVIDES_EXCLUDE", "PV", "PVR", "QA_AM_MAINTAINER_MODE",
"QA_CONFIGURE_OPTIONS", "QA_DESKTOP_FILE", "QA_DT_NEEDED", "QA_EXECSTACK",
"QA_FLAGS_IGNORED", "QA_MULTILIB_PATHS", "QA_PREBUILT", "QA_PRESTRIPPED",
"QA_SONAME", "QA_SONAME_NO_SYMLINK", "QA_TEXTRELS", "QA_WX_LOAD", "RDEPEND",
"REPOSITORY", "REQUIRED_USE", "REQUIRES_EXCLUDE", "RESTRICT", "ROOT", "S",
"SLOT", "SRC_TEST", "SRC_URI", "STRIP_MASK", "SUBSLOT", "SYSROOT", "T",
"WORKDIR",
-- autotest.eclass declared incorrectly
"AUTOTEST_CLIENT_TESTS", "AUTOTEST_CLIENT_SITE_TESTS",
"AUTOTEST_SERVER_TESTS", "AUTOTEST_SERVER_SITE_TESTS", "AUTOTEST_CONFIG",
"AUTOTEST_DEPS", "AUTOTEST_PROFILERS", "AUTOTEST_CONFIG_LIST",
"AUTOTEST_DEPS_LIST", "AUTOTEST_PROFILERS_LIST",
-- cros-board.eclass declared incorrectly
"CROS_BOARDS",
-- Undeclared cros-kernel2 vars
"AFDO_PROFILE_VERSION",
-- haskell-cabal.eclass declared incorrectly
"CABAL_FEATURES",
-- Undeclared haskell-cabal.eclass vars
"CABAL_CORE_LIB_GHC_PV",
-- Undeclared readme.gentoo.eclass vars
"DOC_CONTENTS",
-- Backwards compatibility perl-module.eclass vars
"MODULE_AUTHOR", "MODULE_VERSION",
-- Undeclared perl-module.eclass vars
"mydoc",
-- python-utils-r1.eclass declared incorrectly
"RESTRICT_PYTHON_ABIS", "PYTHON_MODNAME",
-- ABI variables
"ABI", "DEFAULT_ABI",
-- AFDO variables
"AFDO_LOCATION",
-- Linguas
"LINGUAS"
]
specialIntegerVariables = [
"$", "?", "!", "#"
]
@@ -90,7 +174,9 @@ unbracedVariables = specialVariables ++ [
arrayVariables = [
"BASH_ALIASES", "BASH_ARGC", "BASH_ARGV", "BASH_CMDS", "BASH_LINENO",
"BASH_REMATCH", "BASH_SOURCE", "BASH_VERSINFO", "COMP_WORDS", "COPROC",
"DIRSTACK", "FUNCNAME", "GROUPS", "MAPFILE", "PIPESTATUS", "COMPREPLY"
"DIRSTACK", "FUNCNAME", "GROUPS", "MAPFILE", "PIPESTATUS", "COMPREPLY",
-- For Portage
"PATCHES"
]
commonCommands = [
@@ -150,15 +236,17 @@ unaryTestOps = [
"-o", "-v", "-R"
]
-- Variables inspected by Portage tc-export_build_env
portageBuildEnvVariables = [
"CFLAGS", "CXXFLAGS", "CPPFLAGS", "LDFLAGS"
]
shellForExecutable :: String -> Maybe Shell
shellForExecutable name =
case name of
"sh" -> return Sh
"bash" -> return Bash
"bats" -> return Bash
"busybox" -> return BusyboxSh -- Used for directives and --shell=busybox
"busybox sh" -> return BusyboxSh
"busybox ash" -> return BusyboxSh
"dash" -> return Dash
"ash" -> return Dash -- There's also a warning for this.
"ksh" -> return Ksh

View File

@@ -117,7 +117,8 @@ dummySystemInterface = mockedSystemInterface [
cfgParams :: CFGParameters
cfgParams = CFGParameters {
cfLastpipe = False,
cfPipefail = False
cfPipefail = False,
cfAdditionalInitialVariables = []
}
-- An example script to play with

View File

@@ -24,8 +24,8 @@ import ShellCheck.Formatter.Format
import Data.Char
import Data.List
import GHC.Exts
import System.IO
import qualified Data.List.NonEmpty as NE
format :: IO Formatter
format = return Formatter {
@@ -45,12 +45,12 @@ outputResults cr sys =
else mapM_ outputGroup fileGroups
where
comments = crComments cr
fileGroups = NE.groupWith sourceFile comments
fileGroups = groupWith sourceFile comments
outputGroup group = do
let filename = sourceFile (NE.head group)
let filename = sourceFile (head group)
result <- siReadFile sys (Just True) filename
let contents = either (const "") id result
outputFile filename contents (NE.toList group)
outputFile filename contents group
outputFile filename contents warnings = do
let comments = makeNonVirtual warnings contents

View File

@@ -23,8 +23,8 @@ import ShellCheck.Interface
import ShellCheck.Formatter.Format
import Data.List
import GHC.Exts
import System.IO
import qualified Data.List.NonEmpty as NE
format :: IO Formatter
format = return Formatter {
@@ -39,13 +39,13 @@ outputError file error = hPutStrLn stderr $ file ++ ": " ++ error
outputAll cr sys = mapM_ f groups
where
comments = crComments cr
groups = NE.groupWith sourceFile comments
f :: NE.NonEmpty PositionedComment -> IO ()
groups = groupWith sourceFile comments
f :: [PositionedComment] -> IO ()
f group = do
let filename = sourceFile (NE.head group)
let filename = sourceFile (head group)
result <- siReadFile sys (Just True) filename
let contents = either (const "") id result
outputResult filename contents (NE.toList group)
outputResult filename contents group
outputResult filename contents warnings = do
let comments = makeNonVirtual warnings contents

View File

@@ -27,9 +27,9 @@ import Control.DeepSeq
import Data.Aeson
import Data.IORef
import Data.Monoid
import GHC.Exts
import System.IO
import qualified Data.ByteString.Lazy.Char8 as BL
import qualified Data.List.NonEmpty as NE
format :: IO Formatter
format = do
@@ -114,10 +114,10 @@ outputError file msg = hPutStrLn stderr $ file ++ ": " ++ msg
collectResult ref cr sys = mapM_ f groups
where
comments = crComments cr
groups = NE.groupWith sourceFile comments
f :: NE.NonEmpty PositionedComment -> IO ()
groups = groupWith sourceFile comments
f :: [PositionedComment] -> IO ()
f group = do
let filename = sourceFile (NE.head group)
let filename = sourceFile (head group)
result <- siReadFile sys (Just True) filename
let contents = either (const "") id result
let comments' = makeNonVirtual comments contents

View File

@@ -31,9 +31,9 @@ import Data.Ord
import Data.IORef
import Data.List
import Data.Maybe
import GHC.Exts
import System.IO
import System.Info
import qualified Data.List.NonEmpty as NE
wikiLink = "https://www.shellcheck.net/wiki/"
@@ -117,19 +117,19 @@ outputResult options ref result sys = do
color <- getColorFunc $ foColorOption options
let comments = crComments result
appendComments ref comments (fromIntegral $ foWikiLinkCount options)
let fileGroups = NE.groupWith sourceFile comments
let fileGroups = groupWith sourceFile comments
mapM_ (outputForFile color sys) fileGroups
outputForFile color sys comments = do
let fileName = sourceFile (NE.head comments)
let fileName = sourceFile (head comments)
result <- siReadFile sys (Just True) fileName
let contents = either (const "") id result
let fileLinesList = lines contents
let lineCount = length fileLinesList
let fileLines = listArray (1, lineCount) fileLinesList
let groups = NE.groupWith lineNo comments
let groups = groupWith lineNo comments
forM_ groups $ \commentsForLine -> do
let lineNum = fromIntegral $ lineNo (NE.head commentsForLine)
let lineNum = fromIntegral $ lineNo (head commentsForLine)
let line = if lineNum < 1 || lineNum > lineCount
then ""
else fileLines ! fromIntegral lineNum
@@ -139,7 +139,7 @@ outputForFile color sys comments = do
putStrLn (color "source" line)
forM_ commentsForLine $ \c -> putStrLn $ color (severityText c) $ cuteIndent c
putStrLn ""
showFixedString color (toList commentsForLine) (fromIntegral lineNum) fileLines
showFixedString color commentsForLine (fromIntegral lineNum) fileLines
-- Pick out only the lines necessary to show a fix in action
sliceFile :: Fix -> Array Int String -> (Fix, Array Int String)

View File

@@ -1,5 +1,5 @@
{-
Copyright 2012-2024 Vidar Holen
Copyright 2012-2019 Vidar Holen
This file is part of ShellCheck.
https://www.shellcheck.net
@@ -21,14 +21,14 @@
module ShellCheck.Interface
(
SystemInterface(..)
, CheckSpec(csFilename, csScript, csCheckSourced, csIncludedWarnings, csExcludedWarnings, csShellTypeOverride, csMinSeverity, csIgnoreRC, csExtendedAnalysis, csOptionalChecks)
, CheckSpec(csFilename, csScript, csCheckSourced, csIncludedWarnings, csExcludedWarnings, csShellTypeOverride, csMinSeverity, csIgnoreRC, csOptionalChecks)
, CheckResult(crFilename, crComments)
, ParseSpec(psFilename, psScript, psCheckSourced, psIgnoreRC, psShellTypeOverride)
, ParseResult(prComments, prTokenPositions, prRoot)
, AnalysisSpec(asScript, asShellType, asFallbackShell, asExecutionMode, asCheckSourced, asTokenPositions, asExtendedAnalysis, asOptionalChecks)
, AnalysisSpec(..)
, AnalysisResult(arComments)
, FormatterOptions(foColorOption, foWikiLinkCount)
, Shell(Ksh, Sh, Bash, Dash, BusyboxSh)
, Shell(Ksh, Sh, Bash, Dash)
, ExecutionMode(Executed, Sourced)
, ErrorMessage
, Code
@@ -87,7 +87,9 @@ data SystemInterface m = SystemInterface {
-- find the sourced file
siFindSource :: String -> Maybe Bool -> [String] -> String -> m FilePath,
-- | Get the configuration file (name, contents) for a filename
siGetConfig :: String -> m (Maybe (FilePath, String))
siGetConfig :: String -> m (Maybe (FilePath, String)),
-- | Look up Portage Eclass variables
siGetPortageVariables :: m (Map.Map String [String])
}
-- ShellCheck input and output
@@ -100,7 +102,6 @@ data CheckSpec = CheckSpec {
csIncludedWarnings :: Maybe [Integer],
csShellTypeOverride :: Maybe Shell,
csMinSeverity :: Severity,
csExtendedAnalysis :: Maybe Bool,
csOptionalChecks :: [String]
} deriving (Show, Eq)
@@ -125,7 +126,6 @@ emptyCheckSpec = CheckSpec {
csIncludedWarnings = Nothing,
csShellTypeOverride = Nothing,
csMinSeverity = StyleC,
csExtendedAnalysis = Nothing,
csOptionalChecks = []
}
@@ -143,7 +143,8 @@ newSystemInterface =
SystemInterface {
siReadFile = \_ _ -> return $ Left "Not implemented",
siFindSource = \_ _ _ name -> return name,
siGetConfig = \_ -> return Nothing
siGetConfig = \_ -> return Nothing,
siGetPortageVariables = return Map.empty
}
-- Parser input and output
@@ -175,8 +176,8 @@ data AnalysisSpec = AnalysisSpec {
asFallbackShell :: Maybe Shell,
asExecutionMode :: ExecutionMode,
asCheckSourced :: Bool,
asIsPortage :: Bool,
asOptionalChecks :: [String],
asExtendedAnalysis :: Maybe Bool,
asTokenPositions :: Map.Map Id (Position, Position)
}
@@ -186,8 +187,8 @@ newAnalysisSpec token = AnalysisSpec {
asFallbackShell = Nothing,
asExecutionMode = Executed,
asCheckSourced = False,
asIsPortage = False,
asOptionalChecks = [],
asExtendedAnalysis = Nothing,
asTokenPositions = Map.empty
}
@@ -225,7 +226,7 @@ newCheckDescription = CheckDescription {
}
-- Supporting data types
data Shell = Ksh | Sh | Bash | Dash | BusyboxSh deriving (Show, Eq)
data Shell = Ksh | Sh | Bash | Dash deriving (Show, Eq)
data ExecutionMode = Executed | Sourced deriving (Show, Eq)
type ErrorMessage = String
@@ -339,3 +340,4 @@ mockedSystemInterface files = (newSystemInterface :: SystemInterface Identity) {
mockRcFile rcfile mock = mock {
siGetConfig = const . return $ Just (".shellcheckrc", rcfile)
}

View File

@@ -46,7 +46,6 @@ import Text.Parsec.Error
import Text.Parsec.Pos
import qualified Control.Monad.Reader as Mr
import qualified Control.Monad.State as Ms
import qualified Data.List.NonEmpty as NE
import qualified Data.Map.Strict as Map
import Test.QuickCheck.All (quickCheckAll)
@@ -161,7 +160,7 @@ data Context =
deriving (Show)
data HereDocContext =
HereDocPending Id Dashed Quoted String [Context] -- on linefeed, read this T_HereDoc
HereDocPending Token [Context] -- on linefeed, read this T_HereDoc
deriving (Show)
data UserState = UserState {
@@ -239,12 +238,12 @@ addToHereDocMap id list = do
hereDocMap = Map.insert id list map
}
addPendingHereDoc id d q str = do
addPendingHereDoc t = do
state <- getState
context <- getCurrentContexts
let docs = pendingHereDocs state
putState $ state {
pendingHereDocs = HereDocPending id d q str context : docs
pendingHereDocs = HereDocPending t context : docs
}
popPendingHereDocs = do
@@ -1058,16 +1057,6 @@ readAnnotationWithoutPrefix sandboxed = do
"This shell type is unknown. Use e.g. sh or bash."
return [ShellOverride shell]
"extended-analysis" -> do
pos <- getPosition
value <- plainOrQuoted $ many1 letter
case value of
"true" -> return [ExtendedAnalysis True]
"false" -> return [ExtendedAnalysis False]
_ -> do
parseNoteAt pos ErrorC 1146 "Unknown extended-analysis value. Expected true/false."
return []
"external-sources" -> do
pos <- getPosition
value <- plainOrQuoted $ many1 letter
@@ -1205,7 +1194,7 @@ readDollarBracedPart = readSingleQuoted <|> readDoubleQuoted <|>
readDollarBracedLiteral = do
start <- startSpan
vars <- (readBraceEscaped <|> ((\x -> [x]) <$> anyChar)) `reluctantlyTill1` bracedQuotable
vars <- (readBraceEscaped <|> (anyChar >>= \x -> return [x])) `reluctantlyTill1` bracedQuotable
id <- endSpan start
return $ T_Literal id $ concat vars
@@ -1567,7 +1556,7 @@ readGenericLiteral endChars = do
return $ concat strings
readGenericLiteral1 endExp = do
strings <- (readGenericEscaped <|> ((\x -> [x]) <$> anyChar)) `reluctantlyTill1` endExp
strings <- (readGenericEscaped <|> (anyChar >>= \x -> return [x])) `reluctantlyTill1` endExp
return $ concat strings
readGenericEscaped = do
@@ -1846,7 +1835,7 @@ readHereDoc = called "here document" $ do
-- add empty tokens for now, read the rest in readPendingHereDocs
let doc = T_HereDoc hid dashed quoted endToken []
addPendingHereDoc hid dashed quoted endToken
addPendingHereDoc doc
return doc
where
unquote :: String -> (Quoted, String)
@@ -1867,7 +1856,7 @@ readPendingHereDocs = do
docs <- popPendingHereDocs
mapM_ readDoc docs
where
readDoc (HereDocPending id dashed quoted endToken ctx) =
readDoc (HereDocPending (T_HereDoc id dashed quoted endToken _) ctx) =
swapContext ctx $
do
docStartPos <- getPosition
@@ -2381,7 +2370,7 @@ readPipeSequence = do
return $ T_Pipeline id pipes cmds
where
sepBy1WithSeparators p s = do
let elems = (\x -> ([x], [])) <$> p
let elems = p >>= \x -> return ([x], [])
let seps = do
separator <- s
return $ \(a,b) (c,d) -> (a++c, b ++ d ++ [separator])
@@ -2915,8 +2904,8 @@ readLetSuffix = many1 (readIoRedirect <|> try readLetExpression <|> readCmdWord)
kludgeAwayQuotes :: String -> SourcePos -> (String, SourcePos)
kludgeAwayQuotes s p =
case s of
first:second:rest ->
let (last NE.:| backwards) = NE.reverse (second NE.:| rest)
first:rest@(_:_) ->
let (last:backwards) = reverse rest
middle = reverse backwards
in
if first `elem` "'\"" && first == last
@@ -3350,8 +3339,7 @@ readScriptFile sourced = do
verifyEof
let script = T_Annotation annotationId annotations $
T_Script id shebang commands
userstate <- getState
reparseIndices $ reattachHereDocs script (hereDocMap userstate)
reparseIndices script
else do
many anyChar
id <- endSpan start
@@ -3361,8 +3349,8 @@ readScriptFile sourced = do
verifyShebang pos s = do
case isValidShell s of
Just True -> return ()
Just False -> parseProblemAt pos ErrorC 1071 "ShellCheck only supports sh/bash/dash/ksh/'busybox sh' scripts. Sorry!"
Nothing -> parseProblemAt pos ErrorC 1008 "This shebang was unrecognized. ShellCheck only supports sh/bash/dash/ksh/'busybox sh'. Add a 'shell' directive to specify."
Just False -> parseProblemAt pos ErrorC 1071 "ShellCheck only supports sh/bash/dash/ksh scripts. Sorry!"
Nothing -> parseProblemAt pos ErrorC 1008 "This shebang was unrecognized. ShellCheck only supports sh/bash/dash/ksh. Add a 'shell' directive to specify."
isValidShell s =
let good = null s || any (`isPrefixOf` s) goodShells
@@ -3378,7 +3366,6 @@ readScriptFile sourced = do
"sh",
"ash",
"dash",
"busybox sh",
"bash",
"bats",
"ksh"
@@ -3466,8 +3453,9 @@ makeErrorFor parsecError =
pos = errorPos parsecError
getStringFromParsec errors =
headOrDefault "" (mapMaybe f $ reverse errors) ++
" Fix any mentioned problems and try again."
case map f errors of
r -> unwords (take 1 $ catMaybes $ reverse r) ++
" Fix any mentioned problems and try again."
where
f err =
case err of
@@ -3498,7 +3486,8 @@ parseShell env name contents = do
return newParseResult {
prComments = map toPositionedComment $ nub $ parseNotes userstate ++ parseProblems state,
prTokenPositions = Map.map startEndPosToPos (positionMap userstate),
prRoot = Just script
prRoot = Just $
reattachHereDocs script (hereDocMap userstate)
}
Left err -> do
let context = contextStack state
@@ -3516,11 +3505,13 @@ parseShell env name contents = do
-- A final pass for ignoring parse errors after failed parsing
isIgnored stack note = any (contextItemDisablesCode False (codeForParseNote note)) stack
notesForContext list = zipWith ($) [first, second] [(pos, str) | ContextName pos str <- list]
notesForContext list = zipWith ($) [first, second] $ filter isName list
where
first (pos, str) = ParseNote pos pos ErrorC 1073 $
isName (ContextName _ _) = True
isName _ = False
first (ContextName pos str) = ParseNote pos pos ErrorC 1073 $
"Couldn't parse this " ++ str ++ ". Fix to allow more checks."
second (pos, str) = ParseNote pos pos InfoC 1009 $
second (ContextName pos str) = ParseNote pos pos InfoC 1009 $
"The mentioned syntax error was in this " ++ str ++ "."
-- Go over all T_UnparsedIndex and reparse them as either arithmetic or text

View File

@@ -0,0 +1,138 @@
{-# LANGUAGE ApplicativeDo #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE TupleSections #-}
module ShellCheck.PortageVariables (
readPortageVariables
) where
import ShellCheck.Regex
import Control.Exception
import Control.Monad
import Data.Maybe
import System.Directory (listDirectory)
import System.Exit (ExitCode(..))
import System.FilePath
import System.IO
import System.Process
import qualified Data.ByteString as B
import qualified Data.Map as M
type RepoName = String
type RepoPath = String
type EclassName = String
type EclassVar = String
-- | This is used for looking up what eclass variables are inherited,
-- keyed by the name of the eclass.
type EclassMap = M.Map EclassName [EclassVar]
data Repository = Repository
{ repositoryName :: RepoName
, repositoryLocation :: RepoPath
, repositoryEclasses :: [Eclass]
} deriving (Show, Eq, Ord)
data Eclass = Eclass
{ eclassName :: EclassName
, eclassVars :: [EclassVar]
} deriving (Show, Eq, Ord)
readPortageVariables :: IO (M.Map String [String])
readPortageVariables = portageVariables <$> scanRepos
-- | Map from eclass names to a list of eclass variables
portageVariables :: [Repository] -> EclassMap
portageVariables = foldMap $ foldMap go . repositoryEclasses
where
go e = M.singleton (eclassName e) (eclassVars e)
-- | Run @portageq@ to gather a list of repo names and paths, then scan each
-- one for eclasses and ultimately eclass metadata.
scanRepos :: IO [Repository]
scanRepos = do
let cmd = "portageq"
let args = ["repos_config", "/"]
out <- runOrDie cmd args
forM (reposParser $ lines out) $ \(n,p) -> Repository n p <$> getEclasses p
-- | Get the name of the repo and its path from blocks outputted by
-- @portageq@. If the path doesn't exist, this will return @Nothing@.
reposParser :: [String] -> [(RepoName, RepoPath)]
reposParser = f ""
where
segmentRegex = mkRegex "^\\[(.*)\\].*"
locationRegex = mkRegex "^[[:space:]]*location[[:space:]]*=[[:space:]]*(.*)[[:space:]]*$"
f name [] = []
f name (line:rest) =
case (matchRegex segmentRegex line, matchRegex locationRegex line) of
(Just [next], _) -> f next rest
(_, Just [location]) -> (name, location) : f name rest
_ -> f name rest
-- | Scan the repo path for @*.eclass@ files in @eclass/@, then run
-- 'eclassParser' on each of them to produce @[Eclass]@.
--
-- If the @eclass/@ directory doesn't exist, the scan is skipped for that
-- repo.
getEclasses :: RepoPath -> IO [Eclass]
getEclasses repoLoc = do
let eclassDir = repoLoc </> "eclass"
files <- handle catcher $ listDirectory eclassDir
let names = filter (\(_, e) -> e == ".eclass") $ map splitExtension files
forM (names :: [(String, String)]) $ \(name, ext) -> do
contents <- withFile (eclassDir </> name <.> ext) ReadMode readFully
return $ Eclass name $ eclassParser (lines contents)
where
catcher :: IOException -> IO [String]
catcher e = do
hPutStrLn stderr $ "Unable to find .eclass files: " ++ show e
return []
-- | Scan a @.eclass@ file for any @@@ECLASS_VARIABLE:@ comments, generating
-- a list of eclass variables.
eclassParser :: [String] -> [String]
eclassParser lines = mapMaybe match lines
where
varRegex = mkRegex "^[[:space:]]*#[[:space:]]*@ECLASS_VARIABLE:[[:space:]]*([^[:space:]]*)[[:space:]]*$"
match str = head <$> matchRegex varRegex str
-- | Run the command and return the full stdout string (stdin is ignored).
--
-- If the command exits with a non-zero exit code, this will throw an
-- error including the captured contents of stdout and stderr.
runOrDie :: FilePath -> [String] -> IO String
runOrDie cmd args = bracket acquire release $ \(_,o,e,p) -> do
ot <- readFully (fromJust o)
et <- readFully (fromJust e)
ec <- waitForProcess p
case ec of
ExitSuccess -> pure ot
ExitFailure i -> fail $ unlines $ map unwords
$ [ [ show cmd ]
++ map show args
++ [ "failed with exit code", show i]
, [ "stdout:" ], [ ot ]
, [ "stderr:" ], [ et ]
]
where
acquire = createProcess (proc cmd args)
{ std_in = NoStream
, std_out = CreatePipe
, std_err = CreatePipe
}
release (i,o,e,p) = do
_ <- waitForProcess p
forM_ [i,o,e] $ mapM_ hClose
readFully :: Handle -> IO String
readFully handle = do
hSetBinaryMode handle True
str <- hGetContents handle
length str `seq` return str

View File

@@ -76,6 +76,8 @@ archlinux:latest pacman -S -y --noconfirm cabal-install ghc-static base-dev
# Ubuntu LTS
ubuntu:22.04 apt-get update && apt-get install -y cabal-install
ubuntu:20.04 apt-get update && apt-get install -y cabal-install
ubuntu:18.04 apt-get update && apt-get install -y cabal-install
ubuntu:16.04 apt-get update && apt-get install -y cabal-install
# Stack on Ubuntu LTS
ubuntu:22.04 set -e; apt-get update && apt-get install -y curl && curl -sSL https://get.haskellstack.org/ | sh -s - -f && cd /mnt && exec test/stacktest

View File

@@ -18,24 +18,21 @@ import qualified ShellCheck.Parser
main = do
putStrLn "Running ShellCheck tests..."
failures <- filter (not . snd) <$> mapM sequenceA tests
if null failures then exitSuccess else do
putStrLn "Tests failed for the following module(s):"
mapM (putStrLn . ("- ShellCheck." ++) . fst) failures
exitFailure
where
tests =
[ ("Analytics" , ShellCheck.Analytics.runTests)
, ("AnalyzerLib" , ShellCheck.AnalyzerLib.runTests)
, ("ASTLib" , ShellCheck.ASTLib.runTests)
, ("CFG" , ShellCheck.CFG.runTests)
, ("CFGAnalysis" , ShellCheck.CFGAnalysis.runTests)
, ("Checker" , ShellCheck.Checker.runTests)
, ("Checks.Commands" , ShellCheck.Checks.Commands.runTests)
, ("Checks.ControlFlow" , ShellCheck.Checks.ControlFlow.runTests)
, ("Checks.Custom" , ShellCheck.Checks.Custom.runTests)
, ("Checks.ShellSupport", ShellCheck.Checks.ShellSupport.runTests)
, ("Fixer" , ShellCheck.Fixer.runTests)
, ("Formatter.Diff" , ShellCheck.Formatter.Diff.runTests)
, ("Parser" , ShellCheck.Parser.runTests)
results <- sequence [
ShellCheck.Analytics.runTests
,ShellCheck.AnalyzerLib.runTests
,ShellCheck.ASTLib.runTests
,ShellCheck.CFG.runTests
,ShellCheck.CFGAnalysis.runTests
,ShellCheck.Checker.runTests
,ShellCheck.Checks.Commands.runTests
,ShellCheck.Checks.ControlFlow.runTests
,ShellCheck.Checks.Custom.runTests
,ShellCheck.Checks.ShellSupport.runTests
,ShellCheck.Fixer.runTests
,ShellCheck.Formatter.Diff.runTests
,ShellCheck.Parser.runTests
]
if and results
then exitSuccess
else exitFailure