7 SC2094

Joachim Ansorg edited this page 2021-11-12 19:31:23 +01:00

Table of Contents

• Make sure not to read and write the same file in the same pipeline.
• Problematic code:
• Correct code:
• Rationale:
• Exceptions
• Additional Resources

Make sure not to read and write the same file in the same pipeline.

Problematic code:

grep foo file.txt | sed -e 's/foo/bar/g' > file.txt

Correct code:

grep foo file.txt  | sed -e 's/foo/bar/g' > tmpfile && mv tmpfile file.txt

Rationale:

Each step in a pipeline runs in parallel.

In this case, grep foo file.txt will immediately try to read file.txt while sed .. > file.txt will immediately try to truncate it.

This is a race condition, and results in the file being partially or (far more likely) entirely truncated.

Note that this can also be a problem when you write to a file and read from it later in the pipe. The second command (which reads the file) may not see all the output of the first. An exception in this case is a non-greedy file reader like less, for example python foo.py 2> errfile.txt | less - errfile.txt will successfully allow you to see stdout and stderr separately in less.

Exceptions

You can ignore this error if:

• The file is a device or named pipe. These files don't truncate in the same way.
• The command mentions the filename but doesn't read/write it, such as echo log.txt > log.txt.

Additional Resources

• StackOverflow: bash redirect input from file back into same file
• BashPitfalls: cat file | sed s/foo/bar/ > file

ShellCheck

• Installation

  • CentOS6
  • More Installation Guides

• Usage

  • Directive
  • Ignore

• Integrating and extending

  • Misc. contributions
  • Azure Pipelines
  • Buildkite
  • CircleCI
  • DevGuide
  • Integration
  • JUnit
  • Phabricator
  • TravisCI
  • GitLab CI
  • Drone
  • GitHub Actions

• Severity

• Template

Each individual ShellCheck warning has its own wiki page like SC1000. Use GitHub Wiki's "Pages" feature above to find a specific one, or see Checks.