292 lines
18 KiB
Plaintext
292 lines
18 KiB
Plaintext
---
|
|
description: This rule enforces best practices and coding standards for Bash scripting to improve code quality, maintainability, and security. It covers naming conventions, formatting, error handling, security, and performance considerations.
|
|
globs: *.sh
|
|
---
|
|
# Bash Best Practices and Coding Standards
|
|
|
|
This document outlines the best practices and coding standards for writing Bash scripts. Following these guidelines will lead to more maintainable, readable, and robust code.
|
|
|
|
## 1. Code Organization and Structure
|
|
|
|
### 1.1. Directory Structure Best Practices
|
|
|
|
For large Bash projects, a well-defined directory structure is crucial. Consider the following structure:
|
|
|
|
|
|
project_root/
|
|
├── bin/ # Executable scripts (main entry points)
|
|
├── lib/ # Library scripts (reusable functions and modules)
|
|
├── config/ # Configuration files (e.g., settings, environment variables)
|
|
├── data/ # Data files (e.g., CSV, JSON)
|
|
├── log/ # Log files generated by scripts
|
|
├── tmp/ # Temporary files used during script execution
|
|
├── tests/ # Unit and integration tests
|
|
├── docs/ # Documentation (e.g., README, design documents)
|
|
└── README.md # Project README
|
|
|
|
|
|
### 1.2. File Naming Conventions
|
|
|
|
* **Executable Scripts:** Use descriptive names without extensions (e.g., `process_data`, `backup_system`).
|
|
* **Library Scripts:** Use `.sh` extension (e.g., `string_utils.sh`, `database_functions.sh`). For bash-specific code use `.bash`.
|
|
* **Configuration Files:** Use `.conf` or `.cfg` extensions (e.g., `app.conf`, `settings.cfg`).
|
|
* **Test Files:** Use `_test.sh` suffix (e.g., `string_utils_test.sh`).
|
|
|
|
### 1.3. Module Organization
|
|
|
|
* **Separate Concerns:** Divide your code into logical modules, each responsible for a specific task (e.g., database interaction, string manipulation, system monitoring).
|
|
* **Reusable Functions:** Place reusable functions in library scripts (`lib/`) and source them in your main scripts using `source lib/string_utils.sh` or `. lib/string_utils.sh`.
|
|
* **Modular Design:** Aim for a modular design where each module can be easily tested and reused in other projects.
|
|
|
|
### 1.4. Component Architecture
|
|
|
|
* **Functions as Components:** Treat functions as components with well-defined inputs (parameters) and outputs (return values or side effects).
|
|
* **Configuration Management:** Use configuration files to store settings that can be easily modified without changing the script's code.
|
|
* **Logging:** Implement a consistent logging mechanism to track script execution and errors.
|
|
|
|
### 1.5. Code Splitting Strategies
|
|
|
|
* **Function Decomposition:** Break down complex tasks into smaller, more manageable functions.
|
|
* **File Inclusion:** Use `source` or `.` to include reusable code from other files.
|
|
* **External Commands:** Delegate tasks to external commands when appropriate (e.g., `grep`, `sed`, `awk`).
|
|
* **Consider a 'main' function:** Wrap the primary logic of the script into a `main` function to improve readability and maintainability. All scripts that are long enough to contain at least one other function should have a `main` function.
|
|
|
|
## 2. Common Patterns and Anti-patterns
|
|
|
|
### 2.1. Design Patterns
|
|
|
|
* **Facade:** Create a simplified interface to a complex set of functions or modules.
|
|
* **Strategy:** Define a family of algorithms and encapsulate each one in a separate function, making them interchangeable.
|
|
* **Template Method:** Define the skeleton of an algorithm in a function, deferring some steps to subfunctions.
|
|
* **Chain of Responsibility:** Pass a request along a chain of functions until one of them handles it.
|
|
* **Singleton (Carefully):** Use sparingly for global configuration, initialize only once.
|
|
|
|
### 2.2. Recommended Approaches for Common Tasks
|
|
|
|
* **String Manipulation:** Use Bash's built-in string manipulation features (e.g., parameter expansion, substring extraction) instead of external commands like `sed` or `awk` when possible.
|
|
* **File Handling:** Use Bash's built-in file handling commands (e.g., `read`, `write`, `mkdir`, `rm`) for basic operations. For more complex operations, consider `find` with `-exec` or `xargs`.
|
|
* **Looping:** Use `for` loops for iterating over lists of items and `while` loops for conditional execution.
|
|
* **Conditional Statements:** Use `if`, `elif`, and `else` statements for branching logic. Prefer `[[ ]]` over `[ ]` for string comparisons.
|
|
* **Exit Early:** Use `return` or `exit` to exit the script as soon as an error is detected.
|
|
|
|
### 2.3. Anti-patterns and Code Smells
|
|
|
|
* **Hardcoded Values:** Avoid hardcoding values directly in the script. Use constants or configuration files instead.
|
|
* **Unquoted Variables:** Always quote variables to prevent word splitting and globbing issues.
|
|
* **Eval:** Avoid using `eval` as it can introduce security vulnerabilities and make the code difficult to understand.
|
|
* **Backticks:** Use `$(command)` instead of backticks for command substitution.
|
|
* **Relying on Globals:** Minimize the use of global variables to avoid naming conflicts and unexpected side effects. Use `local` within functions.
|
|
* **Ignoring Errors:** Always check the return values of commands to handle errors gracefully. Use `set -e` to exit on errors automatically.
|
|
* **Over-commenting Obvious Code:** Comments should explain *why* the code is doing something, not *what* the code is doing.
|
|
* **Excessive Piping:** While powerful, long pipelines can become unreadable. Consider breaking down complex operations into smaller, more manageable steps with intermediate variables.
|
|
|
|
### 2.4. State Management Best Practices
|
|
|
|
* **Environment Variables:** Use environment variables to store global configuration settings that can be accessed by all scripts and functions.
|
|
* **Temporary Files:** Use temporary files to store intermediate data during script execution. Use `mktemp` to create unique temporary file names and remove them when finished. They should be created under `/tmp` or another location if needed.
|
|
* **Persistent Storage:** For persistent state, consider using a simple database (e.g., SQLite) or a key-value store (e.g., Redis).
|
|
* **Configuration Files:** Store persistent configuration options in configuration files.
|
|
|
|
### 2.5. Error Handling Patterns
|
|
|
|
* **`set -e`:** Exit immediately if a command exits with a non-zero status.
|
|
* **Check Return Values:** Use `$?` to check the return value of a command and handle errors accordingly.
|
|
* **Error Functions:** Create functions to handle common error scenarios (e.g., logging errors, displaying error messages, exiting the script).
|
|
* **Signal Handling:** Use `trap` to handle signals (e.g., `SIGINT`, `SIGTERM`) and perform cleanup actions.
|
|
* **Informative Error Messages:** Provide clear and informative error messages to help users diagnose problems. The error messages should explain the cause and possibly the recommended solution.
|
|
|
|
## 3. Performance Considerations
|
|
|
|
### 3.1. Optimization Techniques
|
|
|
|
* **Built-in Commands:** Prefer built-in commands over external commands whenever possible, as they are generally faster.
|
|
* **String Operations:** Use Bash's built-in string manipulation features instead of external commands like `sed` or `awk` for simple operations.
|
|
* **Avoid Loops:** Minimize the use of loops, as they can be slow in Bash. Consider using `find` with `-exec` or `xargs` for file processing instead.
|
|
* **Command Substitution:** Avoid unnecessary command substitution, as it can be expensive.
|
|
* **Arrays:** Use arrays to store lists of items instead of strings, as they are more efficient for iteration and manipulation.
|
|
* **Parameter Expansion:** Use parameter expansion features like `${var:-default}` to provide default values for variables without using conditional statements.
|
|
* **`set -f`:** Disable globbing when not needed to prevent unintended file expansion.
|
|
|
|
### 3.2. Memory Management
|
|
|
|
* **Limit Data Size:** Avoid loading large files or data structures into memory.
|
|
* **Stream Processing:** Use stream processing techniques to process data incrementally instead of loading it all into memory at once.
|
|
* **Remove Variables:** Unset or reassign variables to release memory when they are no longer needed.
|
|
|
|
### 3.3. Rendering Optimization (If Applicable)
|
|
|
|
* **Minimize Output:** Reduce the amount of output generated by the script, as it can slow down execution.
|
|
* **Use `printf`:** Use `printf` for formatted output instead of `echo`, as it is generally faster and more portable.
|
|
|
|
### 3.4. Bundle Size Optimization (If Applicable)
|
|
|
|
* **N/A:** Bash scripts do not typically have bundle sizes in the same way that web applications do.
|
|
|
|
### 3.5. Lazy Loading
|
|
|
|
* **Source on Demand:** Only source library scripts when they are needed.
|
|
* **Conditional Execution:** Use conditional statements to execute code blocks only when certain conditions are met.
|
|
|
|
## 4. Security Best Practices
|
|
|
|
### 4.1. Common Vulnerabilities and Prevention
|
|
|
|
* **Command Injection:** Avoid using `eval` and carefully sanitize any user input used in commands.
|
|
* **Path Injection:** Sanitize path names to prevent malicious users from manipulating file operations.
|
|
* **Denial of Service (DoS):** Implement resource limits to prevent scripts from consuming excessive CPU, memory, or disk space.
|
|
* **Information Disclosure:** Protect sensitive information (e.g., passwords, API keys) by storing them in environment variables or configuration files with appropriate permissions.
|
|
* **Shellshock:** Ensure your Bash version is patched against the Shellshock vulnerability.
|
|
|
|
### 4.2. Input Validation
|
|
|
|
* **Sanitize User Input:** Carefully sanitize any user input to prevent command injection and other vulnerabilities.
|
|
* **Regular Expressions:** Use regular expressions to validate input formats (e.g., email addresses, phone numbers).
|
|
* **Whitelist Validation:** Validate input against a whitelist of allowed values.
|
|
* **Length Limits:** Enforce length limits on input fields to prevent buffer overflows.
|
|
|
|
### 4.3. Authentication and Authorization
|
|
|
|
* **Avoid Storing Credentials:** Avoid storing credentials directly in scripts or configuration files. Use environment variables or a secure credential store instead.
|
|
* **Principle of Least Privilege:** Run scripts with the minimum necessary privileges.
|
|
* **User Authentication:** Implement user authentication using `sudo` or other authentication mechanisms.
|
|
* **Role-Based Access Control (RBAC):** Implement RBAC to restrict access to sensitive resources based on user roles.
|
|
|
|
### 4.4. Data Protection
|
|
|
|
* **Encryption:** Use encryption to protect sensitive data at rest and in transit.
|
|
* **Data Masking:** Mask sensitive data in log files and other output.
|
|
* **Access Control:** Restrict access to sensitive data to authorized users only.
|
|
|
|
### 4.5. Secure API Communication
|
|
|
|
* **HTTPS:** Use HTTPS for all API communication to encrypt data in transit.
|
|
* **API Keys:** Use API keys to authenticate requests to external APIs.
|
|
* **Rate Limiting:** Implement rate limiting to prevent abuse of your API.
|
|
* **Input Validation:** Validate all input from external APIs to prevent injection attacks.
|
|
|
|
## 5. Testing Approaches
|
|
|
|
### 5.1. Unit Testing
|
|
|
|
* **Test Individual Functions:** Write unit tests to verify the behavior of individual functions.
|
|
* **Mock Dependencies:** Use mocking techniques to isolate functions from their dependencies.
|
|
* **Assert Statements:** Use assert statements to verify that the functions return the expected values.
|
|
* **Testing Frameworks:** Consider using a bash testing framework like Bats (Bash Automated Testing System).
|
|
|
|
### 5.2. Integration Testing
|
|
|
|
* **Test Interactions:** Write integration tests to verify the interactions between different modules or components.
|
|
* **Real Environments:** Run integration tests in a real or simulated environment.
|
|
* **End-to-End Testing:** Write end-to-end tests to verify the entire script from start to finish.
|
|
|
|
### 5.3. End-to-End Testing
|
|
|
|
* **Real Use Cases:** Emulate real-world use cases to test the script's overall functionality.
|
|
* **System-Level Testing:** Verify that the script interacts correctly with the underlying system.
|
|
|
|
### 5.4. Test Organization
|
|
|
|
* **Separate Test Directory:** Create a separate `tests/` directory to store test files.
|
|
* **Naming Convention:** Use a consistent naming convention for test files (e.g., `module_name_test.sh`).
|
|
* **Test Suites:** Organize tests into suites based on functionality or module.
|
|
|
|
### 5.5. Mocking and Stubbing
|
|
|
|
* **Mock External Commands:** Create mock versions of external commands to isolate functions from their dependencies.
|
|
* **Stub Functions:** Create stub versions of functions to control their behavior during testing.
|
|
* **Environment Variable Mocking:** Mock environment variables to test different configurations.
|
|
|
|
## 6. Common Pitfalls and Gotchas
|
|
|
|
### 6.1. Frequent Mistakes
|
|
|
|
* **Forgetting `set -e`:** Not exiting on errors can lead to unexpected behavior.
|
|
* **Unquoted Variables:** Forgetting to quote variables can lead to word splitting and globbing issues.
|
|
* **Incorrect Variable Scope:** Using global variables when local variables are needed can lead to naming conflicts.
|
|
* **Not Checking Return Values:** Not checking return values can lead to errors being ignored.
|
|
* **Over-reliance on Globals** Using global variables when local variables are preferable can lead to unintended side effects.
|
|
* **Not using Functions** Code becomes difficult to maintain if its not broken into manageable functions.
|
|
|
|
### 6.2. Edge Cases
|
|
|
|
* **Empty Variables:** Handle empty variables gracefully.
|
|
* **Special Characters:** Handle special characters in file names and other input.
|
|
* **Large Files:** Handle large files efficiently to prevent memory issues.
|
|
* **Race Conditions:** Avoid race conditions when multiple scripts are running concurrently.
|
|
|
|
### 6.3. Version-Specific Issues
|
|
|
|
* **Bash 3 vs. Bash 4:** Be aware of compatibility issues between different versions of Bash.
|
|
* **POSIX Compliance:** If portability is a concern, stick to POSIX-compliant syntax.
|
|
|
|
### 6.4. Compatibility Concerns
|
|
|
|
* **Operating Systems:** Ensure your scripts are compatible with the target operating systems.
|
|
* **External Tools:** Be aware of the dependencies on external tools and ensure they are available on the target systems.
|
|
|
|
### 6.5. Debugging Strategies
|
|
|
|
* **`set -x`:** Enable tracing to see the commands being executed.
|
|
* **`echo` Statements:** Use `echo` statements to print variable values and debug messages.
|
|
* **Shellcheck:** Use Shellcheck to identify common errors and warnings.
|
|
* **Interactive Debugging:** Use an interactive debugger like `bashdb` to step through the code.
|
|
|
|
## 7. Tooling and Environment
|
|
|
|
### 7.1. Recommended Development Tools
|
|
|
|
* **Text Editor:** Use a text editor with syntax highlighting and code completion for Bash (e.g., VS Code, Sublime Text, Vim).
|
|
* **Shellcheck:** Use Shellcheck to lint your Bash code.
|
|
* **Bats:** Use Bats for unit testing.
|
|
* **Bashdb:** Use Bashdb for interactive debugging.
|
|
* **Git:** Use Git for version control.
|
|
|
|
### 7.2. Build Configuration
|
|
|
|
* **Makefile:** Use a Makefile to automate build and test processes.
|
|
* **Dependency Management:** Use a dependency management tool like `dep` to manage external dependencies.
|
|
|
|
### 7.3. Linting and Formatting
|
|
|
|
* **Shellcheck:** Use Shellcheck to lint your Bash code and enforce coding standards.
|
|
* **shfmt:** Use shfmt for automatic code formatting.
|
|
|
|
### 7.4. Deployment Best Practices
|
|
|
|
* **Package Scripts:** Package your scripts and dependencies into a deployable package (e.g., Debian package, RPM package).
|
|
* **Configuration Management:** Use configuration management tools (e.g., Ansible, Puppet, Chef) to deploy and configure your scripts on target systems.
|
|
* **Automated Deployment:** Use automated deployment pipelines to deploy your scripts quickly and reliably.
|
|
* **Idempotent Scripts:** Ensure your deployment scripts are idempotent, meaning they can be run multiple times without causing unintended side effects.
|
|
|
|
### 7.5. CI/CD Integration
|
|
|
|
* **Continuous Integration (CI):** Integrate your Bash scripts into a CI pipeline to automatically run tests and linting on every commit.
|
|
* **Continuous Deployment (CD):** Integrate your Bash scripts into a CD pipeline to automatically deploy your scripts to production.
|
|
|
|
## 8. Naming Conventions
|
|
|
|
Follow these naming conventions:
|
|
|
|
* **Function Names:** Lowercase, with underscores to separate words (e.g., `process_data`, `calculate_total`).
|
|
* **Variable Names:** Lowercase, with underscores to separate words (e.g., `input_file`, `output_directory`). Loop variables should be similarly named for the variable you're looping through.
|
|
* **Constant Names:** All caps, separated with underscores (e.g., `MAX_RETRIES`, `DEFAULT_TIMEOUT`). Declared at the top of the file.
|
|
* **File Names:** Lowercase, with underscores to separate words (e.g., `data_processing.sh`, `configuration_settings.conf`).
|
|
|
|
## 9. Style and Formatting
|
|
|
|
* **Indentation:** Use 2 spaces for indentation. No tabs.
|
|
* **Line Length:** Limit lines to 80 characters. Use line continuation characters (`\`) to break long lines.
|
|
* **Spacing:** Use spaces around operators and after commas.
|
|
* **Comments:** Write clear and concise comments to explain the purpose and logic of your code. Any function that is not both obvious and short must be commented. Any function in a library must be commented regardless of length or complexity. Use `TODO` comments to mark code that needs to be improved or completed.
|
|
* **Quoting:** Always quote variables unless careful unquoted expansion is required.
|
|
* **Braces:** Use `${variable}` instead of `$variable` for clarity.
|
|
* **Pipelines:** Format pipelines for maximum clarity. Break long pipelines into multiple lines with one pipeline element per line.
|
|
|
|
## 10. Exit Codes
|
|
* Ensure exit codes are consistent throughout the scripts.
|
|
* Use `exit 0` for successful completion.
|
|
* Use non-zero exit codes (e.g., 1, 2, 127, etc.) to indicate failure.
|
|
* Standardize exit codes for specific error conditions across your project.
|
|
* Include custom error messages with meaningful context.
|
|
|
|
Following these best practices and coding standards will help you write more maintainable, readable, secure, and efficient Bash scripts. |