--- description: This rule provides a comprehensive set of best practices for developing Go applications, covering code organization, performance, security, testing, and common pitfalls. globs: *.go --- - # Go Best Practices This document outlines best practices for developing Go applications, covering various aspects of the development lifecycle. - ## 1. Code Organization and Structure - ### 1.1 Directory Structure - **Recommended Structure:** project-name/ ├── cmd/ │ └── project-name/ │ └── main.go # Application entry point ├── internal/ │ ├── app/ # Application-specific business logic │ ├── domain/ # Core domain logic and types │ └── pkg/ # Reusable internal packages ├── pkg/ # External packages (libraries for other projects) ├── api/ # API definitions (protobuf, OpenAPI specs) ├── web/ # Web assets (HTML, CSS, JavaScript) ├── scripts/ # Build, deployment, or utility scripts ├── configs/ # Configuration files ├── .gitignore ├── go.mod ├── go.sum └── README.md - **Explanation:** - `cmd`: Contains the main applications for your project. Each subdirectory should represent a separate application. - `internal`: Holds code that's private to your application. Other projects shouldn't import these. - `internal/app`: High-level application logic. - `internal/domain`: Core business logic, data models, and interfaces. - `internal/pkg`: Reusable utilities and helpers within the internal codebase. - `pkg`: Contains reusable libraries that can be used by other projects. Use this for code you want to share. - `api`: Defines API contracts (e.g., Protocol Buffers or OpenAPI/Swagger definitions). - `web`: Stores static web assets like HTML, CSS, and JavaScript files. - `scripts`: Contains scripts for building, testing, deploying, and other tasks. - `configs`: Houses configuration files for various environments. - ### 1.2 File Naming Conventions - **General:** Use lowercase and snake_case for file names (e.g., `user_service.go`). - **Test Files:** Append `_test.go` to the name of the file being tested (e.g., `user_service_test.go`). - **Main Package:** The file containing the `main` function is typically named `main.go`. - ### 1.3 Module Organization - **Go Modules:** Use Go modules for dependency management. Initialize a module with `go mod init `. The module name should reflect the repository path (e.g., `github.com/your-username/project-name`). - **Versioning:** Follow semantic versioning (SemVer) for your modules. Use tags in your Git repository to represent releases (e.g., `v1.0.0`). - **Vendoring:** Consider vendoring dependencies using `go mod vendor` to ensure reproducible builds, especially for critical applications. However, be mindful of vendor directory size. - ### 1.4 Component Architecture - **Layered Architecture:** Structure your application into layers (e.g., presentation, service, repository, data access). This promotes separation of concerns and testability. - **Clean Architecture:** A variation of layered architecture that emphasizes dependency inversion and testability. Core business logic should not depend on implementation details. - **Microservices:** For larger applications, consider a microservices architecture where different parts of the application are deployed as independent services. - **Dependency Injection:** Use dependency injection to decouple components and make them easier to test. Frameworks like `google/wire` or manual dependency injection can be used. - ### 1.5 Code Splitting - **Package Organization:** Group related functionality into packages. Each package should have a clear responsibility. Keep packages small and focused. - **Interface Abstraction:** Use interfaces to define contracts between components. This allows you to swap implementations without changing the code that depends on the interface. - **Functional Options Pattern:** For functions with many optional parameters, use the functional options pattern to improve readability and maintainability. go type Server struct { Addr string Port int Protocol string Timeout time.Duration } type Option func(*Server) func WithAddress(addr string) Option { return func(s *Server) { s.Addr = addr } } func WithPort(port int) Option { return func(s *Server) { s.Port = port } } func NewServer(options ...Option) *Server { srv := &Server{ Addr: "localhost", Port: 8080, Protocol: "tcp", Timeout: 30 * time.Second, } for _, option := range options { option(srv) } return srv } // Usage server := NewServer(WithAddress("127.0.0.1"), WithPort(9000)) - ## 2. Common Patterns and Anti-patterns - ### 2.1 Design Patterns - **Factory Pattern:** Use factory functions to create instances of complex objects. - **Strategy Pattern:** Define a family of algorithms and encapsulate each one in a separate class, making them interchangeable. - **Observer Pattern:** Define a one-to-many dependency between objects so that when one object changes state, all its dependents are notified and updated automatically. - **Context Pattern:** Use the `context` package to manage request-scoped values, cancellation signals, and deadlines. Pass `context.Context` as the first argument to functions that perform I/O or long-running operations. go func handleRequest(ctx context.Context, req *http.Request) { select { case <-ctx.Done(): // Operation cancelled return default: // Process the request } } - **Middleware Pattern:** Chain functions to process HTTP requests. Middleware can be used for logging, authentication, authorization, and other cross-cutting concerns. go func loggingMiddleware(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { log.Printf("Request: %s %s", r.Method, r.URL.Path) next.ServeHTTP(w, r) }) } - ### 2.2 Recommended Approaches for Common Tasks - **Configuration Management:** Use a library like `spf13/viper` or `joho/godotenv` to load configuration from files, environment variables, and command-line flags. - **Logging:** Use a structured logging library like `sirupsen/logrus` or `uber-go/zap` to log events with context and severity levels. - **Database Access:** Use the `database/sql` package with a driver for your specific database (e.g., `github.com/lib/pq` for PostgreSQL, `github.com/go-sql-driver/mysql` for MySQL). Consider an ORM like `gorm.io/gorm` for more complex database interactions. Use prepared statements to prevent SQL injection. - **HTTP Handling:** Use the `net/http` package for building HTTP servers and clients. Consider using a framework like `gin-gonic/gin` or `go-chi/chi` for more advanced routing and middleware features. Always set appropriate timeouts. - **Asynchronous Tasks:** Use goroutines and channels to perform asynchronous tasks. Use wait groups to synchronize goroutines. - **Input Validation:** Use libraries like `go-playground/validator` for validating input data. Always sanitize user input to prevent injection attacks. - ### 2.3 Anti-patterns and Code Smells - **Ignoring Errors:** Never ignore errors. Always handle errors explicitly, even if it's just logging them. go // Bad _, _ = fmt.Println("Hello, world!") // Good _, err := fmt.Println("Hello, world!") if err != nil { log.Println("Error printing: ", err) } - **Panic Usage:** Avoid using `panic` for normal error handling. Use it only for truly exceptional situations where the program cannot continue. - **Global Variables:** Minimize the use of global variables. Prefer passing state explicitly as function arguments. - **Shadowing Variables:** Avoid shadowing variables, where a variable in an inner scope has the same name as a variable in an outer scope. This can lead to confusion and bugs. - **Unbuffered Channels:** Be careful when using unbuffered channels. They can easily lead to deadlocks if not used correctly. - **Overusing Goroutines:** Don't launch too many goroutines, as it can lead to excessive context switching and resource consumption. Consider using a worker pool to limit the number of concurrent goroutines. - **Mutable Global State:** Avoid modifying global state, especially concurrently, as it can introduce race conditions. - **Magic Numbers/Strings:** Avoid using hardcoded numbers or strings directly in your code. Define them as constants instead. - **Long Functions:** Keep functions short and focused. If a function is too long, break it down into smaller, more manageable functions. - **Deeply Nested Code:** Avoid deeply nested code, as it can be difficult to read and understand. Use techniques like early returns and helper functions to flatten the code structure. - ### 2.4 State Management - **Local State:** For simple components, manage state locally within the component using variables. - **Shared State:** When multiple goroutines need to access and modify shared state, use synchronization primitives like mutexes, read-write mutexes, or atomic operations to prevent race conditions. go var mu sync.Mutex var counter int func incrementCounter() { mu.Lock() defer mu.Unlock() counter++ } - **Channels for State Management:** Use channels to pass state between goroutines. This can be a safer alternative to shared memory and locks. - **Context for Request-Scoped State:** Use `context.Context` to pass request-scoped state, such as user authentication information or transaction IDs. - **External Stores (Redis, Databases):** For persistent state or state that needs to be shared across multiple services, use an external store like Redis or a database. - ### 2.5 Error Handling Patterns - **Explicit Error Handling:** Go treats errors as values. Always check for errors and handle them appropriately. - **Error Wrapping:** Wrap errors with context information to provide more details about where the error occurred. Use `fmt.Errorf` with `%w` verb to wrap errors. go func readFile(filename string) ([]byte, error) { data, err := ioutil.ReadFile(filename) if err != nil { return nil, fmt.Errorf("failed to read file %s: %w", filename, err) } return data, nil } - **Error Types:** Define custom error types to represent specific error conditions. This allows you to handle errors more precisely. go type NotFoundError struct { Resource string } func (e *NotFoundError) Error() string { return fmt.Sprintf("%s not found", e.Resource) } - **Sentinel Errors:** Define constant errors that can be compared directly using `==`. This is simpler than error types but less flexible. go var ErrNotFound = errors.New("not found") func getUser(id int) (*User, error) { if id == 0 { return nil, ErrNotFound } // ... } - **Error Grouping:** Use libraries like `go.uber.org/multierr` to collect multiple errors and return them as a single error. - **Defers for Resource Cleanup:** Use `defer` to ensure that resources are cleaned up, even if an error occurs. go func processFile(filename string) error { file, err := os.Open(filename) if err != nil { return err } defer file.Close() // Ensure file is closed // ... } - ## 3. Performance Considerations - ### 3.1 Optimization Techniques - **Profiling:** Use the `pprof` package to profile your application and identify performance bottlenecks. `go tool pprof` allows you to analyze CPU and memory usage. bash go tool pprof http://localhost:6060/debug/pprof/profile # CPU profiling go tool pprof http://localhost:6060/debug/pprof/heap # Memory profiling - **Benchmarking:** Use the `testing` package to benchmark critical sections of your code. go func BenchmarkFunction(b *testing.B) { for i := 0; i < b.N; i++ { // Code to benchmark } } - **Efficient Data Structures:** Choose the right data structures for your needs. For example, use `sync.Map` for concurrent access to maps. - **String Concatenation:** Use `strings.Builder` for efficient string concatenation, especially in loops. go var sb strings.Builder for i := 0; i < 1000; i++ { sb.WriteString("hello") } result := sb.String() - **Reduce Allocations:** Minimize memory allocations, as garbage collection can be expensive. Reuse buffers and objects when possible. - **Inline Functions:** Use the `//go:inline` directive to inline frequently called functions. However, use this sparingly, as it can increase code size. - **Escape Analysis:** Understand how Go's escape analysis works to minimize heap allocations. Values that don't escape to the heap are allocated on the stack, which is faster. - **Compiler Optimizations:** Experiment with compiler flags like `-gcflags=-S` to see the generated assembly code and understand how the compiler is optimizing your code. - **Caching:** Implement caching strategies to reduce database or network calls. Use in-memory caches like `lru` or distributed caches like Redis. - ### 3.2 Memory Management - **Garbage Collection Awareness:** Be aware of how Go's garbage collector works. Understand the trade-offs between memory usage and CPU usage. - **Reduce Heap Allocations:** Try to allocate memory on the stack whenever possible to avoid the overhead of garbage collection. - **Object Pooling:** Use object pooling to reuse frequently created and destroyed objects. This can reduce the number of allocations and improve performance. - **Slices vs. Arrays:** Understand the difference between slices and arrays. Slices are dynamically sized and backed by an array. Arrays have a fixed size. Slices are generally more flexible, but arrays can be more efficient in some cases. - **Copying Data:** Be mindful of copying data, especially large data structures. Use pointers to avoid unnecessary copies. - ### 3.3 Rendering Optimization (if applicable) - This section is less relevant for back-end Go applications. If your Go application serves HTML templates: - **Template Caching:** Cache parsed templates to avoid reparsing them on every request. - **Efficient Template Engine:** Use an efficient template engine like `html/template` from the standard library. - **Minimize DOM Manipulations (if using JavaScript):** Reduce the number of DOM manipulations in your JavaScript code, as they can be expensive. - ### 3.4 Bundle Size Optimization (if applicable) - This section is mostly irrelevant for back-end Go applications. If your Go application serves static assets: - **Minification:** Minify your CSS and JavaScript files to reduce their size. - **Compression:** Compress your assets using Gzip or Brotli. - **Code Splitting (JavaScript):** Split your JavaScript code into smaller chunks that can be loaded on demand. - ### 3.5 Lazy Loading (if applicable) - This is mostly relevant for front-end applications, or database connections: - **Database Connections:** Only establish database connections when they are needed. - **Expensive Resources:** Load expensive resources (e.g., images, large data structures) only when they are actually used. - ## 4. Security Best Practices - ### 4.1 Common Vulnerabilities - **SQL Injection:** Prevent SQL injection by using parameterized queries or an ORM that automatically escapes user input. - **Cross-Site Scripting (XSS):** If your Go application renders HTML, prevent XSS by escaping user input before rendering it. - **Cross-Site Request Forgery (CSRF):** Protect against CSRF attacks by using CSRF tokens. - **Command Injection:** Avoid executing external commands directly with user input. If you must, sanitize the input carefully. - **Path Traversal:** Prevent path traversal attacks by validating and sanitizing file paths provided by users. - **Denial of Service (DoS):** Protect against DoS attacks by setting appropriate timeouts and resource limits. Use rate limiting to prevent abuse. - **Authentication and Authorization Issues:** Implement robust authentication and authorization mechanisms to protect sensitive data and functionality. - **Insecure Dependencies:** Regularly audit your dependencies for known vulnerabilities. Use tools like `govulncheck` to identify vulnerabilities. - ### 4.2 Input Validation - **Validate All Input:** Validate all input data, including user input, API requests, and data from external sources. - **Use Validation Libraries:** Use validation libraries like `go-playground/validator` to simplify input validation. - **Sanitize Input:** Sanitize user input to remove potentially harmful characters or code. - **Whitelist vs. Blacklist:** Prefer whitelisting allowed values over blacklisting disallowed values. - **Regular Expressions:** Use regular expressions to validate complex input formats. - ### 4.3 Authentication and Authorization - **Use Strong Authentication:** Use strong authentication mechanisms like multi-factor authentication (MFA). - **Password Hashing:** Hash passwords using a strong hashing algorithm like bcrypt or Argon2. - **JWT (JSON Web Tokens):** Use JWT for stateless authentication. Verify the signature of JWTs before trusting them. - **RBAC (Role-Based Access Control):** Implement RBAC to control access to resources based on user roles. - **Least Privilege:** Grant users only the minimum privileges necessary to perform their tasks. - **OAuth 2.0:** Use OAuth 2.0 for delegated authorization, allowing users to grant third-party applications access to their data without sharing their credentials. - ### 4.4 Data Protection - **Encryption:** Encrypt sensitive data at rest and in transit. - **TLS (Transport Layer Security):** Use TLS to encrypt communication between clients and servers. - **Data Masking:** Mask sensitive data in logs and displays. - **Regular Backups:** Regularly back up your data to prevent data loss. - **Access Control:** Restrict access to sensitive data to authorized personnel only. - **Data Minimization:** Collect only the data that is necessary for your application. - ### 4.5 Secure API Communication - **HTTPS:** Use HTTPS for all API communication. - **API Keys:** Use API keys to authenticate clients. - **Rate Limiting:** Implement rate limiting to prevent abuse. - **Input Validation:** Validate all input data to prevent injection attacks. - **Output Encoding:** Encode output data appropriately to prevent XSS attacks. - **CORS (Cross-Origin Resource Sharing):** Configure CORS properly to allow requests from trusted origins only. - ## 5. Testing Approaches - ### 5.1 Unit Testing - **Focus on Individual Units:** Unit tests should focus on testing individual functions, methods, or packages in isolation. - **Table-Driven Tests:** Use table-driven tests to test multiple inputs and outputs for a single function. go func TestAdd(t *testing.T) { testCases := []struct { a, b int expected int }{ {1, 2, 3}, {0, 0, 0}, {-1, 1, 0}, } for _, tc := range testCases { result := Add(tc.a, tc.b) if result != tc.expected { t.Errorf("Add(%d, %d) = %d; expected %d", tc.a, tc.b, result, tc.expected) } } } - **Test Coverage:** Aim for high test coverage. Use `go test -cover` to measure test coverage. - **Clear Assertions:** Use clear and informative assertions. Libraries like `testify` provide helpful assertion functions. - **Test Naming:** Use descriptive test names that clearly indicate what is being tested. - ### 5.2 Integration Testing - **Test Interactions Between Components:** Integration tests should test the interactions between different components of your application. - **Use Real Dependencies (where possible):** Use real dependencies (e.g., real databases) in integration tests, where possible. This provides more realistic testing. - **Mock External Services:** Mock external services that are not under your control. - **Test Data Setup and Teardown:** Set up test data before each test and tear it down after each test to ensure that tests are independent. - ### 5.3 End-to-End Testing - **Test the Entire Application:** End-to-end tests should test the entire application, from the user interface to the backend. - **Automated Browser Testing:** Use automated browser testing tools like Selenium or Cypress to simulate user interactions. - **Test Real-World Scenarios:** Test real-world scenarios to ensure that the application works as expected in production. - **Data Persistence:** Be careful of data persistence between tests. Clean up any generated data after each test run. - ### 5.4 Test Organization - **Test Files:** Place test files in the same directory as the code being tested. Use the `_test.go` suffix. - **Package Tests:** Write tests for each package in your application. - **Test Suites:** Use test suites to group related tests together. - ### 5.5 Mocking and Stubbing - **Interfaces for Mocking:** Use interfaces to define contracts between components, making it easier to mock dependencies. - **Mocking Libraries:** Use mocking libraries like `gomock` or `testify/mock` to generate mocks for interfaces. go //go:generate mockgen -destination=mocks/mock_user_repository.go -package=mocks github.com/your-username/project-name/internal/domain UserRepository type UserRepository interface { GetUser(id int) (*User, error) } - **Stubbing:** Use stubs to replace dependencies with simple, predefined responses. - **Avoid Over-Mocking:** Don't over-mock your code. Mock only the dependencies that are necessary to isolate the unit being tested. - ## 6. Common Pitfalls and Gotchas - ### 6.1 Frequent Mistakes - **Nil Pointer Dereferences:** Be careful of nil pointer dereferences. Always check for nil before accessing a pointer. - **Data Races:** Avoid data races by using synchronization primitives like mutexes or channels. - **Deadlocks:** Be careful of deadlocks when using goroutines and channels. Ensure that channels are closed properly and that goroutines are not waiting on each other indefinitely. - **For Loop Variable Capture:** Be careful when capturing loop variables in goroutines. The loop variable may change before the goroutine is executed. Copy the loop variable to a local variable before passing it to the goroutine. go for _, item := range items { item := item // Copy loop variable to local variable go func() { // Use local variable item }() } - **Incorrect Type Conversions:** Be careful when converting between types. Ensure that the conversion is valid and that you handle potential errors. - **Incorrect Error Handling:** Ignoring or mishandling errors is a common pitfall. Always check errors and handle them appropriately. - **Over-reliance on Global State:** Using global variables excessively leads to tight coupling and makes code difficult to test and reason about. - ### 6.2 Edge Cases - **Integer Overflow:** Be aware of integer overflow when performing arithmetic operations. - **Floating-Point Precision:** Be aware of the limitations of floating-point precision. - **Time Zones:** Be careful when working with time zones. Use the `time` package to handle time zones correctly. - **Unicode Handling:** Be careful when handling Unicode characters. Use the `unicode/utf8` package to correctly encode and decode UTF-8 strings. - ### 6.3 Version-Specific Issues - **Go 1.18 Generics:** Understand how generics work in Go 1.18 and later versions. Use them judiciously to improve code reusability and type safety. - **Module Compatibility:** Be aware of compatibility issues between different versions of Go modules. Use `go mod tidy` to update your dependencies and resolve compatibility issues. - ### 6.4 Compatibility Concerns - **C Interoperability:** Be aware of the complexities of C interoperability when using the `cgo` tool. Ensure that memory is managed correctly and that there are no data races. - **Operating System Differences:** Be aware of differences between operating systems (e.g., file path separators, environment variables). Use the `os` package to handle operating system-specific behavior. - ### 6.5 Debugging Strategies - **Print Statements:** Use `fmt.Println` or `log.Println` to print debugging information. - **Delve Debugger:** Use the Delve debugger (`dlv`) to step through your code and inspect variables. bash dlv debug ./cmd/your-application - **pprof Profiling:** Use the `pprof` package to profile your application and identify performance bottlenecks. - **Race Detector:** Use the race detector (`go run -race`) to identify data races in your code. - **Logging:** Add detailed logging to your application to help diagnose issues in production. - **Core Dumps:** Generate core dumps when your application crashes to help diagnose the cause of the crash. - **Code Reviews:** Have your code reviewed by other developers to catch potential issues. - ## 7. Tooling and Environment - ### 7.1 Recommended Development Tools - **GoLand:** A commercial IDE from JetBrains with excellent Go support. - **Visual Studio Code:** A free and open-source editor with Go support via the Go extension. - **Vim:** A powerful text editor with Go support via plugins. - **gopls:** The official Go language server, providing features like code completion, linting, and formatting. - ### 7.2 Build Configuration - **Makefile:** Use a Makefile to automate build and deployment tasks. makefile build: go build -o bin/your-application ./cmd/your-application run: go run ./cmd/your-application test: go test ./... - **GoReleaser:** Use GoReleaser to automate the release process, including building binaries for multiple platforms, generating checksums, and creating release notes. - **Docker:** Use Docker to containerize your application for easy deployment. dockerfile FROM golang:1.21-alpine AS builder WORKDIR /app COPY go.mod go.sum ./ RUN go mod download COPY . . RUN go build -o /bin/your-application ./cmd/your-application FROM alpine:latest WORKDIR /app COPY --from=builder /bin/your-application . CMD ["./your-application"] - ### 7.3 Linting and Formatting - **gofmt:** Use `gofmt` to automatically format your Go code according to the standard style guidelines. Run it regularly to keep your code consistent. bash gofmt -s -w . - **golint:** Use `golint` to check your code for style and potential issues. - **staticcheck:** Use `staticcheck` for more comprehensive static analysis. - **revive:** A fast, configurable, extensible, flexible, and beautiful linter for Go. - **errcheck:** Use `errcheck` to ensure that you are handling all errors. - **.golangci.yml:** Use a `.golangci.yml` file to configure `golangci-lint` with your preferred linting rules. - ### 7.4 Deployment - **Cloud Platforms:** Deploy your application to cloud platforms like AWS, Google Cloud, or Azure. - **Kubernetes:** Deploy your application to Kubernetes for scalability and high availability. - **Systemd:** Use systemd to manage your application as a service on Linux systems. - **Serverless Functions:** Consider using serverless functions for small, event-driven applications. - ### 7.5 CI/CD Integration - **GitHub Actions:** Use GitHub Actions to automate your CI/CD pipeline. - **GitLab CI:** Use GitLab CI to automate your CI/CD pipeline. - **Jenkins:** Use Jenkins to automate your CI/CD pipeline. - **CircleCI:** Use CircleCI to automate your CI/CD pipeline. - **Automated Testing:** Run unit tests, integration tests, and end-to-end tests automatically as part of your CI/CD pipeline. - **Automated Deployment:** Automate the deployment process to reduce the risk of human error. - **Infrastructure as Code:** Use Infrastructure as Code (IaC) tools like Terraform or CloudFormation to automate the provisioning and management of your infrastructure.