Virtnet.bond/.cursor/rules/peewee.mdc

---
description: Comprehensive guide for Peewee ORM best practices, covering code organization, performance, security, testing, and common pitfalls. Provides actionable guidance for developers to write maintainable and efficient database-driven applications using Peewee.
globs: *.py
---
# Peewee ORM Best Practices and Coding Standards

This document outlines best practices and coding standards for using Peewee ORM in Python development. Following these guidelines will lead to more maintainable, efficient, and secure database-driven applications.

## Library Information:

- Name: peewee
- Tags: database, orm, python, sql

## 1. Code Organization and Structure

### 1.1. Directory Structure


project_root/
├── models/
│   ├── __init__.py
│   ├── user.py
│   ├── product.py
│   └── ...
├── migrations/
│   ├── 0001_initial.py
│   ├── 0002_add_index.py
│   └── ...
├── utils/
│   ├── db.py  # Database connection and initialization
│   └── ...
├── tests/
│   ├── __init__.py
│   ├── test_user.py
│   ├── test_product.py
│   └── ...
├── main.py  # Application entry point
└── requirements.txt


- **models/:** Contains model definitions, each in a separate file.
- **migrations/:** Stores database migration scripts.
- **utils/:** Utility functions, including database connection management.
- **tests/:** Unit and integration tests.
- **main.py:** The main application file.

### 1.2. File Naming Conventions

- Model files: Use lowercase, singular nouns (e.g., `user.py`, `product.py`).
- Migration files: Use a sequential numbering scheme (e.g., `0001_initial.py`, `0002_add_index.py`).
- Test files: Prefix with `test_` (e.g., `test_user.py`).
- Utility files: Use descriptive names (e.g., `db.py`).

### 1.3. Module Organization

- Group related models into the same module.  For example, models related to user management would reside in `models/user.py`.
- Use `__init__.py` files to make directories importable as packages and to provide a convenient way to import all models from the `models` directory.  Example:

python
# models/__init__.py
from .user import User
from .product import Product


### 1.4. Component Architecture

- Separate database interactions from business logic. Create service layers or repositories that handle Peewee queries and data manipulation, keeping models thin and focused on schema definition.
- Favor composition over inheritance where appropriate, using mixins for reusable model functionality.

### 1.5. Code Splitting

- Use separate files for different model definitions to improve readability and maintainability.
- Decompose complex queries into smaller, reusable functions.
- Consider using submodules within `models/` for logical groupings of models when projects grow large.

## 2. Common Patterns and Anti-patterns

### 2.1. Design Patterns

- **Repository Pattern:** Create a layer between the application and the database to abstract data access logic.  This makes it easier to switch databases or change ORM implementations later.
- **Unit of Work:** Manage database transactions within a single unit of work, ensuring consistency and atomicity. Peewee's `atomic()` context manager facilitates this pattern.
- **Data Mapper:** Maps data between domain objects and the database. Peewee models inherently implement this pattern.

### 2.2. Recommended Approaches

- **Explicit Database Connection Management:** Always open and close database connections explicitly, especially in web applications or concurrent environments. Use `with db:` or `@db.connection_context()` to ensure connections are properly handled.  Disable `autoconnect` when initializing the database:

python
db = SqliteDatabase('my_app.db', autoconnect=False)


- **Base Model Class:** Define a base model class that all other models inherit from. This centralizes database configuration and ensures consistency:

python
from peewee import * 

db = SqliteDatabase('my_app.db')

class BaseModel(Model):
    class Meta:
        database = db


- **Relationships and Joins:** Utilize Peewee's built-in support for relationships and joins to simplify complex queries:

python
class User(BaseModel):
    username = CharField()

class Tweet(BaseModel):
    user = ForeignKeyField(User, backref='tweets')
    content = TextField()

query = Tweet.select().join(User).where(User.username == 'john_doe')


- **Migrations:** Use a migration library (like `peewee-migrate`) to manage schema changes effectively.  This ensures that database schemas can evolve smoothly over time.

### 2.3. Anti-patterns and Code Smells

- **Implicit Database Connections:** Relying on Peewee's `autoconnect` feature can lead to unpredictable behavior and connection leaks. Manage connections explicitly.
- **Global Database Instance:** Avoid creating a single global database instance without proper connection management, especially in multi-threaded applications. Use connection pools or context managers to manage connections per request or thread.
- **String Interpolation:** Never use string interpolation to construct SQL queries, as this can lead to SQL injection vulnerabilities. Always use parameterized queries.

python
# Anti-pattern (SQL injection vulnerability!)
username = input("Enter username:")
query = User.select().where(SQL("username = '%s'" % username))

# Correct approach (parameterized query)
query = User.select().where(User.username == username)


- **Over-fetching Data:** Avoid selecting all columns when only a subset is needed. Specify the required fields in the `select()` method.

### 2.4. State Management

- In web applications, manage database connections on a per-request basis. Open a connection at the beginning of the request and close it at the end. Framework integration examples are provided in the base documentation.
- In asynchronous applications, utilize connection pools and context managers to manage connections concurrently.

### 2.5. Error Handling

- Use try-except blocks to handle database errors, such as `IntegrityError` (for unique constraint violations) and `DoesNotExist` (when a record is not found).
- Implement logging to track database errors and diagnose issues.
- Provide informative error messages to the user, without exposing sensitive database details.

## 3. Performance Considerations

### 3.1. Optimization Techniques

- **Indexing:** Add indexes to frequently queried columns to improve query performance.  Use Peewee's indexing features when defining your models or use migrations to create them:

python
class User(BaseModel):
    username = CharField(index=True) # Simple index

    class Meta:
        indexes = (
            (('email', 'is_active'), True),  # Composite index (unique=True)
        )


- **Caching:** Implement caching for frequently accessed data to reduce database load. Consider using a caching library like Redis or Memcached.
- **Connection Pooling:** Use connection pooling to reduce the overhead of establishing new database connections for each request.  Peewee's `playhouse.pool` module provides pooled database classes:

python
from playhouse.pool import PooledSqliteDatabase

db = PooledSqliteDatabase('my_app.db', max_connections=16)


- **Batch Inserts/Updates:** Use `insert_many()` and `update()` methods for bulk operations instead of iterating and executing individual queries.

python
data = [
    {'username': 'user1', 'email': 'user1@example.com'},
    {'username': 'user2', 'email': 'user2@example.com'},
]

with db.atomic(): # Transaction for atomicity
    User.insert_many(data).execute()


- **Query Optimization:**  Use `.prefetch()` to reduce N+1 query problems. Use `.defer()` to avoid fetching unnecessary columns. Profile your queries and analyze execution plans to identify bottlenecks.

### 3.2. Memory Management

- Limit the number of records returned by queries to avoid loading large amounts of data into memory. Use pagination or filtering to retrieve data in smaller chunks.
- Close database connections promptly after use to release resources.
- Be mindful of large data types (e.g., BLOBs) and handle them efficiently to avoid memory exhaustion.

### 3.3. Bundle Size Optimization

- Peewee is a lightweight library, so its direct impact on bundle size is minimal. However, be mindful of dependencies introduced by database drivers or extensions.
- Use a minifier to reduce the size of your Python code before deployment.

### 3.4. Lazy Loading

-  Use `ForeignKeyField` with caution, understanding that accessing the related object will trigger a new database query if the related object is not prefetched. Utilize `.prefetch()` for efficient loading of related data.

## 4. Security Best Practices

### 4.1. Common Vulnerabilities

- **SQL Injection:**  Prevent SQL injection by using parameterized queries and avoiding string concatenation when constructing SQL statements.
- **Cross-Site Scripting (XSS):**  If displaying data retrieved from the database in a web application, sanitize the data to prevent XSS attacks. (This is primarily a front-end concern, but relevant when handling user-generated content stored in the database).
- **Sensitive Data Exposure:**  Avoid storing sensitive data (e.g., passwords, API keys) in plain text. Use proper encryption or hashing techniques. Peewee itself does not provide encryption; use appropriate Python libraries for this (like `bcrypt` for passwords).

### 4.2. Input Validation

- Validate all user inputs before using them in database queries or model fields. This prevents malicious data from being stored in the database or used to exploit vulnerabilities.
- Use Peewee's field validation capabilities to enforce data types, lengths, and other constraints at the model level:

python
class User(BaseModel):
    username = CharField(max_length=50)
    email = CharField(unique=True)
    age = IntegerField(null=True, constraints=[Check('age > 0')]) #CHECK constraint



### 4.3. Authentication and Authorization

- Implement robust authentication and authorization mechanisms to protect sensitive data and restrict access to authorized users only. Peewee can integrate with authentication frameworks like Flask-Login or Django's authentication system.
- Do not rely solely on client-side validation for security. Always perform server-side validation.

### 4.4. Data Protection

- Implement data encryption at rest and in transit to protect sensitive data from unauthorized access.  Consider using database-level encryption or encrypting sensitive fields at the application level.
- Use HTTPS to secure communication between the client and the server.
- Comply with relevant data privacy regulations (e.g., GDPR, CCPA).

### 4.5. Secure API Communication

- Use API keys or tokens to authenticate API requests.
- Implement rate limiting to prevent abuse and denial-of-service attacks.
- Validate API request data and sanitize API responses to prevent vulnerabilities.

## 5. Testing Approaches

### 5.1. Unit Testing

- Unit test model methods, query logic, and other database-related functionality in isolation. Use mocking and stubbing to isolate units of code from external dependencies (e.g., the database itself).
- Utilize an in-memory SQLite database for unit tests to avoid modifying the production database. Bind the models to the in-memory database during testing.

python
import unittest
from peewee import *

db = SqliteDatabase(':memory:')

class BaseModel(Model):
    class Meta:
        database = db

class User(BaseModel):
    username = CharField()

class TestUser(unittest.TestCase):
    def setUp(self):
        db.bind([User])
        db.create_tables([User])

    def tearDown(self):
        db.drop_tables([User])
        db.close()

    def test_user_creation(self):
        user = User.create(username='test_user')
        self.assertEqual(user.username, 'test_user')

if __name__ == '__main__':
    unittest.main()


### 5.2. Integration Testing

- Integration test the interaction between different components of the application, including database interactions.
- Use a separate test database (e.g., a dedicated SQLite file or a testing instance of your production database) for integration tests.
- Ensure that the test database is properly initialized and cleaned up before and after each test.

### 5.3. End-to-End Testing

- End-to-end tests verify the entire application workflow, including database interactions, from the user's perspective.
- Use tools like Selenium or Cypress to automate end-to-end tests.
- Ensure that the end-to-end tests cover critical business scenarios and data flows.

### 5.4. Test Organization

- Organize tests into separate modules or directories based on the component or functionality being tested.
- Use descriptive test names to clearly indicate the purpose of each test.
- Follow the Arrange-Act-Assert pattern to structure tests.

### 5.5. Mocking and Stubbing

- Use mocking and stubbing to isolate units of code during unit testing and to simulate database behavior.
- Use libraries like `unittest.mock` or `pytest-mock` for mocking and stubbing.
- Mock database connections, queries, and model methods as needed.

## 6. Common Pitfalls and Gotchas

### 6.1. Frequent Mistakes

- **Forgetting to Handle Exceptions:** Neglecting to handle database exceptions can lead to application crashes or data corruption.
- **Incorrect Data Types:** Using incorrect data types for model fields can result in data truncation or validation errors.
- **Not Closing Connections:** Failing to close database connections can lead to connection leaks and performance issues.
- **N+1 Query Problem:** Inefficiently fetching related data can result in the N+1 query problem, where a query is executed for each related record. Use `prefetch()` to mitigate this.

### 6.2. Edge Cases

- **Concurrency Issues:**  Be aware of concurrency issues when multiple threads or processes access the database simultaneously. Use transactions and locking mechanisms to ensure data consistency.
- **Large Datasets:**  Handle large datasets efficiently by using pagination, streaming, or batch processing techniques.
- **Database-Specific Features:**  Be aware of database-specific features and syntax differences when writing queries. Use Peewee's database-specific extensions when necessary.

### 6.3. Version-Specific Issues

- Consult the Peewee documentation and release notes for version-specific issues and compatibility concerns.
- Test your application thoroughly after upgrading Peewee or your database driver.

### 6.4. Compatibility Concerns

- Ensure that the Peewee version is compatible with the Python version and the database driver being used.
- Be aware of potential compatibility issues when integrating Peewee with other libraries or frameworks.

### 6.5. Debugging Strategies

- Enable logging to track database queries and errors.
- Use a database profiler to analyze query performance.
- Use a debugger to step through code and inspect variables.
- Simplify complex queries to isolate the source of the problem.

## 7. Tooling and Environment

### 7.1. Recommended Development Tools

- **Integrated Development Environment (IDE):** VS Code, PyCharm, or other IDEs with Python support.
- **Database Client:** DB Browser for SQLite, pgAdmin, MySQL Workbench, or other database clients for interacting with the database.
- **Migration Tool:** Alembic or `peewee-migrate` for managing database schema changes.

### 7.2. Build Configuration

- Use `requirements.txt` or `pyproject.toml` to manage project dependencies.
- Specify the Peewee version and database driver versions in the dependencies file.
- Use a virtual environment to isolate project dependencies from the system environment.

### 7.3. Linting and Formatting

- Use a linter like Flake8 or Pylint to enforce code style and identify potential errors.
- Use a code formatter like Black to automatically format your code according to a consistent style.
- Configure the linter and formatter to adhere to the Peewee coding standards.

### 7.4. Deployment

- Use a deployment tool like Docker or Kubernetes to containerize and deploy your application.
- Configure the application to connect to the production database using environment variables or configuration files.
- Ensure that the database schema is up-to-date before deploying the application. Run database migrations as part of the deployment process.
- Monitor the application and database for performance and security issues.

### 7.5. CI/CD Integration

- Integrate Peewee tests into your CI/CD pipeline to automatically run tests whenever code is committed or deployed.
- Use a CI/CD tool like Jenkins, GitLab CI, or GitHub Actions to automate the build, test, and deployment process.
- Configure the CI/CD pipeline to run database migrations and seed the database with test data.

This guide provides a comprehensive overview of Peewee ORM best practices. By following these guidelines, you can create robust, efficient, and maintainable database-driven applications.