From 5cd9a5266882f9f62c97546f4e3de4d45555de80 Mon Sep 17 00:00:00 2001 From: "Joseph.Rawlings" Date: Thu, 4 Dec 2025 00:11:26 -0500 Subject: [PATCH] refactor: streamline onboarding modal by removing unnecessary slide preparation logic and enhancing loading state management for improved user experience --- .cursor/rules/actix-web.mdc | 318 +++++++++ .cursor/rules/aiohttp.mdc | 338 +++++++++ .cursor/rules/amazon-ec2.mdc | 318 +++++++++ .cursor/rules/amazon-s3.mdc | 367 ++++++++++ .cursor/rules/android-sdk.mdc | 311 ++++++++ .cursor/rules/angular.mdc | 278 ++++++++ .cursor/rules/ansible.mdc | 553 ++++++++++++++ .cursor/rules/ant-design.mdc | 311 ++++++++ .cursor/rules/anyio.mdc | 169 +++++ .cursor/rules/apollo-client.mdc | 178 +++++ .cursor/rules/apollo-graphql.mdc | 217 ++++++ .cursor/rules/asp-net.mdc | 363 ++++++++++ .cursor/rules/astro.mdc | 176 +++++ .cursor/rules/asyncio.mdc | 371 ++++++++++ .cursor/rules/auth0.mdc | 195 +++++ .cursor/rules/autogen.mdc | 166 +++++ .cursor/rules/aws-amplify.mdc | 286 ++++++++ .cursor/rules/aws-cli.mdc | 349 +++++++++ .cursor/rules/aws-dynamodb.mdc | 182 +++++ .cursor/rules/aws-ecs.mdc | 210 ++++++ .cursor/rules/aws-lambda.mdc | 220 ++++++ .cursor/rules/aws-rds.mdc | 324 +++++++++ .cursor/rules/aws.mdc | 344 +++++++++ .cursor/rules/axios.mdc | 185 +++++ .cursor/rules/azure-pipelines.mdc | 176 +++++ .cursor/rules/azure.mdc | 347 +++++++++ .cursor/rules/bash.mdc | 292 ++++++++ .cursor/rules/beautifulsoup4.mdc | 5 + .cursor/rules/behave.mdc | 4 + .cursor/rules/black.mdc | 134 ++++ .cursor/rules/boto3.mdc | 349 +++++++++ .cursor/rules/bottle.mdc | 279 ++++++++ .cursor/rules/bun.mdc | 310 ++++++++ .cursor/rules/c-sharp.mdc | 368 ++++++++++ .cursor/rules/chakra-ui.mdc | 207 ++++++ .cursor/rules/cheerio.mdc | 265 +++++++ .cursor/rules/circleci.mdc | 231 ++++++ .cursor/rules/clerk.mdc | 236 ++++++ .cursor/rules/click.mdc | 356 ++++++++++ .cursor/rules/cloudflare.mdc | 292 ++++++++ .cursor/rules/codemirror.mdc | 240 +++++++ .cursor/rules/crewai.mdc | 179 +++++ .cursor/rules/css.mdc | 65 ++ .cursor/rules/cuda.mdc | 165 +++++ .cursor/rules/customtkinter.mdc | 230 ++++++ .cursor/rules/cypress.mdc | 345 +++++++++ .cursor/rules/d3.mdc | 5 + .cursor/rules/dask.mdc | 268 +++++++ .cursor/rules/databricks.mdc | 337 +++++++++ .cursor/rules/datadog.mdc | 180 +++++ .cursor/rules/deno.mdc | 268 +++++++ .cursor/rules/detox.mdc | 348 +++++++++ .cursor/rules/digitalocean.mdc | 278 ++++++++ .cursor/rules/discord-api.mdc | 190 +++++ .cursor/rules/django-orm.mdc | 306 ++++++++ .cursor/rules/django-rest-framework.mdc | 336 +++++++++ .cursor/rules/django.mdc | 202 ++++++ .cursor/rules/docker.md | 217 ++++++ .cursor/rules/drizzle.mdc | 274 +++++++ .cursor/rules/duckdb.mdc | 285 ++++++++ .cursor/rules/elasticsearch.mdc | 272 +++++++ .cursor/rules/electron.mdc | 150 ++++ .cursor/rules/elk-stack.mdc | 168 +++++ .cursor/rules/emacs.mdc | 220 ++++++ .cursor/rules/esbuild.mdc | 282 ++++++++ .cursor/rules/eslint.mdc | 5 + .cursor/rules/expo.mdc | 200 ++++++ .cursor/rules/express.mdc | 297 ++++++++ .cursor/rules/fabric-js.mdc | 299 ++++++++ .cursor/rules/fastapi.mdc | 336 +++++++++ .cursor/rules/ffmpeg.mdc | 331 +++++++++ .cursor/rules/fiber.mdc | 4 + .cursor/rules/firebase.mdc | 303 ++++++++ .cursor/rules/flake8.mdc | 276 +++++++ .cursor/rules/flask-restful.mdc | 314 ++++++++ .cursor/rules/flask.mdc | 199 ++++++ .cursor/rules/flutter.mdc | 262 +++++++ .cursor/rules/fontawesome.mdc | 163 +++++ .cursor/rules/gcp-cli.mdc | 85 +++ .cursor/rules/gcp.mdc | 196 +++++ .cursor/rules/gensim.mdc | 285 ++++++++ .cursor/rules/git.mdc | 85 +++ .cursor/rules/github-actions.mdc | 261 +++++++ .cursor/rules/gitlab-ci.mdc | 239 +++++++ .cursor/rules/go.mdc | 556 +++++++++++++++ .cursor/rules/godot.mdc | 358 ++++++++++ .cursor/rules/google-maps-js.mdc | 262 +++++++ .cursor/rules/gradle.mdc | 141 ++++ .cursor/rules/grafana.mdc | 5 + .cursor/rules/graphql.mdc | 55 ++ .cursor/rules/guzzle.mdc | 5 + .cursor/rules/hardhat.mdc | 278 ++++++++ .cursor/rules/heroku.mdc | 118 +++ .cursor/rules/htmx.mdc | 319 +++++++++ .cursor/rules/httpx.mdc | 255 +++++++ .cursor/rules/huggingface.mdc | 342 +++++++++ .cursor/rules/hypothesis.mdc | 164 +++++ .cursor/rules/insomnia.mdc | 91 +++ .cursor/rules/ionic.mdc | 339 +++++++++ .cursor/rules/isort.mdc | 223 ++++++ .cursor/rules/java.mdc | 296 ++++++++ .cursor/rules/jax.mdc | 267 +++++++ .cursor/rules/jenkins.mdc | 194 +++++ .cursor/rules/jest.mdc | 139 ++++ .cursor/rules/jetpack-compose.mdc | 168 +++++ .cursor/rules/jquery.mdc | 410 +++++++++++ .cursor/rules/junit.mdc | 256 +++++++ .cursor/rules/keras.mdc | 323 +++++++++ .cursor/rules/kivy.mdc | 347 +++++++++ .cursor/rules/kubernetes.mdc | 353 +++++++++ .cursor/rules/langchain-js.mdc | 179 +++++ .cursor/rules/langchain.mdc | 310 ++++++++ .cursor/rules/langgraph.mdc | 312 ++++++++ .cursor/rules/{core.mdc => laravel.mdc} | 3 - .cursor/rules/lightgbm.mdc | 297 ++++++++ .cursor/rules/llama-index.mdc | 158 ++++ .cursor/rules/llamaindex-js.mdc | 294 ++++++++ .cursor/rules/llvm.mdc | 131 ++++ .cursor/rules/material-ui.mdc | 273 +++++++ .cursor/rules/matplotlib.mdc | 306 ++++++++ .cursor/rules/maven.mdc | 177 +++++ .cursor/rules/microsoft-teams.mdc | 241 +++++++ .cursor/rules/mkdocs.mdc | 270 +++++++ .cursor/rules/mlx.mdc | 282 ++++++++ .cursor/rules/mobx.mdc | 429 +++++++++++ .cursor/rules/mockito.mdc | 249 +++++++ .cursor/rules/modal.mdc | 227 ++++++ .cursor/rules/mongodb.mdc | 148 ++++ .cursor/rules/mypy.mdc | 101 +++ .cursor/rules/neo4j.mdc | 278 ++++++++ .cursor/rules/nestjs.mdc | 178 +++++ .cursor/rules/netlify.mdc | 293 ++++++++ .cursor/rules/next-js.mdc | 288 ++++++++ .cursor/rules/nginx.mdc | 406 +++++++++++ .cursor/rules/nltk.mdc | 304 ++++++++ .cursor/rules/nose2.mdc | 316 ++++++++ .cursor/rules/notion-api.mdc | 263 +++++++ .cursor/rules/numba.mdc | 299 ++++++++ .cursor/rules/numpy.mdc | 271 +++++++ .cursor/rules/nuxt.mdc | 201 ++++++ .cursor/rules/openai.mdc | 246 +++++++ .cursor/rules/opencv-python.mdc | 199 ++++++ .cursor/rules/pandas.mdc | 175 +++++ .cursor/rules/pdoc.mdc | 243 +++++++ .cursor/rules/peewee.mdc | 365 ++++++++++ .cursor/rules/phoenix.mdc | 270 +++++++ .cursor/rules/php.mdc | 334 +++++++++ .cursor/rules/pillow.mdc | 169 +++++ .cursor/rules/playwright.mdc | 151 ++++ .cursor/rules/plotly.mdc | 343 +++++++++ .cursor/rules/poetry.mdc | 333 +++++++++ .cursor/rules/pony.mdc | 275 +++++++ .cursor/rules/postgresql.mdc | 107 +++ .cursor/rules/postman.mdc | 100 +++ .cursor/rules/prisma.mdc | 190 +++++ .cursor/rules/puppeteer.mdc | 284 ++++++++ .cursor/rules/pydantic.mdc | 207 ++++++ .cursor/rules/pygame.mdc | 321 +++++++++ .cursor/rules/pylint.mdc | 324 +++++++++ .cursor/rules/pyqt.mdc | 120 ++++ .cursor/rules/pyramid.mdc | 176 +++++ .cursor/rules/pyright.mdc | 291 ++++++++ .cursor/rules/pyside.mdc | 237 ++++++ .cursor/rules/pytest.mdc | 335 +++++++++ .cursor/rules/python.mdc | 347 +++++++++ .cursor/rules/pytorch.mdc | 381 ++++++++++ .cursor/rules/qwik.mdc | 204 ++++++ .cursor/rules/railway.mdc | 289 ++++++++ .cursor/rules/react-mobx.mdc | 203 ++++++ .cursor/rules/react-native.mdc | 217 ++++++ .cursor/rules/react-query.mdc | 267 +++++++ .cursor/rules/react-redux.mdc | 283 ++++++++ .cursor/rules/react.mdc | 284 ++++++++ .cursor/rules/redis.mdc | 75 ++ .cursor/rules/redux.mdc | 196 +++++ .cursor/rules/remix.mdc | 315 ++++++++ .cursor/rules/requests.mdc | 267 +++++++ .cursor/rules/rich.mdc | 160 +++++ .cursor/rules/riverpod.mdc | 150 ++++ .cursor/rules/rocket.mdc | 113 +++ .cursor/rules/ros.mdc | 277 ++++++++ .cursor/rules/ruby.mdc | 478 +++++++++++++ .cursor/rules/rust.mdc | 417 +++++++++++ .cursor/rules/sanic.mdc | 223 ++++++ .cursor/rules/scikit-image.mdc | 295 ++++++++ .cursor/rules/scikit-learn.mdc | 193 +++++ .cursor/rules/scipy.mdc | 56 ++ .cursor/rules/scrapy.mdc | 288 ++++++++ .cursor/rules/seaborn.mdc | 5 + .cursor/rules/selenium.mdc | 328 +++++++++ .cursor/rules/sentry.mdc | 146 ++++ .cursor/rules/servemux.mdc | 406 +++++++++++ .cursor/rules/setuptools.mdc | 239 +++++++ .cursor/rules/shadcn.mdc | 192 +++++ .cursor/rules/smolagents.mdc | 286 ++++++++ .cursor/rules/socket-io.mdc | 302 ++++++++ .cursor/rules/solidity.mdc | 308 ++++++++ .cursor/rules/solidjs.mdc | 322 +++++++++ .cursor/rules/spacy.mdc | 317 +++++++++ .cursor/rules/sphinx.mdc | 170 +++++ .cursor/rules/spring.mdc | 208 ++++++ .cursor/rules/springboot.mdc | 339 +++++++++ .cursor/rules/sqlalchemy.mdc | 261 +++++++ .cursor/rules/sqlite.mdc | 342 +++++++++ .cursor/rules/statsmodels.mdc | 289 ++++++++ .cursor/rules/streamlit.mdc | 74 ++ .cursor/rules/stripe.mdc | 284 ++++++++ .cursor/rules/supabase.mdc | 283 ++++++++ .cursor/rules/svelte.mdc | 157 ++++ .cursor/rules/sveltekit.mdc | 306 ++++++++ .cursor/rules/tailwind.mdc | 101 +++ .cursor/rules/tauri.mdc | 296 ++++++++ .cursor/rules/tensorflow.mdc | 185 +++++ .cursor/rules/terraform.mdc | 166 +++++ .cursor/rules/three-js.mdc | 311 ++++++++ .cursor/rules/tinygrad.mdc | 175 +++++ .cursor/rules/tkinter.mdc | 292 ++++++++ .cursor/rules/tornado.mdc | 308 ++++++++ .cursor/rules/tortoise-orm.mdc | 235 ++++++ .cursor/rules/tqdm.mdc | 182 +++++ .cursor/rules/transformers.mdc | 174 +++++ .cursor/rules/trio.mdc | 224 ++++++ .cursor/rules/trpc.mdc | 353 +++++++++ .cursor/rules/turbopack.mdc | 188 +++++ .cursor/rules/typer.mdc | 276 +++++++ .cursor/rules/typescript.mdc | 262 +++++++ .cursor/rules/unittest.mdc | 266 +++++++ .cursor/rules/unity.mdc | 471 ++++++++++++ .cursor/rules/unreal-engine.mdc | 363 ++++++++++ .cursor/rules/vercel.mdc | 161 +++++ .cursor/rules/vim.md | 248 +++++++ .cursor/rules/vite.mdc | 224 ++++++ .cursor/rules/vitest.mdc | 363 ++++++++++ .cursor/rules/vllm.mdc | 140 ++++ .cursor/rules/vue.mdc | 298 ++++++++ .cursor/rules/vue3.mdc | 114 +++ .cursor/rules/webpack.mdc | 281 ++++++++ .cursor/rules/xgboost.mdc | 287 ++++++++ .cursor/rules/zod.mdc | 145 ++++ .cursor/rules/zsh.mdc | 200 ++++++ .cursor/rules/zustand.mdc | 672 ++++++++++++++++++ .../resources/views/modal.blade.php | 128 ++-- 242 files changed, 59643 insertions(+), 61 deletions(-) create mode 100644 .cursor/rules/actix-web.mdc create mode 100644 .cursor/rules/aiohttp.mdc create mode 100644 .cursor/rules/amazon-ec2.mdc create mode 100644 .cursor/rules/amazon-s3.mdc create mode 100644 .cursor/rules/android-sdk.mdc create mode 100644 .cursor/rules/angular.mdc create mode 100644 .cursor/rules/ansible.mdc create mode 100644 .cursor/rules/ant-design.mdc create mode 100644 .cursor/rules/anyio.mdc create mode 100644 .cursor/rules/apollo-client.mdc create mode 100644 .cursor/rules/apollo-graphql.mdc create mode 100644 .cursor/rules/asp-net.mdc create mode 100644 .cursor/rules/astro.mdc create mode 100644 .cursor/rules/asyncio.mdc create mode 100644 .cursor/rules/auth0.mdc create mode 100644 .cursor/rules/autogen.mdc create mode 100644 .cursor/rules/aws-amplify.mdc create mode 100644 .cursor/rules/aws-cli.mdc create mode 100644 .cursor/rules/aws-dynamodb.mdc create mode 100644 .cursor/rules/aws-ecs.mdc create mode 100644 .cursor/rules/aws-lambda.mdc create mode 100644 .cursor/rules/aws-rds.mdc create mode 100644 .cursor/rules/aws.mdc create mode 100644 .cursor/rules/axios.mdc create mode 100644 .cursor/rules/azure-pipelines.mdc create mode 100644 .cursor/rules/azure.mdc create mode 100644 .cursor/rules/bash.mdc create mode 100644 .cursor/rules/beautifulsoup4.mdc create mode 100644 .cursor/rules/behave.mdc create mode 100644 .cursor/rules/black.mdc create mode 100644 .cursor/rules/boto3.mdc create mode 100644 .cursor/rules/bottle.mdc create mode 100644 .cursor/rules/bun.mdc create mode 100644 .cursor/rules/c-sharp.mdc create mode 100644 .cursor/rules/chakra-ui.mdc create mode 100644 .cursor/rules/cheerio.mdc create mode 100644 .cursor/rules/circleci.mdc create mode 100644 .cursor/rules/clerk.mdc create mode 100644 .cursor/rules/click.mdc create mode 100644 .cursor/rules/cloudflare.mdc create mode 100644 .cursor/rules/codemirror.mdc create mode 100644 .cursor/rules/crewai.mdc create mode 100644 .cursor/rules/css.mdc create mode 100644 .cursor/rules/cuda.mdc create mode 100644 .cursor/rules/customtkinter.mdc create mode 100644 .cursor/rules/cypress.mdc create mode 100644 .cursor/rules/d3.mdc create mode 100644 .cursor/rules/dask.mdc create mode 100644 .cursor/rules/databricks.mdc create mode 100644 .cursor/rules/datadog.mdc create mode 100644 .cursor/rules/deno.mdc create mode 100644 .cursor/rules/detox.mdc create mode 100644 .cursor/rules/digitalocean.mdc create mode 100644 .cursor/rules/discord-api.mdc create mode 100644 .cursor/rules/django-orm.mdc create mode 100644 .cursor/rules/django-rest-framework.mdc create mode 100644 .cursor/rules/django.mdc create mode 100644 .cursor/rules/docker.md create mode 100644 .cursor/rules/drizzle.mdc create mode 100644 .cursor/rules/duckdb.mdc create mode 100644 .cursor/rules/elasticsearch.mdc create mode 100644 .cursor/rules/electron.mdc create mode 100644 .cursor/rules/elk-stack.mdc create mode 100644 .cursor/rules/emacs.mdc create mode 100644 .cursor/rules/esbuild.mdc create mode 100644 .cursor/rules/eslint.mdc create mode 100644 .cursor/rules/expo.mdc create mode 100644 .cursor/rules/express.mdc create mode 100644 .cursor/rules/fabric-js.mdc create mode 100644 .cursor/rules/fastapi.mdc create mode 100644 .cursor/rules/ffmpeg.mdc create mode 100644 .cursor/rules/fiber.mdc create mode 100644 .cursor/rules/firebase.mdc create mode 100644 .cursor/rules/flake8.mdc create mode 100644 .cursor/rules/flask-restful.mdc create mode 100644 .cursor/rules/flask.mdc create mode 100644 .cursor/rules/flutter.mdc create mode 100644 .cursor/rules/fontawesome.mdc create mode 100644 .cursor/rules/gcp-cli.mdc create mode 100644 .cursor/rules/gcp.mdc create mode 100644 .cursor/rules/gensim.mdc create mode 100644 .cursor/rules/git.mdc create mode 100644 .cursor/rules/github-actions.mdc create mode 100644 .cursor/rules/gitlab-ci.mdc create mode 100644 .cursor/rules/go.mdc create mode 100644 .cursor/rules/godot.mdc create mode 100644 .cursor/rules/google-maps-js.mdc create mode 100644 .cursor/rules/gradle.mdc create mode 100644 .cursor/rules/grafana.mdc create mode 100644 .cursor/rules/graphql.mdc create mode 100644 .cursor/rules/guzzle.mdc create mode 100644 .cursor/rules/hardhat.mdc create mode 100644 .cursor/rules/heroku.mdc create mode 100644 .cursor/rules/htmx.mdc create mode 100644 .cursor/rules/httpx.mdc create mode 100644 .cursor/rules/huggingface.mdc create mode 100644 .cursor/rules/hypothesis.mdc create mode 100644 .cursor/rules/insomnia.mdc create mode 100644 .cursor/rules/ionic.mdc create mode 100644 .cursor/rules/isort.mdc create mode 100644 .cursor/rules/java.mdc create mode 100644 .cursor/rules/jax.mdc create mode 100644 .cursor/rules/jenkins.mdc create mode 100644 .cursor/rules/jest.mdc create mode 100644 .cursor/rules/jetpack-compose.mdc create mode 100644 .cursor/rules/jquery.mdc create mode 100644 .cursor/rules/junit.mdc create mode 100644 .cursor/rules/keras.mdc create mode 100644 .cursor/rules/kivy.mdc create mode 100644 .cursor/rules/kubernetes.mdc create mode 100644 .cursor/rules/langchain-js.mdc create mode 100644 .cursor/rules/langchain.mdc create mode 100644 .cursor/rules/langgraph.mdc rename .cursor/rules/{core.mdc => laravel.mdc} (99%) create mode 100644 .cursor/rules/lightgbm.mdc create mode 100644 .cursor/rules/llama-index.mdc create mode 100644 .cursor/rules/llamaindex-js.mdc create mode 100644 .cursor/rules/llvm.mdc create mode 100644 .cursor/rules/material-ui.mdc create mode 100644 .cursor/rules/matplotlib.mdc create mode 100644 .cursor/rules/maven.mdc create mode 100644 .cursor/rules/microsoft-teams.mdc create mode 100644 .cursor/rules/mkdocs.mdc create mode 100644 .cursor/rules/mlx.mdc create mode 100644 .cursor/rules/mobx.mdc create mode 100644 .cursor/rules/mockito.mdc create mode 100644 .cursor/rules/modal.mdc create mode 100644 .cursor/rules/mongodb.mdc create mode 100644 .cursor/rules/mypy.mdc create mode 100644 .cursor/rules/neo4j.mdc create mode 100644 .cursor/rules/nestjs.mdc create mode 100644 .cursor/rules/netlify.mdc create mode 100644 .cursor/rules/next-js.mdc create mode 100644 .cursor/rules/nginx.mdc create mode 100644 .cursor/rules/nltk.mdc create mode 100644 .cursor/rules/nose2.mdc create mode 100644 .cursor/rules/notion-api.mdc create mode 100644 .cursor/rules/numba.mdc create mode 100644 .cursor/rules/numpy.mdc create mode 100644 .cursor/rules/nuxt.mdc create mode 100644 .cursor/rules/openai.mdc create mode 100644 .cursor/rules/opencv-python.mdc create mode 100644 .cursor/rules/pandas.mdc create mode 100644 .cursor/rules/pdoc.mdc create mode 100644 .cursor/rules/peewee.mdc create mode 100644 .cursor/rules/phoenix.mdc create mode 100644 .cursor/rules/php.mdc create mode 100644 .cursor/rules/pillow.mdc create mode 100644 .cursor/rules/playwright.mdc create mode 100644 .cursor/rules/plotly.mdc create mode 100644 .cursor/rules/poetry.mdc create mode 100644 .cursor/rules/pony.mdc create mode 100644 .cursor/rules/postgresql.mdc create mode 100644 .cursor/rules/postman.mdc create mode 100644 .cursor/rules/prisma.mdc create mode 100644 .cursor/rules/puppeteer.mdc create mode 100644 .cursor/rules/pydantic.mdc create mode 100644 .cursor/rules/pygame.mdc create mode 100644 .cursor/rules/pylint.mdc create mode 100644 .cursor/rules/pyqt.mdc create mode 100644 .cursor/rules/pyramid.mdc create mode 100644 .cursor/rules/pyright.mdc create mode 100644 .cursor/rules/pyside.mdc create mode 100644 .cursor/rules/pytest.mdc create mode 100644 .cursor/rules/python.mdc create mode 100644 .cursor/rules/pytorch.mdc create mode 100644 .cursor/rules/qwik.mdc create mode 100644 .cursor/rules/railway.mdc create mode 100644 .cursor/rules/react-mobx.mdc create mode 100644 .cursor/rules/react-native.mdc create mode 100644 .cursor/rules/react-query.mdc create mode 100644 .cursor/rules/react-redux.mdc create mode 100644 .cursor/rules/react.mdc create mode 100644 .cursor/rules/redis.mdc create mode 100644 .cursor/rules/redux.mdc create mode 100644 .cursor/rules/remix.mdc create mode 100644 .cursor/rules/requests.mdc create mode 100644 .cursor/rules/rich.mdc create mode 100644 .cursor/rules/riverpod.mdc create mode 100644 .cursor/rules/rocket.mdc create mode 100644 .cursor/rules/ros.mdc create mode 100644 .cursor/rules/ruby.mdc create mode 100644 .cursor/rules/rust.mdc create mode 100644 .cursor/rules/sanic.mdc create mode 100644 .cursor/rules/scikit-image.mdc create mode 100644 .cursor/rules/scikit-learn.mdc create mode 100644 .cursor/rules/scipy.mdc create mode 100644 .cursor/rules/scrapy.mdc create mode 100644 .cursor/rules/seaborn.mdc create mode 100644 .cursor/rules/selenium.mdc create mode 100644 .cursor/rules/sentry.mdc create mode 100644 .cursor/rules/servemux.mdc create mode 100644 .cursor/rules/setuptools.mdc create mode 100644 .cursor/rules/shadcn.mdc create mode 100644 .cursor/rules/smolagents.mdc create mode 100644 .cursor/rules/socket-io.mdc create mode 100644 .cursor/rules/solidity.mdc create mode 100644 .cursor/rules/solidjs.mdc create mode 100644 .cursor/rules/spacy.mdc create mode 100644 .cursor/rules/sphinx.mdc create mode 100644 .cursor/rules/spring.mdc create mode 100644 .cursor/rules/springboot.mdc create mode 100644 .cursor/rules/sqlalchemy.mdc create mode 100644 .cursor/rules/sqlite.mdc create mode 100644 .cursor/rules/statsmodels.mdc create mode 100644 .cursor/rules/streamlit.mdc create mode 100644 .cursor/rules/stripe.mdc create mode 100644 .cursor/rules/supabase.mdc create mode 100644 .cursor/rules/svelte.mdc create mode 100644 .cursor/rules/sveltekit.mdc create mode 100644 .cursor/rules/tailwind.mdc create mode 100644 .cursor/rules/tauri.mdc create mode 100644 .cursor/rules/tensorflow.mdc create mode 100644 .cursor/rules/terraform.mdc create mode 100644 .cursor/rules/three-js.mdc create mode 100644 .cursor/rules/tinygrad.mdc create mode 100644 .cursor/rules/tkinter.mdc create mode 100644 .cursor/rules/tornado.mdc create mode 100644 .cursor/rules/tortoise-orm.mdc create mode 100644 .cursor/rules/tqdm.mdc create mode 100644 .cursor/rules/transformers.mdc create mode 100644 .cursor/rules/trio.mdc create mode 100644 .cursor/rules/trpc.mdc create mode 100644 .cursor/rules/turbopack.mdc create mode 100644 .cursor/rules/typer.mdc create mode 100644 .cursor/rules/typescript.mdc create mode 100644 .cursor/rules/unittest.mdc create mode 100644 .cursor/rules/unity.mdc create mode 100644 .cursor/rules/unreal-engine.mdc create mode 100644 .cursor/rules/vercel.mdc create mode 100644 .cursor/rules/vim.md create mode 100644 .cursor/rules/vite.mdc create mode 100644 .cursor/rules/vitest.mdc create mode 100644 .cursor/rules/vllm.mdc create mode 100644 .cursor/rules/vue.mdc create mode 100644 .cursor/rules/vue3.mdc create mode 100644 .cursor/rules/webpack.mdc create mode 100644 .cursor/rules/xgboost.mdc create mode 100644 .cursor/rules/zod.mdc create mode 100644 .cursor/rules/zsh.mdc create mode 100644 .cursor/rules/zustand.mdc diff --git a/.cursor/rules/actix-web.mdc b/.cursor/rules/actix-web.mdc new file mode 100644 index 000000000..1d54a47f4 --- /dev/null +++ b/.cursor/rules/actix-web.mdc @@ -0,0 +1,318 @@ +--- +description: Comprehensive best practices for developing robust, efficient, and maintainable applications using the actix-web framework in Rust. This rule covers coding standards, project structure, performance, security, testing, and common pitfalls. +globs: *.rs +--- +# Actix-web Best Practices: A Comprehensive Guide + +This guide provides a comprehensive overview of best practices for developing applications using the actix-web framework in Rust. It covers various aspects of development, including code organization, performance optimization, security, testing, and common pitfalls. + +## 1. Code Organization and Structure + +A well-organized codebase is crucial for maintainability and scalability. Here's how to structure your actix-web project effectively: + +### 1.1. Directory Structure Best Practices + +Adopt a modular and layered architecture. A common and recommended directory structure is as follows: + + +project_root/ +├── src/ +│ ├── main.rs # Entry point of the application +│ ├── lib.rs # Library file if extracting common functionality +│ ├── modules/ +│ │ ├── mod.rs # Module declaration +│ │ ├── auth/ # Authentication module +│ │ │ ├── mod.rs # Auth module declaration +│ │ │ ├── models.rs # Auth models +│ │ │ ├── routes.rs # Auth routes +│ │ │ ├── handlers.rs # Auth handlers +│ │ ├── users/ # User management module +│ │ │ ├── mod.rs # User module declaration +│ │ │ ├── models.rs # User models +│ │ │ ├── routes.rs # User routes +│ │ │ ├── handlers.rs # User handlers +│ ├── models/ # Data models +│ │ ├── mod.rs +│ │ ├── user.rs # User model +│ │ ├── post.rs # Post model +│ ├── routes/ # Route configurations +│ │ ├── mod.rs +│ │ ├── auth_routes.rs # Authentication routes +│ │ ├── user_routes.rs # User routes +│ ├── handlers/ # Request handlers (controllers) +│ │ ├── mod.rs +│ │ ├── auth_handlers.rs # Authentication handlers +│ │ ├── user_handlers.rs # User handlers +│ ├── middleware/ # Custom middleware components +│ │ ├── mod.rs +│ │ ├── logger.rs # Logging middleware +│ │ ├── auth.rs # Authentication middleware +│ ├── utils/ # Utility functions and modules +│ │ ├── mod.rs +│ │ ├── db.rs # Database connection utility +│ ├── errors/ # Custom error definitions +│ │ ├── mod.rs +│ │ ├── app_error.rs # Application-specific error types +├── tests/ # Integration and unit tests +│ ├── mod.rs +│ ├── api_tests.rs # Integration tests for API endpoints +├── .env # Environment variables +├── Cargo.toml # Project dependencies and metadata +├── Cargo.lock # Dependency lockfile + + +### 1.2. File Naming Conventions + +* **Modules:** Use lowercase, descriptive names (e.g., `auth`, `users`, `posts`). +* **Files:** Use lowercase with underscores (e.g., `user_routes.rs`, `auth_handlers.rs`). +* **Models:** Use singular nouns (e.g., `user.rs`, `post.rs`). +* **Handlers:** Use descriptive names indicating the action performed (e.g., `create_user`, `get_user`). +* **Routes:** Use names indicating the resource they handle (e.g., `user_routes`, `auth_routes`). + +### 1.3. Module Organization + +* **Explicit Module Declarations:** Always declare submodules in `mod.rs` files. This ensures proper module resolution and prevents naming conflicts. +* **Clear Boundaries:** Each module should have a well-defined responsibility. Avoid mixing unrelated functionalities within the same module. +* **Public vs. Private:** Use `pub` keyword judiciously to control visibility. Keep implementation details private to modules to prevent accidental external dependencies. + +### 1.4. Component Architecture + +* **Layered Architecture:** Separate concerns into distinct layers (e.g., data access, business logic, presentation). This improves testability and maintainability. +* **Dependency Injection:** Use dependency injection to provide dependencies to handlers. This makes it easier to test and configure your application. +* **Services:** Encapsulate business logic into services. Handlers should primarily focus on request/response handling and delegate business logic to services. + +### 1.5. Code Splitting Strategies + +* **Feature-Based Splitting:** Group code based on features (e.g., authentication, user management). This makes it easier to understand and maintain related code. +* **Module-Based Splitting:** Split code into modules based on functionality. This improves code organization and reusability. +* **Lazy Loading (Future Enhancement):** For very large applications, consider lazy loading modules or features to reduce initial startup time. This can be accomplished by dynamically enabling parts of your application based on configuration or runtime conditions. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns Specific to Actix-web + +* **Extractor Pattern:** Use extractors to handle different types of incoming data (e.g., `Path`, `Query`, `Json`, `Form`). Extractors simplify handler logic and provide type safety. +* **Middleware Pattern:** Implement custom middleware for tasks like logging, authentication, and request modification. Middleware allows you to intercept and process requests before they reach the handlers. +* **State Management Pattern:** Use `web::Data` to share application state across handlers. This provides a thread-safe way to access shared resources like database connections and configuration settings. +* **Error Handling Pattern:** Define custom error types and implement the `ResponseError` trait for centralized error handling and consistent error responses. + +### 2.2. Recommended Approaches for Common Tasks + +* **Database Integration:** Use an asynchronous database driver like `tokio-postgres` or `sqlx` for efficient database interactions. +* **Authentication:** Implement authentication using JWT (JSON Web Tokens) or other secure authentication mechanisms. +* **Authorization:** Implement role-based access control (RBAC) or attribute-based access control (ABAC) to restrict access to resources based on user roles or attributes. +* **Logging:** Use a logging framework like `tracing` or `log` for structured logging and monitoring. +* **Configuration Management:** Use a configuration library like `config` or `dotenv` to manage application settings from environment variables and configuration files. + +### 2.3. Anti-patterns and Code Smells to Avoid + +* **Long Handler Functions:** Keep handler functions short and focused. Delegate complex logic to services or helper functions. +* **Tight Coupling:** Avoid tight coupling between modules. Use interfaces and dependency injection to decouple components. +* **Ignoring Errors:** Always handle errors gracefully and provide informative error messages to the client. +* **Blocking Operations in Handlers:** Avoid performing blocking operations (e.g., synchronous I/O) in handler functions. Use asynchronous operations to prevent blocking the event loop. +* **Overusing Global State:** Minimize the use of global state. Prefer passing state as dependencies to handler functions. + +### 2.4. State Management Best Practices + +* **Immutable State:** Prefer immutable state whenever possible. This reduces the risk of race conditions and makes it easier to reason about the application. +* **Thread-Safe Data Structures:** Use thread-safe data structures like `Arc>` or `RwLock` to share mutable state across threads. +* **Avoid Direct Mutability:** Avoid directly mutating shared state. Instead, use atomic operations or message passing to coordinate state updates. +* **Dependency Injection:** Use dependency injection to provide state to handler functions. This makes it easier to test and configure the application. + +### 2.5. Error Handling Patterns + +* **Custom Error Types:** Define custom error types to represent different error scenarios in your application. +* **`ResponseError` Trait:** Implement the `ResponseError` trait for custom error types to generate appropriate HTTP responses. +* **Centralized Error Handling:** Use a centralized error handling mechanism (e.g., middleware) to catch and process errors consistently. +* **Informative Error Messages:** Provide informative error messages to the client to help them understand and resolve the issue. +* **Logging Errors:** Log errors with sufficient detail to help diagnose and debug issues. +* **`Result` Type:** Leverage the `Result` type effectively, propagating errors up the call stack using the `?` operator, and handle them at the appropriate level. + +## 3. Performance Considerations + +Optimizing performance is crucial for building scalable and responsive actix-web applications. + +### 3.1. Optimization Techniques + +* **Asynchronous Operations:** Use asynchronous operations for I/O-bound tasks (e.g., database access, network requests) to prevent blocking the event loop. +* **Connection Pooling:** Use connection pooling for database connections to reduce the overhead of establishing new connections. +* **Caching:** Implement caching for frequently accessed data to reduce database load and improve response times. +* **Compression:** Enable compression (e.g., gzip) for responses to reduce the amount of data transmitted over the network. +* **Keep-Alive Connections:** Use keep-alive connections to reuse existing TCP connections and reduce connection establishment overhead. + +### 3.2. Memory Management + +* **Avoid Unnecessary Cloning:** Minimize cloning of data to reduce memory allocations and copying. +* **Use References:** Use references instead of copying data whenever possible. +* **Smart Pointers:** Use smart pointers (e.g., `Box`, `Arc`, `Rc`) to manage memory efficiently. +* **String Handling:** Be mindful of string handling. Use `String` when ownership is needed, and `&str` when a read-only view is sufficient. + +### 3.3. Rendering Optimization + +* **Template Caching:** Cache templates to reduce the overhead of parsing and compiling templates on each request. +* **Minimize DOM Updates:** Minimize DOM updates in the client-side JavaScript code to improve rendering performance. +* **Efficient Serialization:** Ensure your data serialization is efficient, using appropriate data structures and serialization libraries (e.g., `serde_json`). + +### 3.4. Bundle Size Optimization + +* **Dependency Pruning:** Remove unused dependencies from your `Cargo.toml` file to reduce the bundle size. +* **Feature Flags:** Use feature flags to enable or disable optional features at compile time. +* **Code Minification:** Use code minification to reduce the size of your JavaScript and CSS files. + +### 3.5. Lazy Loading Strategies + +* **Lazy Initialization:** Use lazy initialization for expensive resources to defer their creation until they are actually needed. +* **On-Demand Loading:** Load resources on demand (e.g., images, data) to reduce the initial load time. + +## 4. Security Best Practices + +Security is paramount for building robust and reliable actix-web applications. + +### 4.1. Common Vulnerabilities and How to Prevent Them + +* **SQL Injection:** Use parameterized queries or ORMs to prevent SQL injection attacks. +* **Cross-Site Scripting (XSS):** Sanitize user input and escape output to prevent XSS attacks. +* **Cross-Site Request Forgery (CSRF):** Implement CSRF protection to prevent unauthorized requests from other websites. +* **Authentication and Authorization:** Use strong authentication and authorization mechanisms to protect sensitive resources. +* **Denial-of-Service (DoS):** Implement rate limiting and other defense mechanisms to prevent DoS attacks. + +### 4.2. Input Validation + +* **Validate All Input:** Validate all user input to ensure that it conforms to the expected format and range. +* **Use Type Safety:** Use type safety to prevent invalid data from being processed. +* **Regular Expressions:** Use regular expressions to validate complex input patterns. +* **Whitelist vs. Blacklist:** Prefer whitelisting valid input over blacklisting invalid input. + +### 4.3. Authentication and Authorization Patterns + +* **JWT (JSON Web Tokens):** Use JWT for stateless authentication and authorization. +* **OAuth 2.0:** Use OAuth 2.0 for delegated authorization. +* **RBAC (Role-Based Access Control):** Use RBAC to restrict access to resources based on user roles. +* **ABAC (Attribute-Based Access Control):** Use ABAC to restrict access to resources based on user attributes. +* **Password Hashing:** Always hash passwords using a strong hashing algorithm (e.g., bcrypt, Argon2) and store them securely. + +### 4.4. Data Protection Strategies + +* **Encryption:** Encrypt sensitive data at rest and in transit. +* **Data Masking:** Mask sensitive data to prevent unauthorized access. +* **Data Anonymization:** Anonymize data to protect user privacy. +* **Access Control:** Implement strict access control policies to restrict access to sensitive data. + +### 4.5. Secure API Communication + +* **HTTPS:** Use HTTPS for all API communication to encrypt data in transit. +* **TLS Certificates:** Use valid TLS certificates from a trusted certificate authority. +* **API Keys:** Use API keys to authenticate API clients. +* **Rate Limiting:** Implement rate limiting to prevent abuse and DoS attacks. + +## 5. Testing Approaches + +Thorough testing is essential for ensuring the quality and reliability of actix-web applications. + +### 5.1. Unit Testing Strategies + +* **Test Individual Modules:** Unit test individual modules and functions in isolation. +* **Mock Dependencies:** Use mocking to isolate units from external dependencies (e.g., database, API). +* **Test Edge Cases:** Test edge cases and boundary conditions to ensure that the code handles them correctly. +* **Table-Driven Tests:** Use table-driven tests to test multiple scenarios with different inputs and expected outputs. + +### 5.2. Integration Testing + +* **Test API Endpoints:** Integration test API endpoints to ensure that they function correctly together. +* **Test Database Interactions:** Test database interactions to ensure that data is read and written correctly. +* **Test Middleware:** Test middleware to ensure that they correctly process requests and responses. + +### 5.3. End-to-End Testing + +* **Simulate User Interactions:** End-to-end tests simulate user interactions to test the entire application flow. +* **Use a Testing Framework:** Use a testing framework (e.g., Selenium, Cypress) to automate end-to-end tests. + +### 5.4. Test Organization + +* **Test Directory:** Keep tests in a separate `tests` directory. +* **Test Modules:** Organize tests into modules that mirror the application structure. +* **Test Naming:** Use descriptive names for test functions to indicate what they are testing. + +### 5.5. Mocking and Stubbing + +* **Mock External Dependencies:** Mock external dependencies (e.g., database, API) to isolate units from external factors. +* **Use Mocking Libraries:** Use mocking libraries (e.g., `mockall`) to create mock objects and define their behavior. +* **Stub Data:** Use stub data to simulate different scenarios and test edge cases. + +## 6. Common Pitfalls and Gotchas + +Be aware of common pitfalls and gotchas when developing actix-web applications. + +### 6.1. Frequent Mistakes Developers Make + +* **Blocking Operations:** Performing blocking operations in handler functions can block the event loop and degrade performance. +* **Incorrect Error Handling:** Ignoring errors or not handling them correctly can lead to unexpected behavior and security vulnerabilities. +* **Not Validating Input:** Not validating user input can lead to security vulnerabilities and data corruption. +* **Overusing Global State:** Overusing global state can make the application difficult to reason about and test. +* **Not Using Asynchronous Operations:** Not using asynchronous operations for I/O-bound tasks can degrade performance. + +### 6.2. Edge Cases to be Aware Of + +* **Handling Large Requests:** Be mindful of handling large requests and implement appropriate size limits to prevent DoS attacks. +* **Handling Concurrent Requests:** Ensure that the application can handle concurrent requests efficiently and without race conditions. +* **Handling Network Errors:** Handle network errors gracefully and provide informative error messages to the client. +* **Handling Database Connection Errors:** Handle database connection errors gracefully and implement retry mechanisms. + +### 6.3. Version-Specific Issues + +* **Breaking Changes:** Be aware of breaking changes in actix-web and its dependencies. +* **Deprecated Features:** Avoid using deprecated features and migrate to the recommended alternatives. +* **Compatibility:** Ensure that the application is compatible with the target Rust version and operating system. + +### 6.4. Compatibility Concerns + +* **Rust Version:** Ensure compatibility with the supported Rust versions. +* **Operating System:** Test on different operating systems (Linux, macOS, Windows). +* **Browser Compatibility (if applicable):** If the application includes a front-end, test with various browsers. + +### 6.5. Debugging Strategies + +* **Logging:** Use logging to track the application's execution flow and identify potential issues. +* **Debugging Tools:** Use debugging tools (e.g., `gdb`, `lldb`) to inspect the application's state and step through the code. +* **Unit Tests:** Write unit tests to isolate and debug individual modules and functions. +* **Profiling:** Use profiling tools to identify performance bottlenecks. + +## 7. Tooling and Environment + +Using the right tools and environment can significantly improve the development experience and productivity. + +### 7.1. Recommended Development Tools + +* **Rust IDE:** Use a Rust IDE (e.g., Visual Studio Code with the Rust extension, IntelliJ Rust) for code completion, syntax highlighting, and debugging. +* **Cargo:** Use Cargo, the Rust package manager, for managing dependencies and building the application. +* **Rustup:** Use Rustup for managing Rust toolchains and versions. +* **Clippy:** Use Clippy, a Rust linter, for identifying potential code quality issues. +* **Formatter:** Use rustfmt to automatically format the code according to the Rust style guide. + +### 7.2. Build Configuration + +* **Release Mode:** Build the application in release mode for optimized performance. +* **Link-Time Optimization (LTO):** Enable link-time optimization to improve performance. +* **Codegen Units:** Experiment with different codegen unit settings to optimize compilation time and code size. + +### 7.3. Linting and Formatting + +* **Clippy:** Use Clippy to identify potential code quality issues and enforce coding standards. +* **Rustfmt:** Use rustfmt to automatically format the code according to the Rust style guide. +* **Pre-commit Hooks:** Use pre-commit hooks to automatically run Clippy and rustfmt before committing changes. + +### 7.4. Deployment Best Practices + +* **Containerization:** Use containerization (e.g., Docker) to package the application and its dependencies into a portable container. +* **Orchestration:** Use container orchestration (e.g., Kubernetes) to manage and scale the application. +* **Reverse Proxy:** Use a reverse proxy (e.g., Nginx, Apache) to handle incoming requests and route them to the application. +* **Load Balancing:** Use load balancing to distribute traffic across multiple instances of the application. +* **Monitoring:** Implement monitoring to track the application's health and performance. + +### 7.5. CI/CD Integration + +* **Continuous Integration (CI):** Use a CI system (e.g., GitHub Actions, GitLab CI, Jenkins) to automatically build, test, and lint the code on every commit. +* **Continuous Delivery (CD):** Use a CD system to automatically deploy the application to production after it passes all tests. +* **Automated Testing:** Automate unit, integration, and end-to-end tests in the CI/CD pipeline. + +By following these best practices, you can build robust, efficient, and maintainable actix-web applications that meet the highest standards of quality and security. Remember to stay up-to-date with the latest recommendations and adapt them to your specific project needs. \ No newline at end of file diff --git a/.cursor/rules/aiohttp.mdc b/.cursor/rules/aiohttp.mdc new file mode 100644 index 000000000..e5302f5d3 --- /dev/null +++ b/.cursor/rules/aiohttp.mdc @@ -0,0 +1,338 @@ +--- +description: Comprehensive guide for aiohttp development covering code organization, performance, security, testing, and deployment best practices. Provides actionable guidance for developers to build robust and maintainable aiohttp applications. +globs: *.py +--- +# Aiohttp Best Practices + +This document provides a comprehensive guide to aiohttp development, covering code organization, performance, security, testing, and deployment. + +Library Information: +- Name: aiohttp +- Tags: web, python, http-client, async + +## 1. Code Organization and Structure + +### 1.1. Directory Structure Best Practices: + +* **Project Root:** + * `src/`: Contains the main application code. + * `main.py`: Entry point of the application. + * `app.py`: Application factory and setup. + * `routes.py`: Defines application routes. + * `handlers/`: Contains request handlers. + * `user_handlers.py`: User-related handlers. + * `product_handlers.py`: Product-related handlers. + * `middlewares/`: Custom middleware components. + * `logging_middleware.py`: Logging middleware. + * `auth_middleware.py`: Authentication middleware. + * `utils/`: Utility modules. + * `db.py`: Database connection and utilities. + * `config.py`: Configuration management. + * `tests/`: Contains unit and integration tests. + * `conftest.py`: Pytest configuration file. + * `unit/`: Unit tests. + * `integration/`: Integration tests. + * `static/`: Static files (CSS, JavaScript, images). + * `templates/`: Jinja2 or other template files. + * `docs/`: Project documentation. + * `requirements.txt`: Python dependencies. + * `Dockerfile`: Docker configuration file. + * `docker-compose.yml`: Docker Compose configuration. + * `.env`: Environment variables. + * `README.md`: Project description and instructions. + * `.gitignore`: Specifies intentionally untracked files that Git should ignore. + * `.cursor/rules/`: Project specific Cursor AI rules. + +### 1.2. File Naming Conventions: + +* Python files: `snake_case.py` (e.g., `user_handlers.py`, `database_utils.py`). +* Class names: `CamelCase` (e.g., `UserHandler`, `DatabaseConnection`). +* Function names: `snake_case` (e.g., `get_user`, `create_product`). +* Variables: `snake_case` (e.g., `user_id`, `product_name`). +* Constants: `UPPER_SNAKE_CASE` (e.g., `DEFAULT_PORT`, `MAX_CONNECTIONS`). + +### 1.3. Module Organization: + +* Group related functionality into modules. +* Use clear and descriptive module names. +* Avoid circular dependencies. +* Keep modules focused and concise. +* Use packages to organize modules into a hierarchical structure. + +### 1.4. Component Architecture: + +* **Layered Architecture:** Separate the application into distinct layers (e.g., presentation, business logic, data access). +* **Microservices Architecture:** Decompose the application into small, independent services. +* **Hexagonal Architecture (Ports and Adapters):** Decouple the application core from external dependencies. +* **MVC (Model-View-Controller):** Organize the application into models (data), views (presentation), and controllers (logic). + +### 1.5. Code Splitting Strategies: + +* **Route-based splitting:** Load modules based on the requested route. +* **Feature-based splitting:** Divide the application into feature modules. +* **Component-based splitting:** Split the application into reusable components. +* **On-demand loading:** Load modules only when they are needed. +* **Asynchronous loading:** Use `asyncio.gather` or similar techniques to load modules concurrently. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns: + +* **Singleton:** For managing shared resources like database connections or configuration objects. +* **Factory:** For creating instances of classes with complex initialization logic. +* **Strategy:** For implementing different algorithms or behaviors. +* **Observer:** For implementing event-driven systems. +* **Middleware:** For handling cross-cutting concerns like logging, authentication, and error handling. + +### 2.2. Recommended Approaches for Common Tasks: + +* **Request Handling:** Use request handlers to process incoming requests. +* **Routing:** Use `aiohttp.web.RouteTableDef` for defining routes. +* **Middleware:** Implement middleware for request pre-processing and response post-processing. +* **Data Serialization:** Use `aiohttp.web.json_response` for serializing data to JSON. +* **Error Handling:** Implement custom error handlers to handle exceptions gracefully. +* **Session Management:** Use `aiohttp-session` for managing user sessions. +* **WebSockets:** Utilize `aiohttp.web.WebSocketResponse` for handling WebSocket connections. + +### 2.3. Anti-patterns and Code Smells: + +* **Creating a new `ClientSession` for each request:** This is a performance bottleneck. Reuse a single `ClientSession`. +* **Blocking operations in asynchronous code:** Avoid using blocking operations (e.g., `time.sleep`) in asynchronous code. +* **Ignoring exceptions:** Always handle exceptions properly to prevent unexpected behavior. +* **Overusing global variables:** Avoid using global variables as much as possible to maintain code clarity and testability. +* **Tight coupling:** Decouple components to improve maintainability and reusability. +* **Hardcoding configuration:** Use environment variables or configuration files to manage configuration settings. + +### 2.4. State Management: + +* **Application State:** Store application-level state in the `aiohttp.web.Application` instance. +* **Request State:** Store request-specific state in the `aiohttp.web.Request` instance. +* **Session State:** Use `aiohttp-session` to manage user session data. +* **Database:** Use a database like PostgreSQL, MySQL, or MongoDB to store persistent state. +* **Redis/Memcached:** Use in-memory data stores for caching frequently accessed data. + +### 2.5. Error Handling: + +* **Use `try-except` blocks:** Wrap code that may raise exceptions in `try-except` blocks. +* **Handle specific exceptions:** Catch specific exception types instead of using a generic `except Exception`. +* **Log exceptions:** Log exceptions with detailed information for debugging. +* **Return informative error responses:** Return appropriate HTTP status codes and error messages to the client. +* **Implement custom error handlers:** Create custom error handlers to handle specific exception types. +* **Use `aiohttp.web.HTTPException`:** Raise `aiohttp.web.HTTPException` to return HTTP error responses. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques: + +* **Reuse `ClientSession`:** Always reuse a single `ClientSession` instance for making multiple requests. +* **Connection Pooling:** aiohttp automatically uses connection pooling, so reuse your session. +* **Keep-Alive Connections:** Keep-alive connections are enabled by default, reducing connection overhead. +* **Gzip Compression:** Enable Gzip compression for responses to reduce bandwidth usage. +* **Caching:** Implement caching for frequently accessed data to reduce database load. +* **Optimize Database Queries:** Optimize database queries to improve response times. +* **Use Indexes:** Use indexes in your database tables to speed up queries. +* **Limit Payload Size:** Keep request and response payloads as small as possible. +* **Background Tasks:** Use `asyncio.create_task` to offload long-running tasks to the background. +* **Profiling:** Use profiling tools to identify performance bottlenecks. + +### 3.2. Memory Management: + +* **Avoid Memory Leaks:** Ensure that all resources are properly released to prevent memory leaks. +* **Use Generators:** Use generators to process large datasets in chunks. +* **Limit Object Creation:** Minimize the creation of objects to reduce memory overhead. +* **Use Data Structures Efficiently:** Choose appropriate data structures to optimize memory usage. +* **Garbage Collection:** Understand how Python's garbage collection works and optimize your code accordingly. + +### 3.3. Rendering Optimization: + +* **Template Caching:** Cache templates to reduce rendering time. +* **Minimize Template Logic:** Keep template logic simple and move complex logic to request handlers. +* **Use Asynchronous Templates:** Use asynchronous template engines like `aiohttp-jinja2`. +* **Optimize Static Files:** Optimize static files (CSS, JavaScript, images) to reduce page load times. + +### 3.4. Bundle Size Optimization: + +* **Minimize Dependencies:** Reduce the number of dependencies in your project. +* **Tree Shaking:** Use tree shaking to remove unused code from your bundles. +* **Code Minification:** Minify your code to reduce bundle sizes. +* **Code Compression:** Compress your code to further reduce bundle sizes. + +### 3.5. Lazy Loading: + +* **Lazy Load Modules:** Load modules only when they are needed. +* **Lazy Load Images:** Load images only when they are visible in the viewport. +* **Use Asynchronous Loading:** Use `asyncio.gather` or similar techniques to load resources concurrently. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities: + +* **SQL Injection:** Prevent SQL injection by using parameterized queries or an ORM. +* **Cross-Site Scripting (XSS):** Prevent XSS by escaping user input in templates. +* **Cross-Site Request Forgery (CSRF):** Prevent CSRF by using CSRF tokens. +* **Authentication and Authorization Issues:** Implement secure authentication and authorization mechanisms. +* **Denial-of-Service (DoS) Attacks:** Implement rate limiting and other measures to prevent DoS attacks. +* **Insecure Dependencies:** Keep your dependencies up to date to prevent vulnerabilities. + +### 4.2. Input Validation: + +* **Validate all user input:** Validate all user input to prevent malicious data from entering your application. +* **Use a validation library:** Use a validation library like `marshmallow` or `voluptuous` to simplify input validation. +* **Sanitize user input:** Sanitize user input to remove potentially harmful characters. +* **Limit input length:** Limit the length of input fields to prevent buffer overflows. +* **Use regular expressions:** Use regular expressions to validate input patterns. + +### 4.3. Authentication and Authorization: + +* **Use a strong authentication scheme:** Use a strong authentication scheme like OAuth 2.0 or JWT. +* **Store passwords securely:** Store passwords securely using a hashing algorithm like bcrypt. +* **Implement role-based access control (RBAC):** Use RBAC to control access to resources based on user roles. +* **Use secure cookies:** Use secure cookies to protect session data. +* **Implement multi-factor authentication (MFA):** Use MFA to add an extra layer of security. + +### 4.4. Data Protection: + +* **Encrypt sensitive data:** Encrypt sensitive data at rest and in transit. +* **Use HTTPS:** Use HTTPS to encrypt communication between the client and the server. +* **Store data securely:** Store data in a secure location with appropriate access controls. +* **Regularly back up data:** Regularly back up data to prevent data loss. +* **Comply with data privacy regulations:** Comply with data privacy regulations like GDPR and CCPA. + +### 4.5. Secure API Communication: + +* **Use HTTPS:** Always use HTTPS for API communication. +* **Implement API authentication:** Use API keys or tokens to authenticate API requests. +* **Rate limit API requests:** Implement rate limiting to prevent abuse. +* **Validate API requests:** Validate API requests to prevent malicious data from entering your application. +* **Log API requests:** Log API requests for auditing and debugging. + +## 5. Testing Approaches + +### 5.1. Unit Testing: + +* **Test individual components:** Unit tests should test individual components in isolation. +* **Use a testing framework:** Use a testing framework like `pytest` or `unittest`. +* **Write clear and concise tests:** Write clear and concise tests that are easy to understand. +* **Test edge cases:** Test edge cases and boundary conditions. +* **Use mocks and stubs:** Use mocks and stubs to isolate components under test. + +### 5.2. Integration Testing: + +* **Test interactions between components:** Integration tests should test interactions between different components. +* **Test with real dependencies:** Integration tests should use real dependencies whenever possible. +* **Test the entire application flow:** Integration tests should test the entire application flow. +* **Use a testing database:** Use a testing database to isolate integration tests from the production database. + +### 5.3. End-to-End Testing: + +* **Test the entire system:** End-to-end tests should test the entire system from end to end. +* **Use a testing environment:** Use a testing environment that mimics the production environment. +* **Automate end-to-end tests:** Automate end-to-end tests to ensure that the system is working correctly. +* **Use a browser automation tool:** Use a browser automation tool like Selenium or Puppeteer. + +### 5.4. Test Organization: + +* **Organize tests by module:** Organize tests by module to improve test discovery and maintainability. +* **Use descriptive test names:** Use descriptive test names that clearly indicate what the test is verifying. +* **Use test fixtures:** Use test fixtures to set up and tear down test environments. +* **Use test markers:** Use test markers to categorize tests and run specific test suites. + +### 5.5. Mocking and Stubbing: + +* **Use mocks to simulate dependencies:** Use mocks to simulate the behavior of dependencies. +* **Use stubs to provide predefined responses:** Use stubs to provide predefined responses to API calls. +* **Use mocking libraries:** Use mocking libraries like `unittest.mock` or `pytest-mock`. +* **Avoid over-mocking:** Avoid over-mocking, as it can make tests less reliable. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes: + +* **Not handling exceptions properly:** Always handle exceptions to prevent unexpected behavior. +* **Using blocking operations in asynchronous code:** Avoid using blocking operations in asynchronous code. +* **Not closing `ClientSession`:** Always close the `ClientSession` to release resources. +* **Not validating user input:** Always validate user input to prevent security vulnerabilities. +* **Not using HTTPS:** Always use HTTPS for secure communication. + +### 6.2. Edge Cases: + +* **Handling timeouts:** Implement proper timeout handling to prevent requests from hanging indefinitely. +* **Handling connection errors:** Handle connection errors gracefully to prevent application crashes. +* **Handling large payloads:** Handle large payloads efficiently to prevent memory issues. +* **Handling concurrent requests:** Handle concurrent requests properly to prevent race conditions. +* **Handling Unicode encoding:** Be aware of Unicode encoding issues when processing text data. + +### 6.3. Version-Specific Issues: + +* **aiohttp version compatibility:** Be aware of compatibility issues between different aiohttp versions. +* **asyncio version compatibility:** Be aware of compatibility issues between aiohttp and different asyncio versions. +* **Python version compatibility:** Be aware of compatibility issues between aiohttp and different Python versions. + +### 6.4. Compatibility Concerns: + +* **Compatibility with other libraries:** Be aware of compatibility issues between aiohttp and other libraries. +* **Compatibility with different operating systems:** Be aware of compatibility issues between aiohttp and different operating systems. +* **Compatibility with different web servers:** Be aware of compatibility issues between aiohttp and different web servers. + +### 6.5. Debugging Strategies: + +* **Use logging:** Use logging to track application behavior and identify issues. +* **Use a debugger:** Use a debugger to step through code and examine variables. +* **Use a profiler:** Use a profiler to identify performance bottlenecks. +* **Use error reporting tools:** Use error reporting tools to track and fix errors in production. +* **Use a network analyzer:** Use a network analyzer like Wireshark to capture and analyze network traffic. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools: + +* **IDE:** Use an IDE like VS Code, PyCharm, or Sublime Text. +* **Virtual Environment:** Use a virtual environment to isolate project dependencies. +* **Package Manager:** Use a package manager like pip or poetry to manage dependencies. +* **Testing Framework:** Use a testing framework like pytest or unittest. +* **Linting Tool:** Use a linting tool like pylint or flake8 to enforce code style. +* **Formatting Tool:** Use a formatting tool like black or autopep8 to format code automatically. + +### 7.2. Build Configuration: + +* **Use a build system:** Use a build system like Make or tox to automate build tasks. +* **Define dependencies in `requirements.txt` or `pyproject.toml`:** Specify all project dependencies in a `requirements.txt` or `pyproject.toml` file. +* **Use a Dockerfile:** Use a Dockerfile to create a containerized build environment. +* **Use Docker Compose:** Use Docker Compose to manage multi-container applications. + +### 7.3. Linting and Formatting: + +* **Use a consistent code style:** Use a consistent code style throughout the project. +* **Configure linting tools:** Configure linting tools to enforce code style rules. +* **Configure formatting tools:** Configure formatting tools to format code automatically. +* **Use pre-commit hooks:** Use pre-commit hooks to run linters and formatters before committing code. + +### 7.4. Deployment: + +* **Use a web server:** Use a web server like Nginx or Apache to serve the application. +* **Use a process manager:** Use a process manager like Supervisor or systemd to manage the application process. +* **Use a reverse proxy:** Use a reverse proxy to improve security and performance. +* **Use a load balancer:** Use a load balancer to distribute traffic across multiple servers. +* **Use a monitoring system:** Use a monitoring system to track application health and performance. +* **Standalone Server:** aiohttp.web.run_app(), simple but doesn't utilize all CPU cores. +* **Nginx + Supervisord:** Nginx prevents attacks, allows utilizing all CPU cores, and serves static files faster. +* **Nginx + Gunicorn:** Gunicorn launches the app as worker processes, simplifying deployment compared to bare Nginx. + +### 7.5. CI/CD Integration: + +* **Use a CI/CD pipeline:** Use a CI/CD pipeline to automate the build, test, and deployment process. +* **Use a CI/CD tool:** Use a CI/CD tool like Jenkins, GitLab CI, or GitHub Actions. +* **Run tests in the CI/CD pipeline:** Run tests in the CI/CD pipeline to ensure that code changes don't break the application. +* **Automate deployment:** Automate deployment to reduce manual effort and improve consistency. + +## Additional Best Practices: + +* **Session Management:** Always create a `ClientSession` for making requests and reuse it across multiple requests to benefit from connection pooling. Avoid creating a new session for each request, as this can lead to performance issues. +* **Error Handling:** Implement robust error handling in your request handlers. Use try-except blocks to manage exceptions, particularly for network-related errors. For example, handle `ConnectionResetError` to manage client disconnections gracefully. +* **Middleware Usage:** Utilize middleware for cross-cutting concerns such as logging, error handling, and modifying requests/responses. Define middleware functions that accept a request and a handler, allowing you to process requests before they reach your main handler. +* **Graceful Shutdown:** Implement graceful shutdown procedures for your server to ensure that ongoing requests are completed before the application exits. This can be achieved by registering shutdown signals and cleaning up resources. +* **Security Practices:** When deploying, consider using a reverse proxy like Nginx for added security and performance. Configure SSL/TLS correctly to secure your application. +* **Character Set Detection:** If a response does not include the charset needed to decode the body, use `ClientSession` accepts a `fallback_charset_resolver` parameter which can be used to introduce charset guessing functionality. +* **Persistent Session:** Use `Cleanup Context` when creating a persistent session. + +By adhering to these practices, developers can enhance the reliability, performance, and security of their `aiohttp` applications. \ No newline at end of file diff --git a/.cursor/rules/amazon-ec2.mdc b/.cursor/rules/amazon-ec2.mdc new file mode 100644 index 000000000..4465bc6ed --- /dev/null +++ b/.cursor/rules/amazon-ec2.mdc @@ -0,0 +1,318 @@ +--- +description: This rule file provides best practices, coding standards, and security guidelines for developing, deploying, and maintaining applications using the amazon-ec2 library within the AWS ecosystem. It focuses on infrastructure as code (IaC), resource management, performance, and security considerations for robust and scalable EC2-based solutions. +globs: *.tf,*.json,*.yml,*.yaml,*.py,*.js,*.ts,*.sh,*.java,*.go,*.rb,*.m +alwaysApply: false +--- +- ## General Principles + - **Infrastructure as Code (IaC):** Treat your infrastructure as code. Define and provision AWS resources (EC2 instances, security groups, networks) using code (e.g., AWS CloudFormation, AWS CDK, Terraform). This ensures consistency, repeatability, and version control. + - **Security First:** Integrate security best practices into every stage of development, from IaC template creation to instance configuration. Implement the principle of least privilege, regularly patch instances, and utilize security assessment tools. + - **Modularity and Reusability:** Design your infrastructure and application code in modular components that can be reused across multiple projects or environments. + - **Automation:** Automate as much of the infrastructure provisioning, deployment, and management processes as possible. Use CI/CD pipelines for automated testing and deployment. + - **Monitoring and Logging:** Implement comprehensive monitoring and logging to track the health, performance, and security of your EC2 instances and applications. + +- ## 1. Code Organization and Structure + + - **Directory Structure Best Practices:** + - Adopt a logical directory structure that reflects the architecture of your application and infrastructure. + - Example: + + project-root/ + ├── modules/ # Reusable infrastructure modules (e.g., VPC, security groups) + │ ├── vpc/ # VPC module + │ │ ├── main.tf # Terraform configuration for the VPC + │ │ ├── variables.tf # Input variables for the VPC module + │ │ ├── outputs.tf # Output values for the VPC module + │ ├── security_group/ # Security Group module + │ │ ├── ... + ├── environments/ # Environment-specific configurations + │ ├── dev/ # Development environment + │ │ ├── main.tf # Terraform configuration for the Dev environment + │ │ ├── variables.tf # Environment specific variables + │ ├── prod/ # Production environment + │ │ ├── ... + ├── scripts/ # Utility scripts (e.g., deployment scripts, automation scripts) + │ ├── deploy.sh # Deployment script + │ ├── update_ami.py # Python script to update AMI + ├── application/ # Application code + │ ├── src/ # Source code + │ ├── tests/ # Unit and integration tests + ├── README.md + └── ... + + - **File Naming Conventions:** + - Use consistent and descriptive file names. + - Examples: + - `main.tf`: Main Terraform configuration file + - `variables.tf`: Terraform variables file + - `outputs.tf`: Terraform output values file + - `deploy.sh`: Deployment script + - `instance.py`: Python module for instance management + - **Module Organization:** + - Encapsulate reusable infrastructure components into modules (e.g., VPC, security groups, load balancers). + - Each module should have: + - A clear purpose. + - Well-defined input variables and output values. + - Comprehensive documentation. + - Keep modules small and focused. + - **Component Architecture:** + - Design your application as a collection of loosely coupled components. + - Each component should have: + - A well-defined interface. + - Clear responsibilities. + - Independent deployment lifecycle. + - **Code Splitting:** + - Break down large application codebases into smaller, manageable modules. + - Use lazy loading to load modules on demand, reducing initial load time. + - Example (Python): + python + # main.py + import importlib + + def load_module(module_name): + module = importlib.import_module(module_name) + return module + + # Load the module when needed + my_module = load_module('my_module') + my_module.my_function() + + +- ## 2. Common Patterns and Anti-patterns + + - **Design Patterns:** + - **Singleton:** Use when exactly one instance of a class is needed (e.g., a configuration manager). + - **Factory:** Use to create objects without specifying their concrete classes (e.g., creating different types of EC2 instances). + - **Strategy:** Use to define a family of algorithms, encapsulate each one, and make them interchangeable (e.g., different instance termination strategies). + - **Common Tasks:** + - **Creating an EC2 Instance (AWS CLI):** + bash + aws ec2 run-instances --image-id ami-xxxxxxxxxxxxxxxxx --instance-type t2.micro --key-name MyKeyPair --security-group-ids sg-xxxxxxxxxxxxxxxxx + + - **Creating an EC2 Instance (AWS CDK): + typescript + import * as ec2 from 'aws-cdk-lib/aws-ec2'; + + const vpc = new ec2.Vpc(this, 'TheVPC', { maxAzs: 3 }); + + const instance = new ec2.Instance(this, 'EC2Instance', { + vpc: vpc, + instanceType: ec2.InstanceType.of(ec2.InstanceClass.T2, ec2.InstanceSize.MICRO), + machineImage: new ec2.AmazonLinux2023Image({generation: ec2.AmazonLinuxGeneration.AMAZON_LINUX_2}), + }); + + - **Attaching an EBS Volume:** + - Ensure the EBS volume is in the same Availability Zone as the EC2 instance. + - Use the `aws ec2 attach-volume` command or the equivalent SDK call. + - **Anti-patterns:** + - **Hardcoding AWS Credentials:** Never hardcode AWS credentials in your code. Use IAM roles for EC2 instances and IAM users with restricted permissions for local development. + - **Creating Publicly Accessible S3 Buckets:** Avoid creating S3 buckets that are publicly accessible without proper security controls. + - **Ignoring Error Handling:** Always handle exceptions and errors gracefully. Provide meaningful error messages and logging. + - **Over-Permissive Security Groups:** Implement the principle of least privilege. Grant only the minimum necessary permissions to your security groups. + - **State Management:** + - Use state files (e.g., Terraform state) to track the current state of your infrastructure. + - Store state files securely (e.g., in an S3 bucket with encryption and versioning). + - Use locking mechanisms to prevent concurrent modifications to the state file. + - **Error Handling:** + - Implement robust error handling to catch exceptions and prevent application crashes. + - Use try-except blocks to handle potential errors. + - Log error messages with sufficient detail for debugging. + +- ## 3. Performance Considerations + + - **Optimization Techniques:** + - **Instance Type Selection:** Choose the appropriate EC2 instance type based on your application's requirements (CPU, memory, network). + - **EBS Optimization:** Use Provisioned IOPS (PIOPS) EBS volumes for high-performance applications. + - **Caching:** Implement caching mechanisms to reduce database load and improve response times (e.g., using Amazon ElastiCache). + - **Load Balancing:** Distribute traffic across multiple EC2 instances using an Elastic Load Balancer (ELB). + - **Auto Scaling:** Use Auto Scaling groups to automatically scale your EC2 instances based on demand. + - **Memory Management:** + - Monitor memory usage on your EC2 instances. + - Optimize application code to reduce memory consumption. + - Use memory profiling tools to identify memory leaks. + - **Bundle Size Optimization:** + - Minimize the size of your application's deployment package. + - Remove unnecessary dependencies. + - Use code minification and compression. + - Example (Python): + bash + # Create a virtual environment + python3 -m venv .venv + source .venv/bin/activate + + # Install only necessary dependencies + pip install --no-cache-dir -r requirements.txt + + # Create deployment package + zip -r deployment_package.zip * + + - **Lazy Loading:** + - Load application modules on demand to reduce initial load time. + - Use code splitting to break down large modules into smaller chunks. + - Example (JavaScript): + javascript + // main.js + async function loadModule() { + const module = await import('./my_module.js'); + module.myFunction(); + } + + loadModule(); + + +- ## 4. Security Best Practices + + - **Common Vulnerabilities:** + - **SQL Injection:** Prevent SQL injection by using parameterized queries and input validation. + - **Cross-Site Scripting (XSS):** Prevent XSS by sanitizing user input and encoding output. + - **Remote Code Execution (RCE):** Prevent RCE by validating user input and using secure coding practices. + - **Unsecured API endpoints:** Secure API endpoints using authentication and authorization mechanisms. + - **Input Validation:** + - Validate all user input to prevent malicious code from being injected into your application. + - Use regular expressions and data type validation. + - **Authentication and Authorization:** + - Use strong authentication mechanisms (e.g., multi-factor authentication). + - Implement role-based access control (RBAC) to restrict access to sensitive resources. + - Use AWS IAM roles for EC2 instances to grant access to AWS resources. + - **Data Protection:** + - Encrypt sensitive data at rest and in transit. + - Use HTTPS for all API communication. + - Store sensitive data in secure storage (e.g., AWS Secrets Manager). + - **Secure API Communication:** + - Use HTTPS for all API communication. + - Validate API requests and responses. + - Implement rate limiting to prevent abuse. + - Use AWS API Gateway to manage and secure your APIs. + +- ## 5. Testing Approaches + + - **Unit Testing:** + - Write unit tests for individual components to verify their functionality. + - Use mocking and stubbing to isolate components from external dependencies. + - Example (Python): + python + import unittest + from unittest.mock import Mock + + class MyComponent: + def __init__(self, external_dependency): + self.dependency = external_dependency + + def my_function(self, input_data): + result = self.dependency.process_data(input_data) + return result + + class TestMyComponent(unittest.TestCase): + def test_my_function(self): + # Create a mock for the external dependency + mock_dependency = Mock() + mock_dependency.process_data.return_value = "Mocked Result" + + # Create an instance of MyComponent with the mock dependency + component = MyComponent(mock_dependency) + + # Call the function to be tested + result = component.my_function("Test Input") + + # Assert the expected behavior + self.assertEqual(result, "Mocked Result") + mock_dependency.process_data.assert_called_once_with("Test Input") + + if __name__ == '__main__': + unittest.main() + + - **Integration Testing:** + - Write integration tests to verify the interaction between components. + - Test the integration of your application with AWS services. + - **End-to-End Testing:** + - Write end-to-end tests to verify the entire application flow. + - Simulate real user scenarios. + - **Test Organization:** + - Organize your tests into a logical directory structure. + - Use meaningful test names. + - Keep tests independent of each other. + - **Mocking and Stubbing:** + - Use mocking and stubbing to isolate components from external dependencies. + - Example (AWS CDK): + typescript + // Mocking AWS SDK calls in Jest + jest.mock('aws-sdk', () => { + const mEC2 = { + describeInstances: jest.fn().mockReturnValue({ + promise: jest.fn().mockResolvedValue({ Reservations: [] }), + }), + }; + return { + EC2: jest.fn().mockImplementation(() => mEC2), + }; + }); + + +- ## 6. Common Pitfalls and Gotchas + + - **Frequent Mistakes:** + - **Incorrect Security Group Configuration:** Incorrectly configured security groups can expose your EC2 instances to security risks. + - **Insufficient Resource Limits:** Exceeding AWS resource limits can cause application failures. + - **Not Using Auto Scaling:** Not using Auto Scaling can lead to performance bottlenecks and outages during periods of high demand. + - **Forgetting to Terminate Unused Instances:** Forgetting to terminate unused EC2 instances can lead to unnecessary costs. + - **Edge Cases:** + - **Spot Instance Interruptions:** Spot instances can be interrupted with short notice. Design your application to handle spot instance interruptions gracefully. + - **Network Connectivity Issues:** Network connectivity issues can prevent your application from accessing AWS services or other resources. + - **Version-Specific Issues:** + - Be aware of version-specific issues with the amazon-ec2 library or AWS services. + - Consult the documentation for the specific versions you are using. + - **Compatibility Concerns:** + - Ensure compatibility between your application and the underlying operating system and libraries. + - Test your application on different operating systems and browsers. + - **Debugging Strategies:** + - Use logging and monitoring to track the behavior of your application. + - Use debugging tools to identify and fix errors. + - Consult the AWS documentation and community forums for help. + +- ## 7. Tooling and Environment + + - **Recommended Tools:** + - **AWS CLI:** Command-line interface for interacting with AWS services. + - **AWS Management Console:** Web-based interface for managing AWS resources. + - **AWS CloudFormation:** Infrastructure as code service for provisioning and managing AWS resources. + - **AWS CDK:** Cloud Development Kit for defining cloud infrastructure in code. + - **Terraform:** Infrastructure as code tool for provisioning and managing cloud resources. + - **Packer:** Tool for creating machine images. + - **Ansible:** Configuration management tool. + - **Build Configuration:** + - Use a build tool (e.g., Make, Gradle, Maven) to automate the build process. + - Define dependencies and build steps in a build file. + - Example (Python): + makefile + # Makefile + venv: + python3 -m venv .venv + . .venv/bin/activate + pip install -r requirements.txt + + deploy: + zip -r deployment_package.zip * + aws s3 cp deployment_package.zip s3://my-bucket/deployment_package.zip + aws lambda update-function-code --function-name my-function --s3-bucket my-bucket --s3-key deployment_package.zip + + - **Linting and Formatting:** + - Use a linter (e.g., pylint, eslint) to enforce code style and identify potential errors. + - Use a formatter (e.g., black, prettier) to automatically format your code. + - **Deployment Best Practices:** + - Use a deployment pipeline to automate the deployment process. + - Deploy to a staging environment before deploying to production. + - Use blue/green deployments to minimize downtime. + - **CI/CD Integration:** + - Integrate your application with a CI/CD system (e.g., Jenkins, CircleCI, GitLab CI). + - Automate testing, building, and deployment. + +- ## Additional Considerations + + - **Cost Optimization:** Regularly review your AWS resource usage and identify opportunities for cost savings. Consider using Reserved Instances or Spot Instances to reduce costs. + - **Disaster Recovery:** Implement a disaster recovery plan to ensure business continuity in the event of an outage. Use AWS Backup or other backup solutions to protect your data. + - **Compliance:** Ensure that your application complies with relevant regulations and standards (e.g., PCI DSS, HIPAA). + +- ## References + + - [AWS EC2 Best Practices](mdc:https:/docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-best-practices.html) + - [AWS CDK Best Practices](mdc:https:/docs.aws.amazon.com/cdk/v2/guide/best-practices.html) + - [Terraform AWS Provider Best Practices](mdc:https:/docs.aws.amazon.com/prescriptive-guidance/latest/terraform-aws-provider-best-practices/structure.html) \ No newline at end of file diff --git a/.cursor/rules/amazon-s3.mdc b/.cursor/rules/amazon-s3.mdc new file mode 100644 index 000000000..de26fac20 --- /dev/null +++ b/.cursor/rules/amazon-s3.mdc @@ -0,0 +1,367 @@ +--- +description: This rule file provides comprehensive best practices, coding standards, and security guidelines for developing applications using Amazon S3. It aims to ensure secure, performant, and maintainable S3 integrations. +globs: *S3*.js,*S3*.ts,*S3*.jsx,*S3*.tsx,*S3*.py,*S3*.java,*S3*.go,*S3*.csharp +--- +- Always disable public access to S3 buckets unless explicitly needed. Use AWS Identity and Access Management (IAM) policies and bucket policies for access control instead of Access Control Lists (ACLs), which are now generally deprecated. +- Implement encryption for data at rest using Server-Side Encryption (SSE), preferably with AWS Key Management Service (KMS) for enhanced security. +- Use S3 Transfer Acceleration for faster uploads over long distances and enable versioning to protect against accidental deletions. Monitor performance using Amazon CloudWatch and enable logging for auditing. Additionally, consider using S3 Storage Lens for insights into storage usage and activity trends. +- Leverage S3's lifecycle policies to transition objects to cheaper storage classes based on access patterns, and regularly review your storage usage to optimize costs. Utilize S3 Intelligent-Tiering for automatic cost savings based on changing access patterns. + +## Amazon S3 Best Practices and Coding Standards + +This document provides comprehensive best practices, coding standards, and security guidelines for developing applications using Amazon S3. Following these guidelines will help ensure that your S3 integrations are secure, performant, maintainable, and cost-effective. + +### 1. Code Organization and Structure + +#### 1.1. Directory Structure Best Practices + +Organize your code related to Amazon S3 into logical directories based on functionality. + + +project/ +├── src/ +│ ├── s3/ +│ │ ├── utils.js # Utility functions for S3 operations +│ │ ├── uploader.js # Handles uploading files to S3 +│ │ ├── downloader.js # Handles downloading files from S3 +│ │ ├── config.js # Configuration for S3 (bucket name, region, etc.) +│ │ ├── errors.js # Custom error handling for S3 operations +│ │ └── index.js # Entry point for S3 module +│ ├── ... +│ └── tests/ +│ ├── s3/ +│ │ ├── uploader.test.js # Unit tests for uploader.js +│ │ └── ... +│ └── ... +├── ... + + +#### 1.2. File Naming Conventions + +Use descriptive and consistent file names. + +* `uploader.js`: Module for uploading files to S3. +* `downloader.js`: Module for downloading files from S3. +* `s3_service.py`: (Python Example) Defines S3 related services. +* `S3Manager.java`: (Java Example) Manages S3 client and configurations. + +#### 1.3. Module Organization + +* **Single Responsibility Principle:** Each module should have a clear and specific purpose (e.g., uploading, downloading, managing bucket lifecycle). +* **Abstraction:** Hide complex S3 operations behind simpler interfaces. +* **Configuration:** Store S3 configuration details (bucket name, region, credentials) in a separate configuration file. + +Example (JavaScript): +javascript +// s3/uploader.js +import AWS from 'aws-sdk'; +import config from './config'; + +const s3 = new AWS.S3(config.s3); + +export const uploadFile = async (file, key) => { + const params = { + Bucket: config.s3.bucketName, + Key: key, + Body: file + }; + try { + await s3.upload(params).promise(); + console.log(`File uploaded successfully: ${key}`); + } catch (error) { + console.error('Error uploading file:', error); + throw error; // Re-throw for handling in the caller. + } +}; + + +#### 1.4. Component Architecture + +For larger applications, consider a component-based architecture. This can involve creating distinct components for different S3-related tasks. For example: + +* **Upload Component:** Handles file uploads, progress tracking, and error handling. +* **Download Component:** Handles file downloads, progress tracking, and caching. +* **Management Component:** Manages bucket creation, deletion, and configuration. + +#### 1.5. Code Splitting + +If you have a large application using S3, consider using code splitting to reduce the initial load time. This involves breaking your code into smaller chunks that can be loaded on demand. This is especially relevant for front-end applications using S3 for asset storage. + +* **Dynamic Imports:** Use dynamic imports to load S3-related modules only when needed. +* **Webpack/Rollup:** Configure your bundler to create separate chunks for S3 code. + +### 2. Common Patterns and Anti-patterns + +#### 2.1. Design Patterns + +* **Strategy Pattern:** Use a strategy pattern to handle different storage classes or encryption methods. +* **Factory Pattern:** Use a factory pattern to create S3 clients with different configurations. +* **Singleton Pattern:** Use a singleton pattern if you want to use one s3 instance for all the s3 interactions. + +#### 2.2. Recommended Approaches for Common Tasks + +* **Uploading large files:** Use multipart upload for files larger than 5 MB. This allows you to upload files in parallel and resume interrupted uploads. +* **Downloading large files:** Use byte-range fetches to download files in chunks. This is useful for resuming interrupted downloads and for accessing specific portions of a file. +* **Deleting multiple objects:** Use the `deleteObjects` API to delete multiple objects in a single request. This is more efficient than deleting objects one by one. + +#### 2.3. Anti-patterns and Code Smells + +* **Hardcoding credentials:** Never hardcode AWS credentials in your code. Use IAM roles or environment variables. +* **Insufficient error handling:** Always handle errors from S3 operations gracefully. Provide informative error messages and retry failed operations. +* **Ignoring bucket access control:** Properly configure bucket policies and IAM roles to restrict access to your S3 buckets. +* **Overly permissive bucket policies:** Avoid granting overly broad permissions in your bucket policies. Follow the principle of least privilege. +* **Not using versioning:** Failing to enable versioning can lead to data loss if objects are accidentally deleted or overwritten. +* **Assuming immediate consistency:** S3 provides eventual consistency for some operations. Be aware of this and design your application accordingly. +* **Polling for object existence:** Instead of polling, use S3 events to trigger actions when objects are created or modified. +* **Inefficient data retrieval:** Avoid retrieving entire objects when only a portion of the data is needed. Use byte-range fetches or S3 Select to retrieve only the necessary data. + +#### 2.4. State Management + +* **Stateless operations:** Design your S3 operations to be stateless whenever possible. This makes your application more scalable and resilient. +* **Caching:** Use caching to reduce the number of requests to S3. Consider using a CDN (Content Delivery Network) to cache frequently accessed objects. +* **Session management:** If you need to maintain state, store session data in a separate database or cache, not in S3. + +#### 2.5. Error Handling + +* **Retry mechanism:** Implement retry logic with exponential backoff for transient errors. +* **Specific error handling:** Handle different S3 errors differently (e.g., retry 503 errors, log 403 errors). +* **Centralized error logging:** Log all S3 errors to a centralized logging system for monitoring and analysis. + +Example (Python): +python +import boto3 +from botocore.exceptions import ClientError +import time + +s3 = boto3.client('s3') + +def upload_file(file_name, bucket, object_name=None): + """Upload a file to an S3 bucket""" + if object_name is None: + object_name = file_name + + for attempt in range(3): # Retry up to 3 times + try: + response = s3.upload_file(file_name, bucket, object_name) + return True + except ClientError as e: + if e.response['Error']['Code'] == 'NoSuchBucket': + print(f"The bucket {bucket} does not exist.") + return False + elif e.response['Error']['Code'] == 'AccessDenied': + print("Access denied. Check your credentials and permissions.") + return False + else: + print(f"An error occurred: {e}") + if attempt < 2: # Wait and retry + time.sleep(2 ** attempt) + else: + return False + except Exception as e: + print(f"An unexpected error occurred: {e}") + return False + return False # Reached max retries and failed + + +### 3. Performance Considerations + +#### 3.1. Optimization Techniques + +* **Use S3 Transfer Acceleration:** If you are uploading or downloading files from a geographically distant location, use S3 Transfer Acceleration to improve performance. S3 Transfer Acceleration utilizes Amazon CloudFront's globally distributed edge locations. +* **Use multipart upload:** For large files, use multipart upload to upload files in parallel. The documentation states the best practice is to use this for files larger than 5MB. +* **Enable gzip compression:** Compress objects before uploading them to S3 to reduce storage costs and improve download times. Set the `Content-Encoding` header to `gzip` when uploading compressed objects. +* **Use HTTP/2:** Enable HTTP/2 on your S3 bucket to improve performance. +* **Optimize object sizes:** Store related data in a single object to reduce the number of requests to S3. + +#### 3.2. Memory Management + +* **Stream data:** Avoid loading entire files into memory. Use streams to process data in chunks. +* **Release resources:** Release S3 client objects when they are no longer needed. + +#### 3.3. Bundle Size Optimization + +* **Tree shaking:** Use a bundler that supports tree shaking to remove unused code from your bundle. +* **Code splitting:** Split your code into smaller chunks that can be loaded on demand. + +#### 3.4. Lazy Loading + +* **Load images on demand:** Load images from S3 only when they are visible on the screen. +* **Lazy load data:** Load data from S3 only when it is needed. + +### 4. Security Best Practices + +#### 4.1. Common Vulnerabilities + +* **Publicly accessible buckets:** Ensure that your S3 buckets are not publicly accessible. +* **Insufficient access control:** Properly configure bucket policies and IAM roles to restrict access to your S3 buckets. +* **Cross-site scripting (XSS):** Sanitize user input to prevent XSS attacks if you are serving content directly from S3. +* **Data injection:** Validate all data before storing it in S3 to prevent data injection attacks. + +#### 4.2. Input Validation + +* **Validate file types:** Validate the file types of uploaded objects to prevent malicious files from being stored in S3. +* **Validate file sizes:** Limit the file sizes of uploaded objects to prevent denial-of-service attacks. +* **Sanitize file names:** Sanitize file names to prevent directory traversal attacks. + +#### 4.3. Authentication and Authorization + +* **Use IAM roles:** Use IAM roles to grant permissions to applications running on EC2 instances or other AWS services. +* **Use temporary credentials:** Use temporary credentials for applications that need to access S3 from outside of AWS. You can use AWS STS (Security Token Service) to generate temporary credentials. +* **Principle of least privilege:** Grant only the minimum permissions required for each user or application. + +#### 4.4. Data Protection + +* **Encrypt data at rest:** Use server-side encryption (SSE) or client-side encryption to encrypt data at rest in S3. +* **Encrypt data in transit:** Use HTTPS to encrypt data in transit between your application and S3. +* **Enable versioning:** Enable versioning to protect against accidental data loss. +* **Enable MFA Delete:** Require multi-factor authentication to delete objects from S3. +* **Object locking:** Use S3 Object Lock to prevent objects from being deleted or overwritten for a specified period of time. + +#### 4.5. Secure API Communication + +* **Use HTTPS:** Always use HTTPS to communicate with the S3 API. +* **Validate certificates:** Validate the SSL/TLS certificates of the S3 endpoints. +* **Restrict access:** Restrict access to the S3 API using IAM policies and bucket policies. + +### 5. Testing Approaches + +#### 5.1. Unit Testing + +* **Mock S3 client:** Mock the S3 client to isolate your unit tests. +* **Test individual functions:** Test individual functions that interact with S3. +* **Verify error handling:** Verify that your code handles S3 errors correctly. + +Example (JavaScript with Jest): +javascript +// s3/uploader.test.js +import { uploadFile } from './uploader'; +import AWS from 'aws-sdk'; + +jest.mock('aws-sdk', () => { + const mS3 = { + upload: jest.fn().mockReturnThis(), + promise: jest.fn(), + }; + return { + S3: jest.fn(() => mS3), + }; +}); + +describe('uploadFile', () => { + it('should upload file successfully', async () => { + const mockS3 = new AWS.S3(); + mockS3.promise.mockResolvedValue({}); + const file = 'test file content'; + const key = 'test.txt'; + + await uploadFile(file, key); + + expect(AWS.S3).toHaveBeenCalledTimes(1); + expect(mockS3.upload).toHaveBeenCalledWith({ + Bucket: 'your-bucket-name', + Key: key, + Body: file + }); + }); + + it('should handle upload error', async () => { + const mockS3 = new AWS.S3(); + mockS3.promise.mockRejectedValue(new Error('Upload failed')); + const file = 'test file content'; + const key = 'test.txt'; + + await expect(uploadFile(file, key)).rejects.toThrow('Upload failed'); + }); +}); + + +#### 5.2. Integration Testing + +* **Test with real S3 buckets:** Create a dedicated S3 bucket for integration tests. +* **Test end-to-end flows:** Test complete workflows that involve S3 operations. +* **Verify data integrity:** Verify that data is correctly stored and retrieved from S3. + +#### 5.3. End-to-End Testing + +* **Simulate user scenarios:** Simulate real user scenarios to test your application's S3 integration. +* **Monitor performance:** Monitor the performance of your S3 integration under load. + +#### 5.4. Test Organization + +* **Separate test directories:** Create separate test directories for unit tests, integration tests, and end-to-end tests. +* **Descriptive test names:** Use descriptive test names that clearly indicate what is being tested. + +#### 5.5. Mocking and Stubbing + +* **Mock S3 client:** Use a mocking library (e.g., Jest, Mockito) to mock the S3 client. +* **Stub S3 responses:** Stub S3 API responses to simulate different scenarios. +* **Use dependency injection:** Use dependency injection to inject mocked S3 clients into your components. + +### 6. Common Pitfalls and Gotchas + +#### 6.1. Frequent Mistakes + +* **Forgetting to handle errors:** Failing to handle S3 errors can lead to unexpected behavior and data loss. +* **Using incorrect region:** Using the wrong region can result in connection errors and data transfer costs. +* **Exposing sensitive data:** Storing sensitive data in S3 without proper encryption can lead to security breaches. +* **Not cleaning up temporary files:** Failing to delete temporary files after uploading them to S3 can lead to storage waste. +* **Overusing public read access:** Granting public read access to S3 buckets can expose sensitive data to unauthorized users. + +#### 6.2. Edge Cases + +* **Eventual consistency:** S3 provides eventual consistency for some operations. Be aware of this and design your application accordingly. +* **Object size limits:** Be aware of the object size limits for S3. +* **Request rate limits:** Be aware of the request rate limits for S3. +* **Special characters in object keys:** Handle special characters in object keys correctly. + +#### 6.3. Version-Specific Issues + +* **SDK compatibility:** Ensure that your AWS SDK is compatible with the S3 API version. +* **API changes:** Be aware of any API changes that may affect your application. + +#### 6.4. Compatibility Concerns + +* **Browser compatibility:** Ensure that your application is compatible with different browsers if you are using S3 directly from the browser. +* **Serverless environments:** Be aware of any limitations when using S3 in serverless environments (e.g., Lambda). + +#### 6.5. Debugging Strategies + +* **Enable logging:** Enable logging to track S3 API calls and errors. +* **Use S3 monitoring tools:** Use S3 monitoring tools to monitor the performance and health of your S3 buckets. +* **Check S3 access logs:** Analyze S3 access logs to identify potential security issues. +* **Use AWS CloudTrail:** Use AWS CloudTrail to track API calls to S3. +* **Use AWS X-Ray:** Use AWS X-Ray to trace requests through your application and identify performance bottlenecks. + +### 7. Tooling and Environment + +#### 7.1. Recommended Development Tools + +* **AWS CLI:** The AWS Command Line Interface (CLI) is a powerful tool for managing S3 resources. +* **AWS SDK:** The AWS SDK provides libraries for interacting with S3 from various programming languages. +* **S3 Browser:** S3 Browser is a Windows client for managing S3 buckets and objects. +* **Cyberduck:** Cyberduck is a cross-platform client for managing S3 buckets and objects. +* **Cloudberry Explorer:** Cloudberry Explorer is a Windows client for managing S3 buckets and objects. + +#### 7.2. Build Configuration + +* **Use environment variables:** Store S3 configuration details (bucket name, region, credentials) in environment variables. +* **Use a build tool:** Use a build tool (e.g., Maven, Gradle, Webpack) to manage your project dependencies and build your application. + +#### 7.3. Linting and Formatting + +* **Use a linter:** Use a linter (e.g., ESLint, PyLint) to enforce code style and best practices. +* **Use a formatter:** Use a code formatter (e.g., Prettier, Black) to automatically format your code. + +#### 7.4. Deployment + +* **Use infrastructure as code:** Use infrastructure as code (e.g., CloudFormation, Terraform) to automate the deployment of your S3 resources. +* **Use a deployment pipeline:** Use a deployment pipeline to automate the deployment of your application. +* **Use blue/green deployments:** Use blue/green deployments to minimize downtime during deployments. + +#### 7.5. CI/CD Integration + +* **Integrate with CI/CD tools:** Integrate your S3 deployment process with CI/CD tools (e.g., Jenkins, CircleCI, Travis CI). +* **Automate testing:** Automate your unit tests, integration tests, and end-to-end tests as part of your CI/CD pipeline. +* **Automate deployments:** Automate the deployment of your application to S3 as part of your CI/CD pipeline. + +By following these best practices and coding standards, you can ensure that your Amazon S3 integrations are secure, performant, maintainable, and cost-effective. \ No newline at end of file diff --git a/.cursor/rules/android-sdk.mdc b/.cursor/rules/android-sdk.mdc new file mode 100644 index 000000000..ba8608024 --- /dev/null +++ b/.cursor/rules/android-sdk.mdc @@ -0,0 +1,311 @@ +--- +description: This rule file provides comprehensive best practices for Android SDK development, encompassing code organization, performance, security, testing, and common pitfalls. It aims to guide developers in creating robust, maintainable, and efficient Android applications and libraries. +globs: *.java,*.kt,*.xml,*.gradle,*.properties +--- +- MUST be under source control management (SCM). RECOMMENDED source control management is git +- If you intend your source repo to be public ensure you have received the necessary management and legal approval +- MUST use a .gitignore configuration tailored for Android projects (e.g., ignoring `local.properties`, `build/`, `.gradle/`) +- Source code repositories SHOULD follow the naming convention `[platform]-[library name]`, e.g., `android-analytics`, `java-datastore` +- MUST NOT restrict read access to source code repositories unnecessarily +- RECOMMEND to not have source file header comment blocks as these easily get out of date + +## Code Organization and Structure + +- SHOULD follow the standard project layout for Android projects. Key directories: + - `src/main/java`: Java/Kotlin source code + - `src/main/res`: Resources (layouts, drawables, strings, etc.) + - `src/test/java`: Unit tests + - `src/androidTest/java`: Instrumentation tests +- MUST prefix all exported resources (layout files, XML files, drawable IDs, color IDs, dimension IDs, string IDs, etc.) with a unique prefix for the library using `resourcePrefix` in the Gradle build file. Example: `sug` for a Suggestion Module. This avoids naming conflicts when the library is used in other projects. +- Use package names that are descriptive and follow the reverse domain name convention (e.g., `com.example.myapp.feature`). +- Organize code into logical modules or features. Each module should have a clear responsibility and be loosely coupled with other modules. +- Use Kotlin for new projects. If working with Java, target Java 8 or higher to utilize modern language features. +- Consider using a multi-module project structure for larger projects to improve build times and code organization. Modules can represent different features, libraries, or build variants. +- Employ a layered architecture (e.g., presentation, domain, data) to separate concerns and improve testability. + +### Directory Structure Best Practices: + + +root/ +├── app/ +│ ├── build.gradle +│ └── src/ +│ ├── androidTest/ +│ │ └── java/ +│ ├── main/ +│ │ ├── AndroidManifest.xml +│ │ ├── java/ +│ │ │ └── com/example/app/ +│ │ │ ├── MainActivity.kt +│ │ │ ├── model/ +│ │ │ ├── view/ +│ │ │ └── viewmodel/ +│ │ └── res/ +│ │ ├── layout/ +│ │ ├── drawable/ +│ │ ├── values/ +│ │ └── mipmap-xxhdpi/ +│ └── test/ +│ └── java/ +├── library_module/ +│ ├── build.gradle +│ └── src/ +│ ├── main/ +│ │ ├── java/ +│ │ └── res/ +│ └── test/ +├── build.gradle +├── settings.gradle + + +### File Naming Conventions: + +- **Kotlin/Java:** `PascalCase` for class names (e.g., `MainActivity`), `camelCase` for variables and method names (e.g., `userName`, `calculateTotal`). +- **Layout XML:** `snake_case` (e.g., `activity_main.xml`). Use descriptive names reflecting the layout's purpose. +- **Drawable XML:** `snake_case` (e.g., `ic_launcher.xml`). Prefix icons with `ic_`, backgrounds with `bg_`, and other drawables accordingly. +- **Values XML:** `snake_case` (e.g., `strings.xml`, `colors.xml`, `dimens.xml`). +- **Gradle:** `build.gradle` (module-level), `settings.gradle` (project-level). + +### Component Architecture Recommendations: + +- MVVM (Model-View-ViewModel) is the recommended architecture pattern. Use Android Architecture Components (LiveData, ViewModel, Room) to implement it effectively. +- For complex UI logic, consider using a ViewBindingDelegate to simplify view access and reduce boilerplate. +- Use dependency injection (Hilt or Dagger) to manage dependencies and improve testability. + +## Common Patterns and Anti-patterns + +- Use `LiveData` and `Flow` for reactive data streams. +- Utilize coroutines for asynchronous operations to simplify threading and avoid callback hell. +- Avoid performing long-running operations on the main thread (UI thread). Use `Dispatchers.IO` or `Dispatchers.Default` with coroutines or `AsyncTask` for background tasks. +- Don't use `AsyncTask` if possible, prefer coroutines. +- Prefer `ConstraintLayout` over other layout types for complex UIs as it is more flexible and performant. +- Use `RecyclerView` for displaying large lists of data efficiently. Implement view holder pattern correctly for optimal performance. +- Apply data binding to reduce boilerplate code in UI updates. +- Use sealed classes for representing state to ensure exhaustive handling of all possible states. +- Implement proper error handling with try-catch blocks and report errors gracefully to the user. +- Log errors and exceptions using a logging framework (Timber) for debugging and analysis. + +### Anti-patterns and Code Smells to Avoid: + +- **God Classes:** Classes that are too large and have too many responsibilities. Break them down into smaller, more focused classes. +- **Spaghetti Code:** Code that is difficult to read and understand due to complex control flow. Refactor it to improve readability and maintainability. +- **Copy-Paste Programming:** Duplicating code across multiple locations. Extract the common code into reusable functions or classes. +- **Hardcoding Values:** Avoid hardcoding values directly in the code. Use resources (strings, colors, dimensions) for better maintainability and localization. +- **Leaking Context:** Holding onto `Activity` or `Context` instances for longer than necessary, leading to memory leaks. + +### State Management Best Practices: + +- Use `ViewModel` to store UI-related data that survives configuration changes. +- Utilize `SavedStateHandle` in `ViewModel` to persist data across process death. +- Consider using a state management library (e.g., Redux, MVI) for complex applications with complex state. +- Implement appropriate caching strategies to reduce network requests and improve performance. + +### Error Handling Patterns: + +- Use try-catch blocks to handle potential exceptions and prevent app crashes. +- Report errors gracefully to the user with informative error messages. +- Log errors and exceptions using a logging framework (Timber) for debugging and analysis. +- Implement retry mechanisms for transient errors (e.g., network connection issues). + +## Performance Considerations + +- Optimize image loading using libraries like Glide or Coil. Resize images to the required dimensions before loading them into `ImageViews`. +- Use efficient data structures (e.g., `SparseArray` instead of `HashMap` for integer keys). +- Avoid creating unnecessary objects, especially in loops or frequently called methods. +- Use `StringBuilder` instead of string concatenation for building strings efficiently. +- Minimize network requests and optimize data transfer using compression and pagination. +- Analyze app performance using profiling tools (Android Profiler) to identify bottlenecks. +- Utilize the LeakCanary library to detect memory leaks during development. +- Use ProGuard or R8 to shrink, obfuscate, and optimize the code for release builds. + +### Memory Management Considerations: + +- Avoid creating memory leaks by properly managing object lifecycles and releasing resources when they are no longer needed. +- Use `WeakReference` to hold references to objects that may be garbage collected to prevent memory leaks. +- Monitor memory usage using the Android Profiler and identify potential memory leaks. + +### Rendering Optimization: + +- Minimize overdraw by reducing the number of layers that are drawn on top of each other. +- Use hardware acceleration to improve rendering performance. +- Optimize custom views by implementing `onDraw()` method efficiently and avoiding unnecessary allocations. + +### Bundle Size Optimization: + +- Remove unused resources using `shrinkResources` in the Gradle build file. +- Use vector drawables instead of raster images for icons and other simple graphics. +- Enable code shrinking and obfuscation using ProGuard or R8. +- Compress images using tools like TinyPNG or ImageOptim. +- Use app bundles to deliver optimized APKs for different device configurations. +- Utilize dynamic feature modules to defer the download of non-essential features. + +### Lazy Loading Strategies: + +- Implement lazy loading for images and other resources that are not immediately visible on the screen. +- Use pagination or infinite scrolling to load data in chunks as the user scrolls down the list. + +## Security Best Practices + +- MUST use Kotlin or Java. Kotlin is RECOMMENDED +- SHOULD use static code analysis, such as detekt or SwiftLint to minimise errors and minimise debates and formatting fix-up commits in PRs. It is RECOMMENDED to integrate these tools with the CI workflow. SHOULD use quality tools available in common config: Android and iOS +- Android: MUST use the Android Support Annotations for public APIs. Non-Android Java code MUST use the Jetbrains Annotations for public APIs +- Secure API communication and proper permission handling are vital to protect user data. +- MUST use a `consumerProguardFiles` file if obfuscation breaks the libraries function. +- SHOULD NOT have unnecessary exported dependencies, in particular SHOULD NOT leak networking library to consumers. SHOULD NOT leak serialization library to consumers. An exception to this rule are optional adapters for popular libraries. + +### Common Vulnerabilities and Prevention: + +- **SQL Injection:** Avoid using raw SQL queries. Use parameterized queries or an ORM like Room to prevent SQL injection attacks. +- **Cross-Site Scripting (XSS):** Sanitize user input to prevent XSS attacks. +- **Man-in-the-Middle (MITM) Attacks:** Use HTTPS for all network communication to encrypt data in transit and prevent MITM attacks. +- **Insecure Data Storage:** Encrypt sensitive data before storing it on the device. +- **Improper Platform Usage:** Ensure that permissions are properly requested and handled. Review and minimize the permissions your app requests. + +### Input Validation: + +- Validate all user input on both the client and server sides to prevent malicious data from being processed. +- Use regular expressions or validation libraries to enforce input constraints. + +### Authentication and Authorization: + +- Use secure authentication protocols (e.g., OAuth 2.0, OpenID Connect) for user authentication. +- Implement proper authorization mechanisms to control access to resources based on user roles. +- Store user credentials securely using hashing and salting. + +### Data Protection: + +- Encrypt sensitive data at rest using AES or other strong encryption algorithms. +- Securely transmit data over the network using HTTPS. +- Implement data masking or anonymization techniques to protect user privacy. + +### Secure API Communication: + +- Use HTTPS for all API communication. +- Verify the server's SSL certificate to prevent MITM attacks. +- Implement API rate limiting to prevent denial-of-service attacks. + +## Testing Approaches + +- All library (sub-)projects MUST generate coverage test reports (e.g. jacoco, Xcode/slather). +- Unit test coverage MUST be at least 80% and SHOULD be above 85% +- Unit tests MUST NOT rely on reflection to expose otherwise private/internal classes or members. +- Tests SHOULD use JUnit, and MAY use helper libraries such as Mockito or Robolectric + +### Unit Testing Strategies: + +- Write unit tests for all critical components, including `ViewModels`, repositories, and use cases. +- Use mocking frameworks (Mockito) to isolate units under test and mock dependencies. +- Follow the AAA (Arrange, Act, Assert) pattern for writing clear and concise unit tests. + +### Integration Testing Approaches: + +- Write integration tests to verify the interaction between different components, such as `ViewModels` and repositories. +- Use dependency injection to provide mock dependencies for integration tests. + +### End-to-End Testing: + +- Use UI testing frameworks (Espresso, UI Automator) to write end-to-end tests that simulate user interactions. + +### Test Organization: + +- Organize tests into separate directories based on the type of tests (unit, integration, UI). +- Use descriptive names for test classes and methods to clearly indicate what is being tested. + +### Mocking and Stubbing: + +- Use mocking frameworks (Mockito) to create mock objects for dependencies. +- Use stubbing to define the behavior of mock objects for specific test cases. + +## Common Pitfalls and Gotchas + +- Frequent mistakes developers make when using android-sdk +- Edge cases to be aware of when using android-sdk +- Version-specific issues with android-sdk +- Compatibility concerns between android-sdk and other technologies +- Debugging strategies for android-sdk applications + +### Frequent Mistakes: + +- Not handling configuration changes properly, leading to data loss or unexpected behavior. +- Performing long-running operations on the main thread, causing the UI to freeze. +- Leaking `Context` objects, leading to memory leaks. +- Not properly validating user input, leading to security vulnerabilities. +- Not handling exceptions properly, leading to app crashes. +- Overusing `AsyncTask` and not managing thread pools correctly. + +### Edge Cases: + +- Handling network connectivity issues gracefully. +- Managing different screen sizes and densities. +- Dealing with low-memory conditions. +- Handling background tasks properly to avoid ANRs (Application Not Responding). +- Handling different Android versions and API levels. + +### Version-Specific Issues: + +- Be aware of API level differences and use `@TargetApi` and `@RequiresApi` annotations to handle them correctly. +- Test your app on different Android versions to ensure compatibility. + +### Compatibility Concerns: + +- Ensure compatibility with different device manufacturers and hardware configurations. +- Test your app on different screen sizes and densities. + +### Debugging Strategies: + +- Use the Android Debug Bridge (ADB) to connect to devices and emulators for debugging. +- Use the Android Profiler to analyze app performance and identify bottlenecks. +- Use logging statements to track the flow of execution and identify errors. +- Use the debugger to step through the code and inspect variables. + +## Tooling and Environment + +### Recommended Tools: + +- Android Studio: The official IDE for Android development. +- Gradle: The build system for Android projects. +- ADB (Android Debug Bridge): A command-line tool for communicating with Android devices and emulators. +- Android Profiler: A tool for analyzing app performance. +- LeakCanary: A library for detecting memory leaks. +- Timber: A logging library for Android. + +### Build Configuration: + +- MUST use gradle with gradle wrapper to build Android library or Java library +- MUST have consistent minSdk and supportLib versions with other SDKs +- SHOULD use shared build configuration for consistent version with other SDKs/libraries +- MAY use manual configuration with values (same as in shared build configuration), if so SHOULD use a recent minSdk version unless there is an engineering or product decision to support older versions +- SHOULD use latest versions of Android X libraries (if required) + +- Configure build variants for different environments (debug, release, staging). +- Use ProGuard or R8 to shrink, obfuscate, and optimize the code for release builds. +- Configure signing configurations to sign the APK for distribution. + +### Linting and Formatting: + +- Enforce code style guidelines using lint and code formatters (e.g., ktlint for Kotlin). +- Configure lint to run automatically during the build process. +- Resolve lint warnings and errors to improve code quality. + +### Deployment: + +- Publish libraries to a Maven repository (Maven Central or a private repository). +- Deploy apps to the Google Play Store. +- Implement beta testing using Google Play Console. + +### CI/CD Integration: + +- Integrate the build process with a CI/CD system (e.g., Jenkins, CircleCI, GitHub Actions). +- Automate testing, linting, and code formatting as part of the CI/CD pipeline. +- Automate deployment to the Google Play Store. + +## Additional Considerations +- Limit impact on consumers. SHOULD limit dependencies as much as possible primarily due to risk of version clashes +- RECOMMENDED not to have any exported dependency on other SDK modules e.g. if your module needs authentication, do not depend on Authentication SDK. Instead, require your library consumer to pass an access token string to your API instead. +- Expose minimal Surface area. Keep everything private unless it needs to be public. Every public class, method, property & resource MUST have a strong reason for being public. MAY keep internal source code in a /Private folder to allow tools to more easily ignore those files +- MUST hide classes from generated API documentation that are required to be public solely for technical reasons and are not intended for SDK consumers +- SHOULD declare classes as final to avoid unexpected high-jacking of SDK behavior by subclasses +- For better testability of your library it is RECOMMENDED to use dependency injection +- MUST follow Semantic Versioning +- MUST tag source control management revision/commit that is published +- RECOMMENDED For Android: Use gradle-versions-plugin for automated versioning by git tag \ No newline at end of file diff --git a/.cursor/rules/angular.mdc b/.cursor/rules/angular.mdc new file mode 100644 index 000000000..1abddcae4 --- /dev/null +++ b/.cursor/rules/angular.mdc @@ -0,0 +1,278 @@ +--- +description: This rule provides comprehensive guidelines for Angular development, covering coding standards, best practices, performance optimization, security considerations, and testing approaches to ensure maintainable, scalable, and high-performing applications. +globs: *.ts,*.html,*.scss,*.css +--- +# Angular Best Practices and Coding Standards + +This document outlines comprehensive best practices and coding standards for developing Angular applications. Adhering to these guidelines will promote maintainability, scalability, performance, and security. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure Best Practices + +- **Feature-based Modules:** Organize your application into feature modules, where each module encapsulates a specific feature or functionality. +- **Shared Module:** Create a `shared` module for commonly used components, directives, pipes, and services. +- **Core Module:** Create a `core` module for application-wide services and singleton instances (e.g., authentication service, configuration service). +- **Lazy-loaded Modules:** Group related features into lazy-loaded modules to improve initial load time. +- **Directory structure example:** + + + src/ + ├── app/ + │ ├── core/ + │ │ ├── core.module.ts + │ │ └── auth.service.ts + │ ├── shared/ + │ │ ├── shared.module.ts + │ │ └── components/ + │ │ └── ... + │ ├── features/ + │ │ ├── dashboard/ + │ │ │ ├── dashboard.module.ts + │ │ │ ├── dashboard.component.ts + │ │ │ └── dashboard.component.html + │ │ ├── user-management/ + │ │ │ ├── user-management.module.ts + │ │ │ └── ... + │ │ └── ... + │ ├── app.component.ts + │ └── app.module.ts + └── ... + + +### 1.2. File Naming Conventions + +- **Consistent Naming:** Use a consistent naming pattern for all files and symbols, following the `feature.type.ts` convention (e.g., `user.component.ts`, `user.service.ts`). +- **Type Abbreviations:** Use abbreviations for file types (e.g., `component.ts` -> `.component.ts`, `service.ts` -> `.service.ts`, `module.ts` -> `.module.ts`). +- **Descriptive Names:** Use descriptive names that clearly indicate the purpose of the file. + +### 1.3. Module Organization + +- **Single Responsibility:** Each module should have a single responsibility. +- **Declarations:** Declare components, directives, and pipes within the appropriate module. +- **Imports:** Import only the necessary modules in each module. +- **Exports:** Export components, directives, and pipes that need to be used by other modules. +- **forRoot() pattern:** Use the `forRoot()` pattern for modules that provide singleton services to ensure they are only instantiated once in the application. + +### 1.4. Component Architecture + +- **Smart vs. Dumb Components:** Distinguish between smart (container) components that handle data and logic, and dumb (presentational) components that focus on rendering UI. +- **Component Reusability:** Design components for reusability. +- **Input and Output Properties:** Use `@Input()` and `@Output()` properties to pass data and events between components. +- **Change Detection:** Be mindful of change detection strategies (e.g., `OnPush`) to optimize rendering performance. + +### 1.5. Code Splitting Strategies + +- **Lazy Loading:** Implement lazy loading for modules to reduce the initial bundle size and improve load time. +- **Route-based Code Splitting:** Split your application into modules based on routes. +- **Feature-based Code Splitting:** Split your application into modules based on features. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns Specific to Angular + +- **Dependency Injection (DI):** Use DI to manage dependencies between components and services. +- **Observables:** Use RxJS Observables for handling asynchronous data streams. +- **Services:** Use services for encapsulating reusable business logic. +- **Pipes:** Use pipes for transforming data in templates. +- **Directives:** Use directives for manipulating the DOM. + +### 2.2. Recommended Approaches for Common Tasks + +- **Data Binding:** Use data binding (`{{ }}`, `[]`, `()`) to synchronize data between the component and the template. +- **Event Handling:** Use event binding (`()`) to handle user interactions. +- **Form Handling:** Use reactive forms or template-driven forms to manage user input. +- **HTTP Requests:** Use the `HttpClient` to make HTTP requests to backend APIs. + +### 2.3. Anti-patterns and Code Smells to Avoid + +- **Deeply Nested Components:** Avoid deeply nested component hierarchies, which can make it difficult to manage state and events. +- **Large Components:** Break large components into smaller, reusable components. +- **Logic in Templates:** Avoid complex logic in templates; move it to the component class. +- **Direct DOM Manipulation:** Avoid direct DOM manipulation; use Angular's data binding and directives instead. +- **Unnecessary Subscriptions:** Unsubscribe from Observables to prevent memory leaks. Use the `async` pipe in templates to automatically handle subscriptions and unsubscriptions. + +### 2.4. State Management Best Practices + +- **Choose a State Management Library:** Consider using a state management library like NgRx, Akita, or MobX for complex applications. +- **Centralized State:** Store application state in a central store. +- **Immutability:** Treat state as immutable to simplify change detection and debugging. +- **Actions and Reducers:** Use actions to describe state changes and reducers to update the state. +- **Selectors:** Use selectors to efficiently retrieve data from the store. + +### 2.5. Error Handling Patterns + +- **Centralized Error Handling:** Implement a centralized error handling mechanism using an `ErrorHandler`. +- **Error Interceptors:** Use HTTP interceptors to handle errors from backend APIs. +- **User-Friendly Error Messages:** Display user-friendly error messages to the user. +- **Logging:** Log errors to a central logging service for debugging and monitoring. +- **Retry mechanism:** Implement retry mechanism for failed requests using RxJS retry operators. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +- **Change Detection Optimization:** Use `OnPush` change detection strategy for components that only depend on their input properties. +- **TrackBy Function:** Use the `trackBy` function with `ngFor` to optimize rendering of lists. +- **Virtualization:** Use virtual scrolling for large lists to improve performance. +- **Ahead-of-Time (AOT) Compilation:** Use AOT compilation to compile templates during the build process, improving startup time. +- **Minification and Uglification:** Minify and uglify your code to reduce bundle size. + +### 3.2. Memory Management + +- **Unsubscribe from Observables:** Unsubscribe from Observables when they are no longer needed to prevent memory leaks. +- **Avoid Circular Dependencies:** Avoid circular dependencies, which can lead to memory leaks. +- **Detach Event Listeners:** Detach event listeners when they are no longer needed. + +### 3.3. Rendering Optimization + +- **Minimize DOM Manipulation:** Minimize DOM manipulation, as it can be expensive. +- **Use CSS Transforms:** Use CSS transforms instead of modifying layout properties. +- **Debouncing and Throttling:** Use debouncing and throttling to reduce the frequency of event handlers. + +### 3.4. Bundle Size Optimization + +- **Lazy Loading:** Implement lazy loading for modules to reduce the initial bundle size. +- **Tree Shaking:** Use tree shaking to remove unused code from your bundle. +- **Code Splitting:** Split your application into smaller bundles to improve load time. +- **Image Optimization:** Optimize images to reduce their file size. +- **Vendor Libraries:** Use only necessary vendor libraries, and ensure they are up-to-date for optimized versions. + +### 3.5. Lazy Loading Strategies + +- **Route-based Lazy Loading:** Load modules when navigating to a specific route. +- **Feature-based Lazy Loading:** Load modules based on feature requirements. +- **Preloading:** Preload modules in the background to improve perceived performance. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities and How to Prevent Them + +- **Cross-Site Scripting (XSS):** Prevent XSS attacks by sanitizing user input and avoiding the use of `innerHTML`. +- **Cross-Site Request Forgery (CSRF):** Prevent CSRF attacks by using anti-CSRF tokens. +- **Injection Attacks:** Prevent injection attacks by validating user input and using parameterized queries. + +### 4.2. Input Validation + +- **Client-Side Validation:** Perform client-side validation to provide immediate feedback to the user. +- **Server-Side Validation:** Perform server-side validation to ensure data integrity. +- **Whitelist Validation:** Use whitelist validation to allow only specific characters or patterns. + +### 4.3. Authentication and Authorization Patterns + +- **Authentication:** Implement authentication to verify the user's identity. +- **Authorization:** Implement authorization to control access to resources based on the user's roles and permissions. +- **JSON Web Tokens (JWT):** Use JWTs for secure authentication and authorization. +- **OAuth 2.0:** Use OAuth 2.0 for delegating authorization to third-party applications. + +### 4.4. Data Protection Strategies + +- **Encryption:** Encrypt sensitive data at rest and in transit. +- **Hashing:** Hash passwords before storing them in the database. +- **Data Masking:** Mask sensitive data to prevent unauthorized access. + +### 4.5. Secure API Communication + +- **HTTPS:** Use HTTPS for secure communication between the client and the server. +- **API Keys:** Use API keys to authenticate API requests. +- **Rate Limiting:** Implement rate limiting to prevent abuse of your API. + +## 5. Testing Approaches + +### 5.1. Unit Testing Strategies + +- **Test Driven Development (TDD):** Write tests before writing code. +- **Component Testing:** Test components in isolation using mock data. +- **Service Testing:** Test services in isolation using dependency injection. +- **Pipe Testing:** Test pipes to ensure they transform data correctly. + +### 5.2. Integration Testing + +- **Module Testing:** Test modules to ensure they integrate correctly. +- **Component Interaction Testing:** Test the interaction between components. +- **HTTP Testing:** Test HTTP requests and responses. + +### 5.3. End-to-End Testing + +- **User Interface (UI) Testing:** Test the UI to ensure it behaves as expected. +- **Workflow Testing:** Test complete workflows to ensure they function correctly. +- **Cross-Browser Testing:** Test your application in different browsers. + +### 5.4. Test Organization + +- **Test Directory Structure:** Mirror your source code directory structure in your test directory. +- **Test File Naming:** Use consistent naming conventions for test files (e.g., `user.component.spec.ts`). +- **Test Suites:** Group related tests into test suites. + +### 5.5. Mocking and Stubbing + +- **Mock Dependencies:** Mock dependencies to isolate units of code for testing. +- **Stub HTTP Requests:** Stub HTTP requests to avoid making actual API calls during testing. +- **Use Mocking Libraries:** Use mocking libraries like Jasmine or Jest to simplify mocking and stubbing. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes Developers Make + +- **Forgetting to Unsubscribe from Observables:** Always unsubscribe from Observables to prevent memory leaks. +- **Using `any` Type:** Avoid using the `any` type, as it bypasses type checking. +- **Not Handling Errors:** Handle errors gracefully to prevent unexpected behavior. +- **Ignoring Performance:** Optimize your code for performance from the beginning. + +### 6.2. Edge Cases to Be Aware Of + +- **Empty Arrays:** Handle empty arrays correctly. +- **Null Values:** Handle null values correctly. +- **Unexpected Input:** Validate user input to prevent unexpected behavior. + +### 6.3. Version-Specific Issues + +- **Breaking Changes:** Be aware of breaking changes in new versions of Angular and third-party libraries. +- **Deprecated APIs:** Avoid using deprecated APIs. +- **Migration Guides:** Follow migration guides when upgrading to new versions of Angular. + +### 6.4. Compatibility Concerns + +- **Browser Compatibility:** Test your application in different browsers to ensure compatibility. +- **Device Compatibility:** Test your application on different devices to ensure compatibility. + +### 6.5. Debugging Strategies + +- **Browser Developer Tools:** Use browser developer tools to debug your code. +- **Logging:** Use logging to track the flow of execution and identify errors. +- **Debugging Tools:** Use debugging tools like VS Code's debugger to step through your code. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +- **VS Code:** Use VS Code as your code editor. +- **Angular CLI:** Use the Angular CLI for scaffolding, building, and deploying your application. +- **Chrome Developer Tools:** Use Chrome Developer Tools for debugging and performance profiling. + +### 7.2. Build Configuration + +- **Angular CLI Configuration:** Configure your Angular CLI settings in the `angular.json` file. +- **Environment Variables:** Use environment variables to configure your application for different environments. +- **Build Optimization:** Configure your build process to optimize your code for production. + +### 7.3. Linting and Formatting + +- **ESLint:** Use ESLint to enforce coding standards. +- **Prettier:** Use Prettier to format your code automatically. +- **Husky:** Use Husky to run linters and formatters before committing code. + +### 7.4. Deployment Best Practices + +- **Continuous Integration/Continuous Deployment (CI/CD):** Use a CI/CD pipeline to automate the build, test, and deployment process. +- **Deployment Environments:** Use different deployment environments for development, testing, and production. +- **CDN:** Use a content delivery network (CDN) to serve static assets. + +### 7.5. CI/CD Integration + +- **Jenkins:** Use Jenkins for CI/CD. +- **GitHub Actions:** Use GitHub Actions for CI/CD. +- **Azure DevOps:** Use Azure DevOps for CI/CD. +- **GitLab CI:** Use GitLab CI for CI/CD. + +By following these best practices and coding standards, you can build robust, maintainable, and scalable Angular applications. \ No newline at end of file diff --git a/.cursor/rules/ansible.mdc b/.cursor/rules/ansible.mdc new file mode 100644 index 000000000..3325f4eca --- /dev/null +++ b/.cursor/rules/ansible.mdc @@ -0,0 +1,553 @@ +--- +description: This rule file provides comprehensive best practices and coding standards for Ansible projects, covering code organization, common patterns, performance, security, testing, pitfalls, and tooling. It aims to improve maintainability, reusability, and efficiency in Ansible automation. +globs: *.yml,*.yaml,*.j2 +--- +- Adopt and adhere to the following Ansible best practices to ensure maintainable, reusable, and efficient automation code. This guide provides standards for code organization, common patterns, performance optimization, security, testing, tooling, and common pitfalls to avoid. + +## 1. Code Organization and Structure + +- Structure your Ansible projects in a clear and consistent manner to enhance readability and maintainability. + +### 1.1 Directory Structure Best Practices + +- **Root Directory:** + - `ansible.cfg`: Ansible configuration file. This should live in the root directory of your project. + - `inventory/`: Contains your inventory files (hosts, groups, and variables). + - `hosts`: Main inventory file. + - `group_vars/`: Directory for group-specific variables. + - `all/`: Contains variables that apply to all hosts. + - `vars.yml`: Primary file for common variables. + - `vault.yml`: Encrypted variables. + - `/`: Directory for variables specific to a group. + - `vars.yml`: Group-specific variables. + - `vault.yml`: Encrypted group variables. + - `host_vars/`: Directory for host-specific variables. + - `/`: Directory for a specific host. + - `vars.yml`: Host-specific variables. + - `vault.yml`: Encrypted host variables. + - `playbooks/`: Directory for your Ansible playbooks. + - `site.yml`: Main playbook, orchestrating other playbooks. + - `.yml`: Playbooks tailored to different environments (e.g., staging.yml, production.yml). + - `.yml`: Playbooks for specific components (e.g., webservers.yml, databases.yml). + - `roles/`: Directory for reusable Ansible roles. + - `/`: + - `tasks/`: Contains tasks for the role. + - `main.yml`: Entry point for the role's tasks. + - `handlers/`: Contains handlers for the role. + - `main.yml`: Entry point for the role's handlers. + - `defaults/`: Contains default variable values for the role. + - `main.yml`: Default variable values. + - `vars/`: Contains role-specific variables. + - `main.yml`: Role-specific variables. + - `meta/`: Contains metadata about the role. + - `main.yml`: Role metadata (e.g., dependencies, author). + - `templates/`: Contains Jinja2 templates for the role. + - `files/`: Contains static files used by the role. + - `library/`: (Optional) Custom Ansible modules. + - `filter_plugins/`: (Optional) Custom Jinja2 filter plugins. + - `lookup_plugins/`: (Optional) Custom lookup plugins. + - `plugins/`: All of the above (library, filter, lookup, etc) should be placed in respective subfolders in this directory + - `requirements.yml`: Lists roles to be downloaded from Ansible Galaxy. + - `README.md`: Project documentation. + +### 1.2 File Naming Conventions + +- **Playbooks:** + - Use lowercase with hyphens for readability (e.g., `webservers.yml`, `database-setup.yml`). + - Name your main playbook `site.yml` for consistency. +- **Roles:** + - Use lowercase with underscores (e.g., `common_setup`, `nginx_configuration`). + - Role names should be descriptive and concise. +- **Variables Files:** + - Use `vars.yml` for general variables and `defaults.yml` for role defaults. + - Use `vault.yml` for encrypted variables. + - Group-specific variables: `.yml`. + - Host-specific variables: `.yml`. +- **Tasks and Handlers:** + - `main.yml` as the entry point for tasks and handlers within roles. +- **Templates:** + - Give descriptive names to templates, using the `.j2` extension (e.g., `nginx.conf.j2`, `application.properties.j2`). +- **Inventory:** + - Name your inventory file `hosts` or follow a consistent naming convention, such as `_hosts` (e.g., `staging_hosts`, `production_hosts`). + +### 1.3 Module Organization + +- Keep custom modules in the `library/` directory at the project root. +- Organize modules into subdirectories based on functionality (e.g., `library/custom_modules/database`, `library/custom_modules/monitoring`). +- Document each custom module with clear usage examples and return values. +- Utilize `module_utils` to create shared code used by multiple modules. +- Follow a consistent naming convention for custom modules (e.g., `my_custom_module.py`). +- All modules should contain argument spec and proper return values (changed, failed, original_message, message, etc). + +### 1.4 Component Architecture Recommendations + +- **Roles as Building Blocks:** + - Divide your infrastructure into logical components (e.g., web servers, databases, load balancers) and create roles for each. + - Keep roles single-purposed and focused on specific functionality to promote reusability. +- **Playbooks for Orchestration:** + - Use playbooks to orchestrate roles and define the desired state of your infrastructure. + - The main `site.yml` playbook should act as the entry point, calling other component-specific playbooks. +- **Variables for Configuration:** + - Externalize configuration data using variables in `group_vars/` and `host_vars/`. + - Use default variables within roles to provide sane defaults and allow for customization. +- **Handlers for Event-Driven Configuration:** + - Use handlers to manage service restarts and other event-driven tasks. + - Ensure handlers are idempotent and only triggered when necessary. + +### 1.5 Code Splitting Strategies + +- **Role Decomposition:** + - Break down complex roles into smaller, more manageable roles to improve reusability and maintainability. +- **Task Includes:** + - Use `include_tasks` to split long task lists into smaller, more focused files. +- **Dynamic Includes:** + - Employ `include_vars`, `include_role`, and `include_tasks` with dynamic variables to load configuration data or tasks based on conditions. +- **Blocks for Logical Grouping:** + - Use blocks to group related tasks, providing better error handling and readability. + +## 2. Common Patterns and Anti-patterns + +- Learn and apply common Ansible design patterns to solve infrastructure automation problems efficiently while avoiding pitfalls. + +### 2.1 Design Patterns + +- **Idempotency:** + - Design tasks to be idempotent, meaning they should produce the same result regardless of how many times they are executed. + - Use the `changed_when` and `failed_when` directives to customize task result conditions. +- **Variable Prioritization:** + - Understand Ansible's variable precedence and use it effectively to override default values. + - Variable precedence order (highest to lowest): extra vars (-e), command line arguments (-e), role vars, include vars, block vars, task vars, role defaults, inventory vars, registered vars, facts. +- **Handlers for Service Management:** + - Utilize handlers to restart services when configuration files change. Use `notify` directive in tasks to trigger handlers. +- **Template Management:** + - Use the `template` module to manage configuration files. Keep templates simple and focused on configuration logic. +- **Looping with `with_items`:** + - Use `with_items` or `loop` to iterate over lists of items, such as packages to install or users to create. +- **Conditions with `when`:** + - Use the `when` directive to conditionally execute tasks based on variables or facts. +- **Delegation with `delegate_to`:** + - Use the `delegate_to` directive to run tasks on a specific host, such as when managing a load balancer or database server. +- **Error Handling with `block` and `rescue`:** + - Wrap tasks in `block` directives and use `rescue` to handle errors gracefully. +- **Register variables for complex tasks:** + - Register variables to capture output from a task and use it in subsequent tasks. +- **Meta Tasks for Role Management:** + - Use meta tasks (e.g., `meta: clear_host_errors`, `meta: end_play`) to manage role execution and handle errors. +- **Facts for dynamic configuration:** + - Utilize facts to gather system information and dynamically configure tasks. +- **Loops for repetitive tasks:** + - Use loops to perform repetitive tasks, such as creating multiple users or installing multiple packages. Always name the loop so the end user can identify the tasks better. + +### 2.2 Recommended Approaches for Common Tasks + +- **Package Management:** + - Use the appropriate package manager module (e.g., `apt`, `yum`, `dnf`, `homebrew`) for the target system. + - Always specify the `state` (e.g., `present`, `latest`, `absent`) when managing packages. +- **File Management:** + - Use the `file` module to create, modify, or delete files and directories. + - Use the `copy` module to copy files from the control node to the target nodes. + - Use the `template` module to manage configuration files with Jinja2 templates. +- **Service Management:** + - Use the `service` or `systemd` module to manage services. + - Always specify the `state` (e.g., `started`, `stopped`, `restarted`) when managing services. + - Use handlers to restart services when configuration files change. +- **User Management:** + - Use the `user` module to create, modify, or delete users. + - Use the `group` module to manage groups. + - Use the `authorized_key` module to manage SSH keys. +- **Cron Job Management:** + - Use the `cron` module to create, modify, or delete cron jobs. +- **Firewall Management:** + - Use the `firewalld` or `ufw` module to manage firewalls. +- **SELinux Management:** + - Use the `selinux` module to manage SELinux policies. +- **Line-in-File management:** + - Use the `lineinfile` module to ensure a specific line exists in a file. + - Use the `blockinfile` module to ensure a block of lines exist in a file. + +### 2.3 Anti-patterns and Code Smells + +- **Shell and Command Modules:** + - Avoid using the `shell` and `command` modules unless absolutely necessary. + - Prefer Ansible's built-in modules for specific tasks, as they provide idempotency and better error handling. +- **Hardcoded Values:** + - Avoid hardcoding values in your playbooks or roles. + - Use variables in `group_vars/` and `host_vars/` to externalize configuration data. +- **Inconsistent Naming Conventions:** + - Follow a consistent naming convention for variables, tasks, roles, and files. + - This improves readability and maintainability. +- **Unnecessary Complexity:** + - Keep your playbooks and roles as simple as possible. + - Avoid over-engineering or introducing unnecessary complexity. +- **Lack of Idempotency:** + - Ensure your tasks are idempotent to avoid unintended side effects. + - Use the `changed_when` and `failed_when` directives to customize task result conditions. +- **Ignoring Errors:** + - Always handle errors gracefully by using `block` and `rescue` directives. + - Use the `fail` module to explicitly fail a playbook when a critical error occurs. +- **Storing Secrets in Plain Text:** + - Never store sensitive data in plain text. + - Use Ansible Vault to encrypt variables and files. +- **Lack of Documentation:** + - Always document your playbooks, roles, and custom modules. + - This improves collaboration and makes it easier to maintain your automation code. +- **Copying the same block of code:** + - Refactor and call the common tasks using `include_tasks` to improve readability and maintainability +- **Using loops when a module can handle multiple arguments:** + - Some modules can perform multiple tasks by passing a list of arguments (e.g. `yum` module), so avoid using loops as much as possible. +- **Creating complex templates:** + - Templates should be simple and contain logic that is easy to read and maintain. + - Avoid implementing complex functionality that increases maintenance cost and reduces code readability. +- **Writing unoptimized Jinja expressions:** + - Use Jinja filters for data manipulation, rather than implementing the logic using Ansible modules + +### 2.4 State Management + +- **Use Ansible to manage the desired state of your infrastructure.** Avoid manual changes outside of Ansible. +- **Leverage facts to gather system information and dynamically configure tasks.** +- **Use handlers to react to state changes and ensure services are restarted or reconfigured when necessary.** +- **Use variables to store configuration data and customize the desired state for different environments.** + +### 2.5 Error Handling Patterns + +- **Use `block` and `rescue` to handle errors gracefully.** +- **Use the `fail` module to explicitly fail a playbook when a critical error occurs.** +- **Register variables to capture the output of tasks and use the output in error handling logic.** +- **Use the `ignore_errors` directive to continue playbook execution even when a task fails (use with caution).** +- **Use the `meta: clear_host_errors` task to clear errors on a host and continue playbook execution.** +- **Implement rollback mechanisms to revert changes in case of failures.** + +## 3. Performance Considerations + +- Optimize your Ansible code to improve execution speed and resource utilization. + +### 3.1 Optimization Techniques + +- **Use SSH multiplexing:** + - Enable SSH multiplexing in your `ansible.cfg` file to reduce the overhead of establishing SSH connections. + - `pipelining = True` +- **Use ControlPersist:** + - Enable ControlPersist in your SSH configuration to keep SSH connections alive for longer periods of time. +- **Optimize Inventory:** + - Use a dynamic inventory to manage large numbers of hosts. + - Cache the inventory data to reduce the time it takes to load the inventory. +- **Minimize Fact Gathering:** + - Disable fact gathering for tasks that don't require them. + - Use the `gather_facts: false` directive in your playbooks or tasks. +- **Parallel Execution:** + - Increase the number of forks to execute tasks in parallel. + - Use the `-f` or `--forks` option when running Ansible playbooks. + - Be mindful of resource limitations and avoid overloading your control node or target hosts. +- **Avoid Loops When Possible:** + - Prefer modules that can handle lists of items instead of using loops. + - For example, use the `yum` or `apt` module with a list of packages to install instead of using a loop with the `package` module. +- **Optimize Jinja2 Templates:** + - Use Jinja2 filters to perform data manipulation instead of using Ansible modules. + - Cache the results of Jinja2 filters to avoid recomputing them repeatedly. +- **Use Async Tasks:** + - Use async tasks with poll to execute tasks in the background and reduce the time it takes to run your playbooks. +- **Use Connection Type `smart` or `persistent`:** + - These connection types are optimized for performance, especially for larger deployments. +- **Use Caching:** + - Utilize Ansible's fact caching to avoid gathering the same facts repeatedly. +- **Use strategy plugins:** + - Use plugins such as free, debug and host_pinned in your Ansible configuration file to maximize execution performance and concurrency. + +### 3.2 Memory Management + +- **Limit the number of forks:** + - Reduce the number of forks if your control node is running out of memory. +- **Use smaller inventories:** + - Split large inventories into smaller inventories to reduce memory usage. +- **Avoid loading large files into variables:** + - Use the `slurp` module to read large files into variables only when necessary. +- **Compress large files before transferring them:** + - Use the `compress` module to compress large files before transferring them to target hosts. + +### 3.3 Rendering Optimization + +- **Cache Jinja2 templates:** + - Enable Jinja2 template caching to reduce the time it takes to render templates. +- **Use filters and tests:** + - Instead of modules, use filters and tests to avoid calling Ansible tasks. +- **Limit Variable Scope:** + - Declare variables in the narrowest scope possible to minimize memory usage and improve performance. + +## 4. Security Best Practices + +- Secure your Ansible environment and protect sensitive data. + +### 4.1 Common Vulnerabilities and Prevention + +- **Plaintext Secrets:** + - *Vulnerability:* Storing passwords, API keys, and other sensitive data in plaintext in playbooks or variable files. + - *Prevention:* Use Ansible Vault to encrypt sensitive data. + - Example: + + # Encrypt a variable file + ansible-vault encrypt group_vars/all/vault.yml + + # Use the variable in a playbook + - name: Deploy application with secret key + template: + src: app.conf.j2 + dest: /etc/app/app.conf + +- **Unprotected SSH Keys:** + - *Vulnerability:* Leaving private SSH keys unprotected on the control node or target hosts. + - *Prevention:* Restrict access to the control node and use SSH key-based authentication with proper permissions. +- **Command Injection:** + - *Vulnerability:* Using user-supplied data directly in shell commands without proper sanitization. + - *Prevention:* Avoid using the `shell` and `command` modules unless absolutely necessary. + - Use Ansible's built-in modules whenever possible. + - If you must use `shell` or `command`, sanitize user input properly. +- **Privilege Escalation:** + - *Vulnerability:* Granting unnecessary privileges to Ansible users or tasks. + - *Prevention:* Use the `become` directive only when necessary. + - Grant the minimum required privileges to Ansible users. +- **Unvalidated Input:** + - *Vulnerability:* Using unvalidated input from external sources (e.g., APIs, user input) without proper sanitization. + - *Prevention:* Validate all input from external sources before using it in your playbooks or tasks. + - Use the `assert` module to validate input. +- **Insecure Communication:** + - *Vulnerability:* Using insecure communication protocols (e.g., HTTP) to transfer sensitive data. + - *Prevention:* Use HTTPS for all communication with APIs and other external services. +- **Weak SSH Keys:** + - *Vulnerability:* Using weak or compromised SSH keys. + - *Prevention:* Generate strong SSH keys with a key length of at least 2048 bits. + - Rotate SSH keys regularly. +- **Unauthorized Access:** + - *Vulnerability:* Allowing unauthorized access to the control node or target hosts. + - *Prevention:* Use strong passwords and multi-factor authentication. + - Restrict access to the control node and target hosts to authorized users only. + +### 4.2 Input Validation + +- **Validate all input from external sources before using it in your playbooks or tasks.** +- **Use the `assert` module to validate input.** +- **Use regular expressions to validate input strings.** +- **Use the `int` and `float` filters to validate numeric input.** +- **Use the `bool` filter to validate boolean input.** +- **Use the `list` and `dict` filters to validate list and dictionary input.** + +### 4.3 Authentication and Authorization + +- **Use SSH key-based authentication instead of passwords.** +- **Restrict access to the control node and target hosts to authorized users only.** +- **Use the `become` directive to escalate privileges only when necessary.** +- **Use the `sudo` module to execute tasks with elevated privileges.** +- **Use the `acl` module to manage access control lists (ACLs) on files and directories.** +- **Integrate with external authentication providers (e.g., LDAP, Active Directory) for centralized user management.** + +### 4.4 Data Protection + +- **Use Ansible Vault to encrypt sensitive data.** +- **Use HTTPS for all communication with APIs and other external services.** +- **Use the `slurp` module to read sensitive data from files into variables only when necessary.** +- **Use the `compress` module to compress sensitive data before transferring it to target hosts.** +- **Use the `hash` filter to hash sensitive data before storing it in files or databases.** +- **Implement data masking and tokenization techniques to protect sensitive data at rest and in transit.** + +### 4.5 Secure API Communication + +- **Use HTTPS for all communication with APIs.** +- **Use strong authentication mechanisms (e.g., API keys, OAuth 2.0) to protect your APIs.** +- **Validate all input from APIs before using it in your playbooks or tasks.** +- **Use the `uri` module to interact with APIs securely.** +- **Implement rate limiting and throttling to protect your APIs from abuse.** +- **Monitor your APIs for security vulnerabilities and performance issues.** + +## 5. Testing Approaches + +- Implement a comprehensive testing strategy to ensure your Ansible code is reliable and error-free. + +### 5.1 Unit Testing + +- **Unit test custom modules, filter plugins, and lookup plugins.** +- **Use a testing framework such as `pytest` to write and run unit tests.** +- **Mock external dependencies to isolate the code being tested.** +- **Write tests for all possible scenarios, including edge cases and error conditions.** +- **Use continuous integration to run unit tests automatically on every commit.** +- **Test modules for different possible input arguments and check their results.** + +### 5.2 Integration Testing + +- **Integration test your playbooks and roles to ensure they work together correctly.** +- **Use a testing framework such as `Molecule` to automate integration testing.** +- **Create a test environment that closely resembles your production environment.** +- **Test all critical functionality, including package installation, configuration management, and service management.** +- **Use continuous integration to run integration tests automatically on every commit.** +- **Use an ephemeral infrastructure using Vagrant or Docker to test the playbooks on a fresh machine.** +- **Test for idempotency by running the same playbook multiple times.** + +### 5.3 End-to-End Testing + +- **End-to-end test your entire infrastructure to ensure all components work together correctly.** +- **Use a testing framework such as `Testinfra` or `Inspec` to write and run end-to-end tests.** +- **Create a test environment that is as close as possible to your production environment.** +- **Test all critical business processes to ensure they work as expected.** +- **Use continuous integration to run end-to-end tests automatically on every commit.** + +### 5.4 Test Organization + +- **Organize your tests into a directory structure that mirrors your Ansible code.** +- **Create separate directories for unit tests, integration tests, and end-to-end tests.** +- **Use descriptive names for your test files and test functions.** +- **Document your tests with clear explanations of what they are testing.** +- **Use a consistent naming convention for your tests to improve readability and maintainability.** + +### 5.5 Mocking and Stubbing Techniques + +- **Use mocking and stubbing to isolate your code during unit testing.** +- **Mock external dependencies such as APIs, databases, and operating system commands.** +- **Use a mocking framework such as `unittest.mock` (Python) to create mocks and stubs.** +- **Write tests that verify that your code interacts with the mocked dependencies correctly.** +- **Avoid mocking Ansible modules, instead create a module that is a wrapper around it.** + +## 6. Common Pitfalls and Gotchas + +- Be aware of common mistakes and edge cases when working with Ansible. + +### 6.1 Frequent Mistakes + +- **Incorrect YAML Syntax:** + - YAML is whitespace-sensitive. Incorrect indentation or missing colons can cause syntax errors. + - Use a YAML linter to validate your syntax. +- **Variable Scope Issues:** + - Variables can be defined at different levels (e.g., playbook, role, inventory). Understanding variable precedence is crucial. + - Use the `vars_prompt` directive to prompt users for variables. +- **Incorrect Module Usage:** + - Using the wrong module for a task can lead to unexpected results or errors. + - Refer to the Ansible documentation for module usage examples and best practices. +- **Lack of Idempotency:** + - Tasks that are not idempotent can cause unintended side effects when run repeatedly. + - Use the `changed_when` and `failed_when` directives to customize task result conditions. +- **Ignoring Errors:** + - Ignoring errors can lead to cascading failures or inconsistent state. + - Use the `block` and `rescue` directives to handle errors gracefully. +- **Misunderstanding Facts:** + - Facts are variables that contain information about the target hosts. Misunderstanding or misusing facts can lead to incorrect behavior. + - Use the `ansible_facts` variable to access all available facts. +- **Over-reliance on the `shell` module:** + - Avoid the shell module unless absolutely necessary as using it hinders the playbook's idempotency. +- **Not testing playbooks:** + - Always test your playbooks before running them in production to avoid any unexpected issues. + +### 6.2 Edge Cases + +- **Handling Large Files:** + - Transferring large files can be slow and consume a lot of resources. Use the `synchronize` module or the `archive` and `unarchive` modules to optimize file transfers. +- **Managing Complex Dependencies:** + - Managing complex dependencies between tasks and roles can be challenging. Use the `block` and `rescue` directives to handle errors gracefully. +- **Working with Dynamic Inventories:** + - Dynamic inventories can be complex to set up and maintain. Use a dynamic inventory plugin or script to manage your inventory automatically. +- **Handling Network Latency:** + - Network latency can affect the performance of your playbooks. Use the `async` directive to execute tasks asynchronously. +- **Special characters in variables:** + - Avoid using special characters that can interfere with playbook's execution. + +### 6.3 Version-Specific Issues + +- **Module Compatibility:** + - Some modules may not be compatible with all versions of Ansible. + - Check the module documentation for compatibility information. +- **Deprecated Features:** + - Some features may be deprecated in newer versions of Ansible. + - Be aware of deprecated features and migrate to the recommended alternatives. +- **Behavioral Changes:** + - Some modules may have different behavior in different versions of Ansible. + - Test your playbooks with different versions of Ansible to ensure they behave as expected. + +### 6.4 Compatibility Concerns + +- **Operating System Compatibility:** + - Some playbooks and roles may only be compatible with specific operating systems. + - Check the playbook or role documentation for compatibility information. +- **Python Version Compatibility:** + - Ansible requires Python to be installed on the target hosts. Ensure that the target hosts have a compatible version of Python installed. +- **Module Dependencies:** + - Some modules may have external dependencies that need to be installed on the target hosts. + - Check the module documentation for dependency information. + +### 6.5 Debugging Strategies + +- **Use the `-v` or `--verbose` option to increase the verbosity of Ansible output.** +- **Use the `-C` or `--check` option to perform a dry run of your playbook.** +- **Use the `--diff` option to see the changes that will be made to the target hosts.** +- **Use the `debug` module to print variables and facts.** +- **Use the `assert` module to validate variables and facts.** +- **Use the `pause` module to pause playbook execution and inspect the target hosts.** +- **Use the `register` directive to capture the output of tasks and use the output in debugging logic.** +- **Check the Ansible logs on the control node and the target hosts for error messages.** + +## 7. Tooling and Environment + +- Utilize recommended tools and environment configurations to streamline your Ansible development and deployment processes. + +### 7.1 Recommended Development Tools + +- **Text Editor or IDE:** + - Use a text editor or IDE with YAML and Jinja2 syntax highlighting and auto-completion. + - Popular options include VS Code, Sublime Text, Atom, and PyCharm. + - Install Ansible-specific plugins for improved code completion and linting. +- **Ansible Lint:** + - Use `ansible-lint` to check your playbooks and roles for common errors and best practices violations. + - Configure `ansible-lint` to use a custom style guide or to enforce specific rules. +- **Molecule:** + - Use `Molecule` to test your roles in a Docker or Vagrant environment. + - Molecule provides a framework for writing and running integration tests for your roles. +- **Vagrant:** + - Use `Vagrant` to create virtual machines for testing your playbooks and roles. + - Vagrant allows you to easily create and destroy test environments. +- **Docker:** + - Use `Docker` to containerize your Ansible control node and target hosts. + - Docker provides a lightweight and portable environment for running your Ansible code. +- **Testinfra:** + - Use `Testinfra` to write and run end-to-end tests for your infrastructure. + - Testinfra allows you to verify the state of your target hosts after running your playbooks. + +### 7.2 Build Configuration + +- **Use a build tool such as `Make` or `Grunt` to automate your build process.** +- **Create a `Makefile` or `Gruntfile.js` that defines the tasks for linting, testing, and deploying your Ansible code.** +- **Use environment variables to configure your build process.** +- **Use a version control system such as `Git` to track changes to your build configuration.** +- **Keep your build configuration simple and easy to understand.** + +### 7.3 Linting and Formatting + +- **Use `ansible-lint` to check your playbooks and roles for common errors and best practices violations.** +- **Configure `ansible-lint` to use a custom style guide or to enforce specific rules.** +- **Use a YAML linter to validate your YAML syntax.** +- **Use a Jinja2 linter to validate your Jinja2 syntax.** +- **Automate linting and formatting as part of your build process.** +- **Use auto-formatting tools such as `Prettier` to automatically format your YAML and Jinja2 code.** +- **Follow a consistent naming convention for your variables, tasks, roles, and files.** + +### 7.4 Deployment Best Practices + +- **Use a deployment tool such as `Ansible Tower` or `AWX` to manage your deployments.** +- **Use a continuous integration and continuous delivery (CI/CD) pipeline to automate your deployments.** +- **Use a version control system such as `Git` to track changes to your playbooks and roles.** +- **Use a rollback mechanism to revert changes in case of failures.** +- **Use a monitoring tool such as `Nagios` or `Zabbix` to monitor your infrastructure.** +- **Use a logging tool such as `ELK Stack` or `Splunk` to collect and analyze your logs.** +- **Always test your playbooks in a staging environment before deploying them to production.** + +### 7.5 CI/CD Integration + +- **Integrate your Ansible code into your CI/CD pipeline to automate testing and deployment.** +- **Use a CI/CD tool such as `Jenkins`, `GitLab CI`, or `Travis CI` to automate your pipeline.** +- **Create a CI/CD pipeline that includes linting, testing, and deployment steps.** +- **Use environment variables to configure your CI/CD pipeline.** +- **Use a version control system such as `Git` to trigger your CI/CD pipeline on every commit.** +- **Use automated testing to verify that your infrastructure changes work as expected.** +- **Use automated deployment to deploy your infrastructure changes to your target environment.** +- **Automate configuration management:** + - Ensure consistent configurations for servers and applications. +- **Implement infrastructure as code:** + - Track and manage infrastructure in source control. +- **Automate application deployments:** + - Deploy new versions of software reliably and efficiently. +- **Orchestrate multi-tier deployments:** + - Coordinate deployments across various layers of infrastructure. \ No newline at end of file diff --git a/.cursor/rules/ant-design.mdc b/.cursor/rules/ant-design.mdc new file mode 100644 index 000000000..f949dae0c --- /dev/null +++ b/.cursor/rules/ant-design.mdc @@ -0,0 +1,311 @@ +--- +description: This rule enforces best practices and coding standards for projects using the Ant Design (antd) UI library within React applications. It covers code organization, performance, security, testing, and common pitfalls to ensure maintainable and efficient applications. +globs: *.js,*.jsx,*.ts,*.tsx +--- +# Ant Design (antd) Best Practices and Coding Standards + +This document outlines the recommended best practices for developing React applications using the Ant Design (antd) UI library. Following these guidelines will lead to more maintainable, performant, and secure applications. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure + +- **`src/components/`**: Contains reusable React components, including those utilizing antd components. +- **`src/pages/`**: Contains components representing different application routes or pages. +- **`src/layouts/`**: Contains layout components that provide a consistent structure across pages. +- **`src/services/`**: Contains modules for interacting with APIs and handling data fetching. +- **`src/utils/`**: Contains utility functions and helper modules. +- **`src/styles/`**: Contains global styles, theme customizations, and component-specific stylesheets. +- **`src/assets/`**: Contains static assets such as images, fonts, and icons. +- **`src/context/`**: (Optional) If using context API, store all context definition files here. + +Example: + + +my-app/ +├── src/ +│ ├── components/ +│ │ ├── Button.jsx +│ │ ├── Input.jsx +│ │ └── ... +│ ├── pages/ +│ │ ├── HomePage.jsx +│ │ ├── LoginPage.jsx +│ │ └── ... +│ ├── layouts/ +│ │ ├── MainLayout.jsx +│ │ └── ... +│ ├── services/ +│ │ ├── api.js +│ │ └── ... +│ ├── utils/ +│ │ ├── date-formatter.js +│ │ └── ... +│ ├── styles/ +│ │ ├── global.css +│ │ ├── theme.js +│ │ └── ... +│ ├── App.jsx +│ └── index.js +└── ... + + +### 1.2. File Naming Conventions + +- **Components**: Use PascalCase for component file names (e.g., `MyComponent.jsx`, `UserProfile.tsx`). +- **Styles**: Use kebab-case for style file names (e.g., `my-component.css`, `user-profile.module.scss`). +- **Modules**: Use camelCase for module file names (e.g., `api.js`, `dateFormatter.ts`). + +### 1.3. Module Organization + +- Group related components, styles, and assets within the same directory. +- Create separate modules for API interactions, data transformations, and utility functions. + +### 1.4. Component Architecture + +- **Presentational Components**: Focus on rendering UI elements and receiving data via props. +- **Container Components**: Handle data fetching, state management, and logic, passing data to presentational components. +- Use functional components with hooks whenever possible for simplicity and reusability. + +### 1.5. Code Splitting + +- Utilize React.lazy and Suspense to load components on demand, improving initial load time. +- Split routes into separate chunks to minimize the initial bundle size. +- Consider using dynamic imports for less frequently used components or modules. + +Example: + +jsx +import React, { Suspense } from 'react'; + +const MyComponent = React.lazy(() => import('./MyComponent')); + +function MyPage() { + return ( + Loading...}> + + + ); +} + + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +- **Higher-Order Components (HOCs)**: Use for cross-cutting concerns like authentication or data fetching. Prefer hooks where possible. +- **Render Props**: An alternative to HOCs for sharing code between components. Prefer hooks where possible. +- **Compound Components**: Create reusable components with implicit state sharing (e.g., `Tabs` and `Tab` components). + +### 2.2. Recommended Approaches + +- **Form Handling**: Use `antd`'s `Form` component for managing form state, validation, and submission. +- **Data Display**: Leverage `antd`'s `Table`, `List`, and `Card` components for structured data presentation. +- **Navigation**: Use `antd`'s `Menu` and `Breadcrumb` components for creating intuitive navigation. + +### 2.3. Anti-patterns and Code Smells + +- **Direct DOM Manipulation**: Avoid directly manipulating the DOM; let React manage updates. +- **Over-reliance on `any` type**: Using `any` in TypeScript defeats the purpose of static typing. Provide explicit types. +- **Mutating Props**: Treat props as read-only and avoid modifying them directly. +- **Inline Styles**: Keep styles in CSS files or use styled-components for better organization and maintainability. Prefer CSS Modules or Styled Components for component specific styles. + +### 2.4. State Management + +- **Component State**: Use `useState` for simple, local component state. +- **Context API**: Use for sharing state across multiple components without prop drilling. +- **Redux/Zustand**: Consider for complex applications with global state and predictable state transitions. +- **MobX**: Consider for complex applications where you want to observe changes in your data and derive calculations from that data. + +### 2.5. Error Handling + +- **Try-Catch Blocks**: Use for handling synchronous errors. +- **Error Boundaries**: Use to catch errors during rendering and prevent the entire application from crashing. +- **Global Error Handling**: Implement a global error handler to log errors and provide user feedback. + +Example (Error Boundary): + +jsx +class ErrorBoundary extends React.Component { + constructor(props) { + super(props); + this.state = { hasError: false }; + } + + static getDerivedStateFromError(error) { + return { hasError: true }; + } + + componentDidCatch(error, errorInfo) { + console.error('Caught error: ', error, errorInfo); + } + + render() { + if (this.state.hasError) { + return

Something went wrong.

; + } + + return this.props.children; + } +} + +export default ErrorBoundary; + +// Usage: + + + + + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +- **Memoization**: Use `React.memo` to prevent unnecessary re-renders of components with the same props. +- **Pure Components**: Extend `React.PureComponent` for class components to perform shallow prop comparisons. +- **Virtualization**: Use `antd`'s `Table` and `List` components with virtualization for large datasets. +- **Debouncing/Throttling**: Use for event handlers that trigger frequent updates (e.g., search input). + +### 3.2. Memory Management + +- **Avoid Memory Leaks**: Properly clean up event listeners and timers in `useEffect` hooks. +- **Release Resources**: Release unused objects and data structures to free up memory. + +### 3.3. Rendering Optimization + +- **ShouldComponentUpdate**: Implement `shouldComponentUpdate` (for class components) or use `React.memo` (for functional components) to prevent unnecessary re-renders. +- **Immutable Data**: Use immutable data structures to simplify change detection. + +### 3.4. Bundle Size Optimization + +- **Modular Imports**: Import only the necessary components from `antd` to reduce bundle size (e.g., `import { Button } from 'antd';`). Use `babel-plugin-import` for automatic modular imports. +- **Tree Shaking**: Ensure your build process supports tree shaking to remove unused code. +- **Code Splitting**: As mentioned earlier, split your code into smaller chunks to reduce the initial bundle size. + +### 3.5. Lazy Loading + +- Use `React.lazy` and `Suspense` to load components on demand. +- Implement lazy loading for images and other assets using libraries like `react-lazyload`. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities + +- **Cross-Site Scripting (XSS)**: Prevent XSS by sanitizing user input and encoding output. +- **Cross-Site Request Forgery (CSRF)**: Protect against CSRF attacks by implementing CSRF tokens. +- **SQL Injection**: Avoid directly embedding user input in SQL queries; use parameterized queries or ORMs. + +### 4.2. Input Validation + +- **Server-Side Validation**: Always validate user input on the server-side. +- **Client-Side Validation**: Use `antd`'s `Form` component for client-side validation to provide immediate feedback to the user. +- **Sanitize Input**: Sanitize user input to remove potentially harmful characters or code. + +### 4.3. Authentication and Authorization + +- **Secure Authentication**: Use secure authentication mechanisms like JWT (JSON Web Tokens) or OAuth. +- **Role-Based Access Control (RBAC)**: Implement RBAC to control access to different parts of the application based on user roles. + +### 4.4. Data Protection + +- **Encryption**: Encrypt sensitive data both in transit and at rest. +- **Data Masking**: Mask sensitive data in the UI to prevent unauthorized access. + +### 4.5. Secure API Communication + +- **HTTPS**: Use HTTPS to encrypt communication between the client and the server. +- **API Rate Limiting**: Implement rate limiting to prevent abuse and denial-of-service attacks. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +- Test individual components in isolation to ensure they function correctly. +- Use testing libraries like Jest and React Testing Library. +- Mock dependencies to isolate the component being tested. + +### 5.2. Integration Testing + +- Test the interaction between multiple components or modules. +- Use testing libraries like React Testing Library and Cypress. + +### 5.3. End-to-End Testing + +- Test the entire application from the user's perspective. +- Use testing frameworks like Cypress or Playwright. + +### 5.4. Test Organization + +- Create a `tests/` directory at the root of your project. +- Place test files alongside the components or modules they test (e.g., `MyComponent.test.jsx`). +- Use descriptive test names to clearly indicate what is being tested. + +### 5.5. Mocking and Stubbing + +- Use mocking libraries like Jest's `jest.mock()` to mock external dependencies. +- Use stubbing to replace functions or methods with predefined behavior. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +- **Not using modular imports**: Importing the entire `antd` library can significantly increase bundle size. +- **Ignoring TypeScript errors**: Failing to address TypeScript errors can lead to runtime issues. +- **Not handling asynchronous operations correctly**: Failing to handle promises or async/await can lead to unhandled rejections and unexpected behavior. +- **Not localizing strings correctly**: Hardcoding strings instead of using `antd` i18n features. + +### 6.2. Edge Cases + +- **Handling different screen sizes and devices**: Ensuring responsive design using `antd` grid system. +- **Accessibility**: Consider accessibility when using components, making sure to include `aria` attributes. +- **Browser compatibility**: Test the app on various browsers (Chrome, Firefox, Safari, Edge, etc). + +### 6.3. Version-Specific Issues + +- **Breaking changes**: Be aware of breaking changes when upgrading `antd` versions. +- **Deprecated APIs**: Avoid using deprecated APIs and migrate to the recommended alternatives. +- **CSS class conflicts:** Potential issues with CSS specificity or conflicts with global styles. Use CSS Modules or Styled Components for more robust style isolation. + +### 6.4. Compatibility Concerns + +- **React version**: Ensure compatibility between `antd` and your React version. +- **Other UI libraries**: Avoid conflicts with other UI libraries by using consistent styling and naming conventions. + +### 6.5. Debugging + +- **Use browser developer tools**: Inspect the DOM, network requests, and console output. +- **Use React DevTools**: Inspect the component tree, props, and state. +- **Use logging and debugging statements**: Add `console.log` statements to trace the execution flow and inspect variable values. + +## 7. Tooling and Environment + +### 7.1. Recommended Tools + +- **IDE**: VS Code, WebStorm. +- **Build Tool**: Webpack, Parcel, Rollup, esbuild. +- **Testing Libraries**: Jest, React Testing Library, Cypress. +- **Linting**: ESLint, Prettier. + +### 7.2. Build Configuration + +- **Optimize for production**: Use production-specific build configurations to minimize bundle size and improve performance. +- **Configure code splitting**: Set up code splitting to load components on demand. +- **Enable tree shaking**: Ensure your build process supports tree shaking to remove unused code. + +### 7.3. Linting and Formatting + +- **ESLint**: Use ESLint with recommended React and `antd` plugins to enforce coding standards and detect potential errors. +- **Prettier**: Use Prettier to automatically format your code for consistency. +- **Stylelint:** Use Stylelint to enforce consistent style practices. + +### 7.4. Deployment + +- **Choose a hosting platform**: Netlify, Vercel, AWS, Google Cloud, Azure. +- **Configure environment variables**: Set up environment variables for API keys, database credentials, and other sensitive information. +- **Use a CDN**: Use a Content Delivery Network (CDN) to cache static assets and improve loading times. + +### 7.5. CI/CD Integration + +- **Set up a CI/CD pipeline**: Use tools like Jenkins, Travis CI, CircleCI, or GitHub Actions to automate testing, building, and deployment. +- **Automate testing**: Run unit, integration, and end-to-end tests in your CI/CD pipeline. +- **Automate deployment**: Automate the deployment process to reduce manual effort and errors. \ No newline at end of file diff --git a/.cursor/rules/anyio.mdc b/.cursor/rules/anyio.mdc new file mode 100644 index 000000000..269810b7e --- /dev/null +++ b/.cursor/rules/anyio.mdc @@ -0,0 +1,169 @@ +--- +description: This rule provides best practices and coding standards for developing with the AnyIO library, focusing on structured concurrency, task management, and proper use of synchronization primitives to create robust, cross-backend asynchronous applications. +globs: *.py +--- +# AnyIO Best Practices and Coding Standards + +This document provides comprehensive guidelines for developing with the AnyIO library, covering code organization, common patterns, performance considerations, security best practices, testing approaches, common pitfalls, and recommended tooling. + +## Library Information: +- Name: anyio +- Tags: python, async, compatibility-layer + +## 1. Code Organization and Structure + +- **Directory Structure:** Organize your project to separate concerns clearly. + - `src/`: Contains the main application code. + - `tests/`: Contains unit, integration, and end-to-end tests. + - `examples/`: Contains example code showcasing AnyIO features. + - `docs/`: Contains project documentation (using Sphinx, for example). + - `tasks.py`: Invoke tasks for building, testing, and deploying (using Invoke). +- **File Naming Conventions:** + - Use descriptive and consistent file names. + - `my_component.py` for a component's source code. + - `my_component_test.py` for its tests. + - `conftest.py` in the `tests/` directory for test fixtures. +- **Module Organization:** + - Group related functionality into modules. + - Use packages to further structure your code. + - Employ clear and concise module-level docstrings. +- **Component Architecture:** + - Design components with clear responsibilities. + - Use interfaces or abstract base classes for decoupling. + - Consider using dependency injection to improve testability. +- **Code Splitting:** + - Break down large functions into smaller, more manageable ones. + - Decompose complex modules into simpler sub-modules. + - Consider lazy-loading modules or components to improve startup time. + +## 2. Common Patterns and Anti-patterns + +- **Design Patterns:** + - **Task Groups (Nurseries):** Use task groups (`anyio.create_task_group()`) for structured concurrency. This is the most crucial pattern in AnyIO. + - **Resource Acquisition Is Initialization (RAII):** Use `async with` statements for managing resources (e.g., sockets, files) to ensure proper cleanup, even in case of exceptions or cancellations. + - **Observer Pattern:** Use AnyIO's `Event` or `Condition` primitives to implement observer patterns for asynchronous communication. + - **Producer-Consumer Pattern:** Use `anyio.Queue` for implementing producer-consumer patterns with asynchronous tasks. +- **Recommended Approaches:** + - Use `anyio.sleep()` for non-blocking delays. + - Employ `anyio.to_thread.run_sync()` to run blocking code in a separate thread to avoid blocking the event loop. + - Utilize `anyio.Path` for asynchronous file I/O operations. +- **Anti-patterns:** + - **Blocking the Event Loop:** Avoid performing long-running synchronous operations in the main event loop. Use `anyio.to_thread.run_sync()` to offload such tasks to worker threads. + - **Ignoring Exceptions:** Always handle exceptions gracefully within tasks and task groups. Unhandled exceptions can crash your application. + - **Unstructured Concurrency:** Avoid creating tasks without a task group. This can lead to resource leaks and difficult-to-debug issues. + - **Spin Locks:** Do not use busy-waiting or spin locks in asynchronous code. Use synchronization primitives like `Lock` or `Condition` instead. +- **State Management:** + - Use immutable data structures whenever possible to avoid race conditions. + - Employ `Lock` or `Condition` objects to protect shared mutable state. + - Consider using atomic operations for simple state updates. +- **Error Handling:** + - Wrap code in `try...except` blocks to catch exceptions. + - Use `try...finally` blocks to ensure cleanup, even in case of exceptions. + - Leverage task groups to handle exceptions across multiple tasks. + - Implement retry mechanisms for transient errors. + +## 3. Performance Considerations + +- **Optimization Techniques:** + - **Asynchronous I/O:** Utilize AnyIO's asynchronous I/O functions (e.g., `anyio.Path`, `anyio.open_file`) for optimal performance when dealing with I/O-bound operations. + - **Connection Pooling:** Implement connection pooling for database connections or other network resources to reduce overhead. + - **Caching:** Cache frequently accessed data to minimize I/O operations. + - **Minimize Context Switching:** Reduce unnecessary context switching by batching operations and avoiding excessive `await` calls. +- **Memory Management:** + - Avoid creating large, unnecessary data structures. + - Use generators or iterators to process large datasets in a memory-efficient manner. + - Properly close resources (e.g., sockets, files) to release memory. + - Use the `del` statement or `weakref` to break circular references and allow garbage collection. +- **Rendering Optimization:** N/A (AnyIO is not directly involved in rendering). +- **Bundle Size Optimization:** N/A (AnyIO itself is a dependency, not a frontend bundle). +- **Lazy Loading:** + - Use lazy loading to defer the initialization of components until they are needed. + - Implement code splitting to load modules on demand. + +## 4. Security Best Practices + +- **Common Vulnerabilities:** + - **Denial of Service (DoS):** Protect against DoS attacks by limiting the number of concurrent connections or requests. + - **Injection Attacks:** Sanitize user inputs to prevent injection attacks (e.g., SQL injection, command injection). + - **Cross-Site Scripting (XSS):** N/A (AnyIO is not directly involved in front-end development). +- **Input Validation:** + - Validate all user inputs to ensure they conform to expected formats and ranges. + - Use input validation libraries to simplify the process. + - Escape or sanitize user inputs before using them in database queries or system commands. +- **Authentication and Authorization:** + - Implement authentication to verify the identity of users. + - Implement authorization to control access to resources based on user roles. + - Use secure password hashing algorithms (e.g., bcrypt, Argon2). +- **Data Protection:** + - Use encryption to protect sensitive data at rest and in transit. + - Store cryptographic keys securely. + - Comply with relevant data privacy regulations (e.g., GDPR, CCPA). +- **Secure API Communication:** + - Use HTTPS for all API communication to encrypt data in transit. + - Implement proper authentication and authorization mechanisms for API endpoints. + - Protect against Cross-Site Request Forgery (CSRF) attacks if your API is used by web applications. + +## 5. Testing Approaches + +- **Unit Testing:** + - Write unit tests for individual components to verify their functionality. + - Use mocking or stubbing to isolate components from their dependencies. + - Aim for high test coverage to ensure that all code paths are tested. +- **Integration Testing:** + - Write integration tests to verify the interaction between different components. + - Test the integration with external services (e.g., databases, APIs). +- **End-to-End Testing:** + - Write end-to-end tests to verify the entire application flow. + - Simulate real user interactions to test the application from a user's perspective. +- **Test Organization:** + - Organize tests into separate modules based on the components or features they test. + - Use a consistent naming convention for test files and functions. + - Employ test fixtures to set up and tear down test environments. +- **Mocking and Stubbing:** + - Use mocking libraries (e.g., `unittest.mock`, `pytest-mock`) to replace dependencies with mock objects. + - Stub external services or APIs to simulate different scenarios. + - Verify that mock objects are called with the expected arguments. + +## 6. Common Pitfalls and Gotchas + +- **Frequent Mistakes:** + - **Not using `async with` for resources:** Forgetting to use `async with` for resources that require asynchronous cleanup can lead to resource leaks. + - **Mixing asyncio and Trio idioms:** Avoid mixing idioms from different asynchronous frameworks. Stick to AnyIO's API. + - **Incorrectly using `to_thread.run_sync`:** Ensure that the code passed to `to_thread.run_sync` is actually blocking. +- **Edge Cases:** + - **Cancellation:** Be aware of how cancellation works in AnyIO and handle it gracefully. + - **Timeouts:** Implement timeouts to prevent tasks from running indefinitely. + - **Signal Handling:** Properly handle signals (e.g., `SIGINT`, `SIGTERM`) to ensure graceful shutdown. +- **Version-Specific Issues:** + - Consult the AnyIO changelog for known issues and breaking changes in different versions. +- **Compatibility Concerns:** + - Ensure that AnyIO is compatible with the versions of asyncio or Trio you are using. + - Be aware of potential compatibility issues with other libraries that use asynchronous programming. +- **Debugging Strategies:** + - Use logging to trace the execution flow of your code. + - Employ debuggers (e.g., `pdb`, `ipdb`) to step through your code and inspect variables. + - Use asynchronous debugging tools to debug asynchronous code. + +## 7. Tooling and Environment + +- **Recommended Development Tools:** + - **Python:** Use the latest stable version of Python (3.10+ recommended). + - **Poetry/Pip:** Use Poetry or Pip for dependency management. + - **Pytest:** Use Pytest for testing. + - **Black/Ruff:** Use Black and Ruff for code formatting and linting. + - **VS Code/PyCharm:** Use VS Code or PyCharm as your IDE. +- **Build Configuration:** + - Use a `pyproject.toml` file to configure Poetry or Pip. + - Specify dependencies and build settings in the `pyproject.toml` file. + - Use a `setup.py` file for projects that need to be installed with `pip install -e .` +- **Linting and Formatting:** + - Configure Black and Ruff to enforce consistent code formatting and style. + - Use a pre-commit hook to automatically format and lint code before committing. +- **Deployment:** + - Package your application into a Docker container for easy deployment. + - Use a process manager (e.g., systemd, Supervisor) to manage your application. + - Deploy your application to a cloud platform (e.g., AWS, Azure, GCP). +- **CI/CD Integration:** + - Use a CI/CD pipeline to automate the build, test, and deployment process. + - Integrate your CI/CD pipeline with your version control system (e.g., GitHub, GitLab, Bitbucket). + - Use automated testing to ensure that code changes do not introduce regressions. \ No newline at end of file diff --git a/.cursor/rules/apollo-client.mdc b/.cursor/rules/apollo-client.mdc new file mode 100644 index 000000000..3bf155f37 --- /dev/null +++ b/.cursor/rules/apollo-client.mdc @@ -0,0 +1,178 @@ +--- +description: This rule provides comprehensive best practices for using Apollo Client in your projects, covering code organization, performance, security, testing, and common pitfalls. It aims to guide developers in building robust and maintainable GraphQL-powered applications. +globs: *.js,*.jsx,*.ts,*.tsx,*.gql,*.graphql +--- +- **Code Organization and Structure:** + - **Directory Structure:** + - Organize GraphQL queries and mutations in a dedicated directory (e.g., `graphql` or `queries`). + - Group related queries and mutations into subdirectories based on features or modules. + - Consider a separate directory for GraphQL types/schemas if you're managing them locally. + - Example: + + src/ + graphql/ + user/ + getUser.gql + createUser.gql + product/ + getProduct.gql + updateProduct.gql + + - **File Naming Conventions:** + - Use descriptive names for GraphQL query and mutation files (e.g., `getUser.gql`, `updateProduct.gql`). + - Employ a consistent naming scheme (e.g., `[operationName].[operationType].gql`). + - **Module Organization:** + - Encapsulate Apollo Client logic (e.g., initialization, hooks) within reusable modules. + - Create custom hooks for common data fetching patterns (e.g., `useUser`, `useProduct`). + - **Component Architecture:** + - Favor a component-based architecture for UI development. + - Decouple data fetching logic from UI components using hooks or render props. + - Utilize higher-order components (HOCs) or render props for cross-cutting concerns (e.g., authentication, error handling). + - **Code Splitting:** + - Use dynamic imports to lazy-load components and queries, improving initial load time. + - Split large queries into smaller fragments to reduce bundle size. + +- **Common Patterns and Anti-patterns:** + - **Design Patterns:** + - **Repository Pattern:** Abstract data access logic behind a repository interface. + - **Facade Pattern:** Provide a simplified interface to a complex subsystem (e.g., Apollo Client). + - **Hook Composition:** Combine multiple custom hooks to create more complex data fetching logic. + - **Recommended Approaches:** + - Use the `useQuery` and `useMutation` hooks for data fetching and manipulation. + - Leverage Apollo Client's cache for improved performance and offline capabilities. + - Implement optimistic updates to provide a better user experience. + - Use GraphQL fragments to reuse common data structures across multiple queries and mutations. + - **Anti-patterns:** + - Avoid directly manipulating the Apollo Client cache outside of mutations. + - Don't fetch the entire schema unnecessarily; only fetch what you need. + - Avoid deeply nested components with complex data fetching logic. + - **State Management:** + - Use Apollo Client's cache as the primary source of truth for application state. + - Integrate with external state management libraries (e.g., Redux, Zustand) when necessary for complex state requirements. + - Consider using Apollo Link State for managing local client-side state. + - **Error Handling:** + - Implement global error handling with Apollo Link. + - Display user-friendly error messages to the user. + - Retry failed queries or mutations automatically. + +- **Performance Considerations:** + - **Optimization Techniques:** + - Use the `fetchPolicy` option in `useQuery` to control cache behavior. + - Implement pagination for large datasets. + - Use batching and caching at the server-side to reduce network requests. + - **Memory Management:** + - Unsubscribe from subscriptions when components unmount to prevent memory leaks. + - Avoid storing large amounts of data in the Apollo Client cache. + - **Rendering Optimization:** + - Use React's `memo` or `useMemo` to prevent unnecessary re-renders. + - Implement virtualization for rendering large lists. + - **Bundle Size Optimization:** + - Use code splitting to reduce the initial bundle size. + - Remove unused dependencies. + - Minify and compress your code. + - **Lazy Loading:** + - Lazy-load components and queries using dynamic imports. + +- **Security Best Practices:** + - **Common Vulnerabilities:** + - **GraphQL Injection:** Prevent injection attacks by validating user input and using parameterized queries. + - **Denial of Service (DoS):** Limit query complexity and depth to prevent DoS attacks. + - **Information Disclosure:** Avoid exposing sensitive data in error messages. + - **Input Validation:** + - Validate all user input on both the client and server sides. + - Use GraphQL's built-in validation capabilities. + - **Authentication and Authorization:** + - Implement proper authentication and authorization mechanisms. + - Use JWT (JSON Web Tokens) or other secure authentication protocols. + - Follow the principle of least privilege when granting access to data. + - **Data Protection:** + - Encrypt sensitive data at rest and in transit. + - Use HTTPS for all API communication. + - **Secure API Communication:** + - Use Apollo Link to add security headers to API requests. + - Implement CORS (Cross-Origin Resource Sharing) to prevent cross-site scripting (XSS) attacks. + +- **Testing Approaches:** + - **Unit Testing:** + - Test individual components and hooks in isolation. + - Mock Apollo Client's API to control data fetching behavior. + - **Integration Testing:** + - Test the integration of components with Apollo Client. + - Use a test GraphQL server to simulate real API interactions. + - **End-to-end Testing:** + - Test the entire application flow from the user interface to the backend. + - Use tools like Cypress or Puppeteer for end-to-end testing. + - **Test Organization:** + - Organize tests in a directory structure that mirrors the source code. + - Use descriptive names for test files and test cases. + - **Mocking and Stubbing:** + - Use mocking libraries (e.g., Jest, Sinon) to mock Apollo Client's API. + - Create stub GraphQL responses for testing different scenarios. + +- **Common Pitfalls and Gotchas:** + - **Frequent Mistakes:** + - Incorrect cache configuration leading to stale data. + - Over-fetching data in GraphQL queries. + - Ignoring error handling in `useQuery` and `useMutation`. + - **Edge Cases:** + - Handling network errors and offline scenarios. + - Dealing with large datasets and complex queries. + - Managing cache invalidation and updates. + - **Version-Specific Issues:** + - Be aware of breaking changes between Apollo Client versions. + - Consult the Apollo Client documentation for migration guides. + - **Compatibility Concerns:** + - Ensure compatibility with different browsers and devices. + - Test your application on different platforms. + - **Debugging Strategies:** + - Use Apollo Client DevTools to inspect queries, mutations, and cache state. + - Log API requests and responses for debugging purposes. + +- **Tooling and Environment:** + - **Recommended Tools:** + - Apollo Client DevTools for debugging and inspecting the cache. + - GraphQL Playground or GraphiQL for exploring and testing GraphQL APIs. + - Apollo VS Code extension for syntax highlighting and autocompletion. + - **Build Configuration:** + - Use a build tool like Webpack or Parcel to bundle your code. + - Configure Babel to transpile your code to compatible JavaScript versions. + - **Linting and Formatting:** + - Use ESLint and Prettier to enforce code style and best practices. + - Configure linting rules specific to Apollo Client and GraphQL. + - **Deployment:** + - Use a deployment platform like Netlify, Vercel, or AWS Amplify. + - Configure environment variables for API endpoints and other sensitive data. + - **CI/CD Integration:** + - Use a CI/CD pipeline to automate testing and deployment. + - Integrate linters, formatters, and test runners into the pipeline. + +- **Specific Best Practices based on Exa Search Results:** + - Ensure all queries for a specific type return an ID for normalization and caching (RedwoodJS Community). + - Store queries in separate `.gql` files and use tools like `graphql-import` for better organization (Reddit). + - Use Apollo Client's `watchQuery` method for real-time data updates (Medium). + - Follow GraphQL query best practices for efficient API design (Apollo GraphQL). + +- **Additional considerations** + - When creating queries and mutations, follow best practices to get the most out of both GraphQL and Apollo tooling. + +- **Example Directory Structure** + +src/ + components/ + UserCard.tsx + ProductDetails.tsx + graphql/ + queries/ + getUser.gql + getProduct.gql + mutations/ + createUser.gql + updateProduct.gql + hooks/ + useUser.ts + useProduct.ts + utils/ + apolloClient.ts // Initialize Apollo Client here + + +This is a comprehensive guide for using Apollo Client. Please adapt these guidelines to your specific project requirements and coding style. \ No newline at end of file diff --git a/.cursor/rules/apollo-graphql.mdc b/.cursor/rules/apollo-graphql.mdc new file mode 100644 index 000000000..56e4e02aa --- /dev/null +++ b/.cursor/rules/apollo-graphql.mdc @@ -0,0 +1,217 @@ +--- +description: This rule provides comprehensive best practices and coding standards for developing with Apollo GraphQL. It covers schema design, security, performance, testing, and deployment. +globs: *.js,*.jsx,*.ts,*.tsx,*.graphql +--- +- Always use UV when installing dependencies +- Always use python 3.12 +- Always use classes instead of function + +# Apollo GraphQL Best Practices + +This document provides a comprehensive guide to best practices and coding standards for developing GraphQL APIs and applications using the Apollo ecosystem. + +## Library Information: +- Name: apollo-graphql +- Tags: web, api, graphql, javascript + +## 1. Schema Design + +- **Clarity and Consistency:** + - Use descriptive naming conventions for types, fields, and arguments (e.g., `getUserById` instead of `getUser`). + - Maintain consistency across the schema in terms of naming and structure. + - Define a clear schema that reflects your business domain and data model. + +- **Interfaces and Unions:** + - Implement interfaces and unions for shared features and common data structures. + - Use interfaces to define contracts for types that implement shared behaviors. + - Use unions when a field can return different object types that don't share a common interface. + +- **Nullability:** + - Every field is nullable by default unless explicitly marked as non-null using `!`. Carefully consider nullability for each field. + - Use non-null types (`!`) only when you can guarantee that the field will always return a value. + - Handle potential errors gracefully and return `null` for nullable fields when appropriate. + +- **Demand-Driven Schema Design:** + - Design your schema to serve client use cases and product requirements. + - Intentionally design your schema to serve client use cases and product requirements. + +- **Avoid Autogenerated Schemas:** + - Avoid autogenerating schemas, especially the fields on the root operation types + +## 2. Security + +- **Disable Introspection in Production:** + - Disable introspection in production environments to prevent unauthorized access to your schema. + - Restrict access to staging environments where introspection is enabled. + +- **Input Validation:** + - Validate all user inputs to prevent injection attacks and data corruption. + - Use custom scalars or directives to enforce validation rules on input types. + - Sanitize and escape user inputs before using them in resolvers. + +- **Authentication and Authorization:** + - Implement robust authentication and authorization mechanisms to protect your API. + - Use industry-standard protocols like OAuth 2.0 or JWT for authentication. + - Implement fine-grained authorization checks at the field level. + - Enforcing authentication and authorization in the router protects your underlying APIs from malicious operations + +- **Rate Limiting and Depth Limiting:** + - Implement rate limiting to prevent abuse and denial-of-service attacks. + - Limit query depth to prevent complex queries from overwhelming your server. + +- **Whitelisting Queries:** + - Consider whitelisting queries to restrict the operations that can be executed against your API. + +- **Obfuscate Error Details:** + - Remove verbose error details from API responses in your production graph. + - Only selectively expose error details to clients in production. + +- **Data Validation and Sanitization:** + - As a schema design best practice, you should deliberately design your schema to serve client use cases and product requirements. + +## 3. Performance Optimization + +- **Batching and Caching:** + - Utilize batching techniques (e.g., DataLoader) to reduce the number of requests to backend data sources. + - Implement caching at different levels (e.g., server-side, client-side) to improve response times. + +- **Pagination:** + - Implement pagination strategies to manage large datasets effectively. + - Use cursor-based pagination for efficient retrieval of paginated data. + +- **N+1 Problem:** + - Use tools like DataLoader to address the N+1 query problem and ensure efficient data fetching. + +- **Server-Side Batching & Caching:** + - GraphQL is designed in a way that allows you to write clean code on the server, where every field on every type has a focused single-purpose function for resolving that value. + +- **Automatic Persisted Queries (APQ):** + - Consider implementing automatic persisted queries (APQ) to optimize network usage and improve security + +## 4. Code Organization and Structure + +- **Directory Structure:** + + src/ + schema/ + types/ + *.graphql + resolvers/ + *.js + dataSources/ + *.js + utils/ + *.js + index.js // Entry point + + +- **File Naming Conventions:** + - Use PascalCase for type definitions (e.g., `UserType.graphql`). + - Use camelCase for resolver functions (e.g., `getUserById.js`). + +- **Module Organization:** + - Organize your code into reusable modules based on functionality (e.g., user management, product catalog). + - Create separate modules for schema definitions, resolvers, and data sources. + +- **Component Architecture:** + - Follow a component-based architecture for building GraphQL applications. + - Create reusable components for common UI elements and data fetching logic. + +- **Code Splitting:** + - Use code splitting to reduce the initial bundle size and improve page load times. + - Consider splitting your application into smaller chunks based on routes or features. + +## 5. Common Patterns and Anti-patterns + +- **Design Patterns:** + - **Data Source Pattern:** Decouple data fetching logic from resolvers using data sources. + - **Schema Stitching:** Combine multiple GraphQL APIs into a single, unified schema. + +- **Recommended Approaches:** + - Use a GraphQL client library (e.g., Apollo Client, Relay) for efficient data fetching and caching. + - Implement custom directives to add additional functionality to your schema. + +- **Anti-patterns:** + - **Over-fetching/Under-fetching:** Avoid returning more or less data than required by the client. + - **Chatty APIs:** Reduce the number of round trips between the client and the server. + - **God Objects:** Avoid creating large, monolithic types with too many fields. + +- **State Management:** + - Use a state management library (e.g., Redux, Zustand, Jotai) to manage client-side state. + - Consider using Apollo Client's local state management features for simple state requirements. + +- **Error Handling:** + - Use a consistent error handling mechanism across your application. + - Return informative error messages to the client. + - Log errors on the server for debugging purposes. + +## 6. Testing Approaches + +- **Unit Testing:** + - Unit test individual resolvers and data sources. + - Mock external dependencies to isolate the code under test. + +- **Integration Testing:** + - Integrate test your GraphQL API with your database and other backend services. + - Use a testing framework like Jest or Mocha for writing integration tests. + +- **End-to-End Testing:** + - Use end-to-end testing to verify the entire application flow. + - Use a testing tool like Cypress or Puppeteer for writing end-to-end tests. + +- **Test Organization:** + - Organize your tests into separate directories based on functionality. + - Use descriptive names for your test files and test cases. + +- **Mocking and Stubbing:** + - Use mocking and stubbing techniques to isolate the code under test and simulate external dependencies. + +## 7. Common Pitfalls and Gotchas + +- **N+1 Problem:** Be aware of the N+1 query problem and use DataLoader or other batching techniques to solve it. +- **Schema Evolution:** Plan for schema evolution and use techniques like adding new fields and deprecating old ones to avoid breaking changes. +- **Performance Bottlenecks:** Monitor your API for performance bottlenecks and use profiling tools to identify slow resolvers. +- **Nullability:** Ensure that non-null fields never return `null` to avoid unexpected errors. + +## 8. Tooling and Environment + +- **Development Tools:** + - Use a GraphQL IDE like GraphiQL or Apollo Studio for exploring and testing your API. + - Use code generation tools to generate types and resolvers from your schema. + +- **Build Configuration:** + - Use a build tool like Webpack or Parcel to bundle your code for production. + - Configure your build tool to optimize your code and reduce bundle size. + +- **Linting and Formatting:** + - Use a linter like ESLint or Prettier to enforce code style and prevent errors. + +- **Deployment:** + - Deploy your GraphQL API to a production environment like AWS Lambda, Google Cloud Functions, or a Node.js server. + - Use a serverless platform for easy scaling and management. + +- **CI/CD Integration:** + - Integrate your GraphQL API with a CI/CD pipeline for automated testing and deployment. + +## 9. Additional Best Practices + +- **Versioning:** While GraphQL promotes continuous evolution, consider versioning your API if you need to make breaking changes. +- **Documentation:** Provide comprehensive documentation for your GraphQL API using tools like GraphQL Docs. +- **Monitoring:** Monitor your GraphQL API for performance and errors using tools like Apollo Studio or New Relic. +- **Error Messages and Notifications:** You can also opt for union types to represent an error and to prompt suggestions to users, though this is a more expensive choice. + +## 10. Global Identification + +- Another way to organize components, besides Pagination, is by using a global identification. Originally proposed on Relay and similar to URIs, this method has become a more general good practice though it is not considered mandatory — especially if you are not planning on supporting Relay in your application + +## 11. Rate Limiting + +- Like any other web API, setting limits is a good strategy to avoid, for example, an overload of requests per minute. There are a few ways this can be done in GraphQL + +## 12. Authentication and Authorization + +- Often interchanged in their meaning, authentication is the act of determining who a user is and whether they are logged in or not. Authorization, on the other hand, is the act of determining if a user is allowed to do an action or see something. + +## 13. Safelisting with Persisted queries + +- Beyond operation limits, GraphOS enables first-party apps to register trusted operations in a persisted query list ( PQL) or safelist. \ No newline at end of file diff --git a/.cursor/rules/asp-net.mdc b/.cursor/rules/asp-net.mdc new file mode 100644 index 000000000..460e7cccf --- /dev/null +++ b/.cursor/rules/asp-net.mdc @@ -0,0 +1,363 @@ +--- +description: This rule file provides best practices and coding standards for ASP.NET Core development, covering various aspects from code organization to security and performance optimization. +globs: *.cs +--- +# ASP.NET Core Best Practices and Coding Standards + +This document outlines recommended best practices and coding standards for developing ASP.NET Core applications. Following these guidelines will help ensure code clarity, maintainability, performance, and security. + +## 1. Code Organization and Structure + +### Directory Structure Best Practices: + +A well-defined directory structure is crucial for maintaining a scalable and organized ASP.NET Core project. Here's a suggested structure: + + +ProjectName/ +├── src/ +│ ├── ProjectName.Web/ +│ │ ├── Controllers/ +│ │ ├── Models/ +│ │ ├── Views/ +│ │ ├── Pages/ (for Razor Pages) +│ │ ├── wwwroot/ +│ │ ├── appsettings.json +│ │ ├── Program.cs +│ │ ├── Startup.cs +│ ├── ProjectName.Data/ +│ │ ├── Models/ +│ │ ├── Contexts/ +│ │ ├── Repositories/ +│ │ ├── Migrations/ +│ ├── ProjectName.Services/ +│ │ ├── Interfaces/ +│ │ ├── Implementations/ +│ ├── ProjectName.Models/ (if different from Data Models) +├── tests/ +│ ├── ProjectName.UnitTests/ +│ ├── ProjectName.IntegrationTests/ +├── artifacts/ +├── .editorconfig +├── .gitignore +├── ProjectName.sln + + +* **src:** Contains the source code of the application. +* **ProjectName.Web:** The main web application project. This can also be named `API` for web API projects. +* **Controllers:** Contains ASP.NET Core controllers. +* **Models:** Contains view models and other data transfer objects (DTOs) specific to the web layer. +* **Views:** Contains Razor views (.cshtml files). +* **Pages:** Contains Razor Pages (if using Razor Pages model). +* **wwwroot:** Contains static files like CSS, JavaScript, and images. +* **ProjectName.Data:** Contains data access logic, Entity Framework Core contexts, and models representing database entities. +* **Models (Data):** Defines entity models that correspond to database tables. +* **Contexts:** Contains the DbContext class. +* **Repositories:** Contains repositories for data access. +* **Migrations:** Contains Entity Framework Core migrations. +* **ProjectName.Services:** Contains business logic and application services. +* **Interfaces:** Defines interfaces for services. +* **Implementations:** Contains concrete implementations of service interfaces. +* **ProjectName.Models:** (Optional) Contains domain models that are shared across multiple layers. This layer promotes separation of concerns if your data layer models should not be exposed at the API or Web layer. +* **tests:** Contains unit and integration tests. +* **ProjectName.UnitTests:** Contains unit tests. +* **ProjectName.IntegrationTests:** Contains integration tests. +* **artifacts:** Contains build artifacts. +* **.editorconfig:** Defines code style rules. +* **.gitignore:** Specifies intentionally untracked files that Git should ignore. +* **ProjectName.sln:** The solution file. + +### File Naming Conventions: + +* **Classes:** PascalCase (e.g., `UserController`, `ProductService`) +* **Interfaces:** IPascalCase (e.g., `IProductService`, `IUserRepository`) +* **Methods:** PascalCase (e.g., `GetProduct`, `CreateUser`) +* **Variables (local):** camelCase (e.g., `productName`, `userId`) +* **Parameters:** camelCase (e.g., `productId`, `userName`) +* **Constants:** PascalCase (e.g., `DefaultPageSize`, `MaxRetries`) +* **Enums:** PascalCase (e.g., `OrderStatus`, `UserRole`) +* **Namespaces:** PascalCase, mirroring the directory structure (e.g., `ProjectName.Web.Controllers`, `ProjectName.Services.Implementations`) +* **Files:** Match the class/interface name (e.g., `UserController.cs`, `IProductService.cs`) + +### Module Organization: + +* **Feature-based Modules:** Group related functionality into modules based on features (e.g., `Authentication`, `Products`, `Orders`). Each feature can have its own folder within the `src` directory, containing its own controllers, models, services, and data access components. This enhances cohesion and reduces coupling. +* **Vertical Slices:** Organize modules around specific use cases or user stories. Each slice contains all the code necessary to implement a particular feature, from the UI to the data access layer. This promotes faster development and easier maintenance. + +### Component Architecture: + +* **Layered Architecture:** Divide the application into layers (e.g., Presentation, Application, Domain, Infrastructure) to separate concerns and promote testability. Each layer has a specific responsibility and interacts with other layers through well-defined interfaces. +* **Microservices:** For larger applications, consider breaking down the application into smaller, independent microservices. Each microservice handles a specific business capability and can be deployed and scaled independently. + +### Code Splitting Strategies: + +* **Feature-based Splitting:** Split code based on features, loading modules only when they are needed. This can improve initial load time and reduce memory consumption. +* **Route-based Splitting:** Load code only when a specific route is accessed. This is especially useful for large applications with many routes. +* **On-Demand Loading:** Load code dynamically when a user interacts with a specific component. This can improve perceived performance and reduce the initial download size. + +## 2. Common Patterns and Anti-patterns + +### Design Patterns: + +* **Dependency Injection (DI):** Use the built-in DI container to manage dependencies and promote loose coupling. +* **Repository Pattern:** Abstract data access logic behind repositories to improve testability and maintainability. Avoid exposing EF Core `IQueryable` outside of the repository. +* **Unit of Work Pattern:** Encapsulate multiple operations into a single transaction to ensure data consistency. +* **CQRS (Command Query Responsibility Segregation):** Separate read and write operations into separate models to optimize performance and scalability. +* **Mediator Pattern:** Decouple components by routing messages through a central mediator. Libraries like MediatR are excellent for this. +* **Specification Pattern:** Encapsulate complex query logic into reusable specification objects. +* **Strategy Pattern:** Define a family of algorithms, encapsulate each one, and make them interchangeable. + +### Recommended Approaches for Common Tasks: + +* **Configuration:** Use the `IOptions` pattern to access configuration settings in a strongly-typed manner. +* **Logging:** Use the `ILogger` interface for logging. Configure logging providers in `appsettings.json` and leverage structured logging. +* **Exception Handling:** Implement global exception handling middleware to handle unhandled exceptions and return appropriate error responses. +* **Data Validation:** Use data annotations and the `ModelState` property for validating user input. Implement custom validation attributes for complex validation rules. +* **Asynchronous Programming:** Use `async` and `await` for I/O-bound operations to avoid blocking threads. +* **HTTP Requests:** Utilize `HttpClientFactory` for creating and managing `HttpClient` instances. +* **Background Tasks:** Use hosted services or background worker services for running long-running tasks asynchronously. + +### Anti-patterns and Code Smells: + +* **Tight Coupling:** Avoid tight coupling between components. Use interfaces and dependency injection to promote loose coupling. +* **God Classes:** Avoid creating large classes with too many responsibilities. Break down large classes into smaller, more manageable classes. +* **Shotgun Surgery:** Avoid making changes to multiple classes when a single responsibility changes. This indicates poor separation of concerns. +* **Primitive Obsession:** Avoid using primitive types to represent domain concepts. Create value objects to encapsulate domain logic and validation rules. +* **Feature Envy:** Avoid classes that access the data of another class more than their own. Move the method to the class where the data resides. +* **Sync-over-Async:** Avoid using `Task.Wait()` or `Task.Result` in asynchronous methods, as this can lead to thread pool starvation. Always use `await`. +* **Ignoring Exceptions:** Avoid catching exceptions and not logging or handling them appropriately. +* **Magic Strings/Numbers:** Avoid hardcoding values directly in the code. Use constants or configuration settings instead. +* **Returning raw Entities:** Do not expose EF Core entities directly to the client. Create DTOs or View Models to represent the data that needs to be returned. + +### State Management Best Practices: + +* **Stateless Controllers:** Design controllers to be stateless. Avoid storing state in controller fields. +* **Session State:** Use session state sparingly. Session state can impact scalability. Consider using a distributed cache for session state. +* **Cookies:** Use cookies for storing small amounts of non-sensitive data. Be mindful of cookie size limits and security concerns. +* **TempData:** Use TempData for passing data between redirects. TempData is stored in session or cookies. +* **Caching:** Use caching (memory cache, distributed cache, response caching) to improve performance and reduce database load. Cache aggressively, but invalidate appropriately. + +### Error Handling Patterns: + +* **Global Exception Handling:** Implement global exception handling middleware to catch unhandled exceptions and return consistent error responses. +* **Exception Filters:** Use exception filters to handle exceptions at the controller level. +* **Try-Catch Blocks:** Use try-catch blocks for handling expected exceptions. +* **Problem Details:** Return problem details (RFC 7807) in error responses to provide more detailed information about the error. +* **Idempotency:** Design APIs to be idempotent where possible, so that retries are safe. + +## 3. Performance Considerations + +### Optimization Techniques: + +* **Asynchronous Programming:** Use `async` and `await` for I/O-bound operations. +* **Caching:** Implement caching strategies to reduce database load and improve response times. +* **Response Compression:** Enable response compression to reduce the size of HTTP responses. +* **Minification and Bundling:** Minify and bundle CSS and JavaScript files to reduce the number of HTTP requests and improve load times. +* **Image Optimization:** Optimize images to reduce their file size without sacrificing quality. +* **Database Optimization:** Optimize database queries and indexing to improve data access performance. +* **Connection Pooling:** Use connection pooling to reuse database connections and reduce connection overhead. +* **HTTP/2:** Use HTTP/2 for improved performance and reduced latency. +* **Precompiled Views:** Precompile Razor views to improve startup time. +* **Use Span and Memory:** Utilize these types for efficient memory manipulation, reducing allocations and copies. +* **Object Pooling:** Employ object pooling for frequently created and destroyed objects to reduce garbage collection overhead. `ArrayPool` is especially effective for large arrays. +* **Optimize LINQ Usage:** Use LINQ effectively, avoiding unnecessary iterations and allocations. Utilize `AsNoTracking()` for read-only queries and be mindful of client-side evaluation. +* **Use System.Text.Json:** Prefer `System.Text.Json` over `Newtonsoft.Json` where possible, as it offers better performance and is designed for UTF-8 text. + +### Memory Management: + +* **Minimize Object Allocations:** Reduce object allocations, especially in hot code paths. Consider using object pooling and value types. +* **Avoid Large Objects:** Avoid allocating large objects (>= 85,000 bytes) as they are stored on the large object heap and can cause performance issues. +* **Dispose of Resources:** Dispose of disposable resources (e.g., database connections, file streams) properly to prevent memory leaks. +* **Garbage Collection:** Understand how the garbage collector works and avoid patterns that trigger frequent garbage collections. + +### Rendering Optimization: + +* **Partial Views:** Use partial views to reuse code and reduce code duplication. +* **View Components:** Use view components to encapsulate complex rendering logic. +* **Tag Helpers:** Use tag helpers to create reusable UI components. +* **Client-Side Rendering:** Consider using client-side rendering frameworks (e.g., React, Angular, Vue.js) for complex UI interactions. +* **Lazy Loading:** Implement lazy loading for images and other assets to improve initial load time. + +### Bundle Size Optimization: + +* **Tree Shaking:** Use tree shaking to remove unused code from JavaScript bundles. +* **Code Splitting:** Split code into smaller chunks that can be loaded on demand. +* **Compression:** Compress JavaScript and CSS files using gzip or Brotli. +* **Dead Code Elimination:** Remove unused code from the application. + +### Lazy Loading Strategies: + +* **Lazy Loading of Images:** Load images only when they are visible in the viewport. +* **Lazy Loading of Modules:** Load modules only when they are needed. +* **Lazy Loading of Data:** Load data only when it is requested. + +## 4. Security Best Practices + +### Common Vulnerabilities and Prevention: + +* **SQL Injection:** Use parameterized queries or ORMs to prevent SQL injection attacks. +* **Cross-Site Scripting (XSS):** Sanitize user input and encode output to prevent XSS attacks. +* **Cross-Site Request Forgery (CSRF):** Use anti-forgery tokens to prevent CSRF attacks. +* **Authentication and Authorization:** Implement strong authentication and authorization mechanisms to protect sensitive data and functionality. +* **Clickjacking:** Protect against clickjacking attacks by using the X-Frame-Options header. +* **Denial of Service (DoS):** Implement rate limiting and other measures to prevent DoS attacks. +* **Open Redirects:** Validate redirect URLs to prevent open redirect vulnerabilities. +* **Insecure Direct Object References (IDOR):** Implement proper authorization checks to prevent unauthorized access to resources. + +### Input Validation: + +* **Server-Side Validation:** Always validate user input on the server-side. +* **Whitelisting:** Use whitelisting to allow only valid characters and data formats. +* **Data Annotations:** Use data annotations to validate model properties. +* **Custom Validation:** Implement custom validation attributes for complex validation rules. +* **Sanitization:** Sanitize user input to remove potentially harmful characters. + +### Authentication and Authorization: + +* **Authentication Schemes:** Choose an appropriate authentication scheme (e.g., cookies, JWT, OAuth2) based on the application requirements. +* **Authorization Policies:** Use authorization policies to define access control rules. +* **Role-Based Access Control (RBAC):** Implement RBAC to grant users access to specific resources based on their roles. +* **Claims-Based Authorization:** Use claims-based authorization to grant users access to specific resources based on their claims. +* **Secure Password Storage:** Use strong password hashing algorithms (e.g., bcrypt, Argon2) to store passwords securely. +* **Multi-Factor Authentication (MFA):** Implement MFA to enhance security. +* **Implement OWASP Recommendations:** Follow the OWASP (Open Web Application Security Project) guidelines for secure authentication and authorization. + +### Data Protection: + +* **Encryption:** Encrypt sensitive data at rest and in transit. +* **Data Protection API:** Use the Data Protection API to protect sensitive data. +* **Key Management:** Implement proper key management practices to protect encryption keys. +* **HTTPS:** Use HTTPS to encrypt communication between the client and the server. + +### Secure API Communication: + +* **API Keys:** Use API keys to authenticate API clients. +* **OAuth 2.0:** Use OAuth 2.0 to authorize API clients. +* **JWT (JSON Web Tokens):** Use JWT for secure API authentication and authorization. +* **Rate Limiting:** Implement rate limiting to prevent API abuse. +* **Input Validation:** Implement proper input validation on the server-side to prevent injection attacks. +* **Output Encoding:** Encode API responses to prevent XSS attacks. +* **CORS (Cross-Origin Resource Sharing):** Configure CORS to allow only authorized domains to access the API. + +## 5. Testing Approaches + +### Unit Testing: + +* **Test-Driven Development (TDD):** Write unit tests before writing the code. +* **Arrange, Act, Assert (AAA):** Follow the AAA pattern in unit tests. +* **Mocking:** Use mocking frameworks (e.g., Moq, NSubstitute) to isolate the code being tested. +* **Code Coverage:** Aim for high code coverage. +* **Focus on Business Logic:** Unit test the business logic of the application. + +### Integration Testing: + +* **Test Data Access:** Test the interaction between the application and the database. +* **Test External Services:** Test the interaction between the application and external services. +* **Use a Test Database:** Use a separate test database for integration tests. +* **Seed the Database:** Seed the test database with test data before running integration tests. + +### End-to-End Testing: + +* **Test the Entire Application Flow:** Test the entire application flow from the user interface to the database. +* **Use Automation Tools:** Use automation tools (e.g., Selenium, Cypress) to automate end-to-end tests. +* **Test in a Production-Like Environment:** Test in a production-like environment to ensure that the application works as expected in production. + +### Test Organization: + +* **Separate Test Projects:** Create separate test projects for unit tests, integration tests, and end-to-end tests. +* **Organize Tests by Feature:** Organize tests by feature or module. +* **Use Naming Conventions:** Use consistent naming conventions for test classes and methods. + +### Mocking and Stubbing: + +* **Use Mocking Frameworks:** Use mocking frameworks to create mock objects and stubs. +* **Mock Dependencies:** Mock external dependencies to isolate the code being tested. +* **Verify Interactions:** Verify that the code being tested interacts with its dependencies as expected. +* **Avoid Over-Mocking:** Avoid mocking too many dependencies. Focus on mocking external dependencies and complex objects. + +## 6. Common Pitfalls and Gotchas + +### Frequent Mistakes: + +* **Incorrect Dependency Injection Configuration:** Incorrectly configuring dependency injection can lead to runtime errors. +* **Not Handling Exceptions Properly:** Not handling exceptions properly can lead to application crashes. +* **Blocking Calls:** Making blocking calls in asynchronous methods can lead to thread pool starvation. +* **Incorrect Data Validation:** Incorrect data validation can lead to security vulnerabilities and data corruption. +* **Over-Engineering:** Over-engineering the application can lead to unnecessary complexity. +* **Premature Optimization:** Optimizing the application too early can lead to wasted effort and code that is difficult to maintain. +* **Ignoring Security Best Practices:** Ignoring security best practices can lead to security vulnerabilities. +* **Not Writing Tests:** Not writing tests can lead to code that is difficult to maintain and prone to errors. +* **Leaking Database Connections:** Failing to properly dispose of database connections can exhaust connection resources and cause performance issues. +* **Using Sync over Async:** Blocking asynchronous operations can lead to thread pool starvation and performance degradation. +* **Not using HttpClientFactory correctly:** Improper disposal of `HttpClient` instances leads to socket exhaustion. Always use `HttpClientFactory`. + +### Edge Cases: + +* **Handling Null Values:** Handle null values properly to prevent null reference exceptions. +* **Handling Empty Collections:** Handle empty collections properly to prevent unexpected behavior. +* **Handling Large Data Sets:** Handle large data sets efficiently to prevent performance issues. +* **Handling Time Zones:** Handle time zones correctly to prevent data inconsistencies. + +### Version-Specific Issues: + +* **Breaking Changes:** Be aware of breaking changes between different versions of ASP.NET Core. +* **Deprecated Features:** Be aware of deprecated features and plan to migrate to newer features. +* **Performance Improvements:** Take advantage of performance improvements in newer versions of ASP.NET Core. +* **Security Patches:** Stay up-to-date with security patches to protect against vulnerabilities. + +### Compatibility Concerns: + +* **.NET Framework vs .NET Core:** Be aware of the differences between .NET Framework and .NET Core and choose the appropriate framework for the application. +* **Operating System Compatibility:** Ensure that the application is compatible with the target operating system. +* **Browser Compatibility:** Ensure that the application is compatible with the target browsers. + +### Debugging Strategies: + +* **Logging:** Use logging to track down errors and unexpected behavior. +* **Debugging Tools:** Use debugging tools (e.g., Visual Studio debugger) to step through the code and examine variables. +* **Profiling Tools:** Use profiling tools to identify performance bottlenecks. +* **Remote Debugging:** Use remote debugging to debug applications running on remote servers. +* **Diagnostic Tools:** Use diagnostic tools (e.g., Application Insights) to monitor the health and performance of the application. + +## 7. Tooling and Environment + +### Recommended Development Tools: + +* **Visual Studio:** A comprehensive IDE for developing ASP.NET Core applications. +* **Visual Studio Code:** A lightweight and versatile code editor for developing ASP.NET Core applications. +* **.NET CLI:** A command-line tool for creating, building, and running ASP.NET Core applications. +* **NuGet Package Manager:** A package manager for managing dependencies in ASP.NET Core applications. +* **Postman:** A tool for testing APIs. +* **Fiddler:** A tool for debugging HTTP traffic. + +### Build Configuration: + +* **Use a Build Server:** Use a build server (e.g., Azure DevOps, Jenkins) to automate the build process. +* **Configure Build Configurations:** Configure build configurations (e.g., Debug, Release) to optimize the application for different environments. +* **Use NuGet Package Restore:** Use NuGet package restore to automatically download dependencies during the build process. +* **Version Control:** Use version control (e.g., Git) to track changes to the code. + +### Linting and Formatting: + +* **.editorconfig:** Use an .editorconfig file to define code style rules. +* **Code Analyzers:** Use code analyzers to enforce code style rules and detect potential errors. +* **StyleCop:** Use StyleCop to enforce coding style guidelines. +* **ReSharper:** Use ReSharper to improve code quality and productivity. +* **Format on Save:** Configure the IDE to format code on save. + +### Deployment Best Practices: + +* **Choose a Hosting Environment:** Choose an appropriate hosting environment (e.g., Azure, AWS, on-premises server) based on the application requirements. +* **Configure Application Settings:** Configure application settings (e.g., connection strings, API keys) for the production environment. +* **Use HTTPS:** Use HTTPS to encrypt communication between the client and the server. +* **Enable Logging:** Enable logging to track errors and unexpected behavior in the production environment. +* **Monitor the Application:** Monitor the health and performance of the application in the production environment. +* **Use a Deployment Pipeline:** Automate the deployment process with a CI/CD pipeline. +* **Consider Containerization:** Deploy your application using containers (e.g., Docker) to ensure consistency across environments. +* **Implement Health Checks:** Use ASP.NET Core's built-in health checks endpoint to monitor the health of your application. + +### CI/CD Integration: + +* **Use a CI/CD Pipeline:** Use a CI/CD pipeline (e.g., Azure DevOps, GitHub Actions, GitLab CI) to automate the build, test, and deployment process. +* **Automate Testing:** Automate unit tests, integration tests, and end-to-end tests in the CI/CD pipeline. +* **Automate Deployment:** Automate the deployment process in the CI/CD pipeline. +* **Implement Rollbacks:** Implement rollbacks to quickly revert to a previous version of the application if there are problems with the new deployment. \ No newline at end of file diff --git a/.cursor/rules/astro.mdc b/.cursor/rules/astro.mdc new file mode 100644 index 000000000..a258e5774 --- /dev/null +++ b/.cursor/rules/astro.mdc @@ -0,0 +1,176 @@ +--- +description: This rule provides comprehensive best practices and coding standards for developing Astro projects. It covers code organization, performance, security, testing, and common pitfalls. +globs: *.astro +--- +# Astro Library Best Practices and Coding Standards + +This document outlines the recommended best practices and coding standards for developing Astro projects to ensure maintainability, performance, and security. + +## 1. Code Organization and Structure + +### 1.1 Project Structure + +- **`src/` Directory:** This directory contains the core source code of your Astro project. + - **`src/pages/`:** This directory is crucial for routing. Every `.astro`, `.md`, or `.mdx` file in this directory automatically becomes a route. Use a clear and consistent naming convention for your pages (e.g., `about.astro`, `blog/index.astro`, `blog/[slug].astro`). + - **`src/components/`:** Store reusable UI components in this directory. Organize components into subdirectories based on their functionality or feature area (e.g., `src/components/Header/`, `src/components/Card/`). + - **`src/layouts/`:** Layouts define the overall structure of your pages (e.g., header, footer, navigation). Use layouts to maintain a consistent look and feel across your website. + - **`src/content/`:** (Recommended for content-driven sites) Use the Content Collections feature to manage structured content like blog posts, documentation, or product listings. Define schemas for your content types to ensure data consistency. + - **`src/styles/`:** Store global styles and CSS variables in this directory. Consider using CSS Modules or a CSS-in-JS solution for component-specific styling. + - **`src/scripts/`:** Place client-side JavaScript files in this directory. Use modules and avoid polluting the global scope. + - **`src/assets/`:** Store static assets like images, fonts, and other media files in this directory. +- **`public/` Directory:** This directory contains static assets that don't need processing, such as `robots.txt`, `favicon.ico`, and other files that should be served directly. Avoid placing images that require optimization in this directory; use the `src/assets/` directory instead. +- **`astro.config.mjs`:** This file contains the Astro configuration options, including integrations, build settings, and more. Keep this file well-organized and documented. +- **`.env`:** Store environment variables in this file. Use a library like `dotenv` to load environment variables into your application. + +### 1.2 Component Design + +- **Atomic Design Principles:** Consider using Atomic Design principles to structure your components into atoms, molecules, organisms, templates, and pages. This promotes reusability and maintainability. +- **Single Responsibility Principle:** Each component should have a single, well-defined responsibility. Avoid creating large, complex components that do too much. +- **Props and Slots:** Use props to pass data to components and slots to allow components to accept children. Define prop types using TypeScript to ensure type safety. + +## 2. Common Patterns and Anti-patterns + +### 2.1 UI Framework Integration + +- **Island Architecture:** Embrace Astro's island architecture to minimize JavaScript and improve performance. Only hydrate interactive components. +- **Client Directives:** Use `client:load`, `client:idle`, `client:visible`, `client:media`, and `client:only` directives appropriately to control when components are hydrated. Avoid over-hydrating components. +- **Framework Component Composition:** Compose components from different frameworks (React, Vue, Svelte, etc.) within the same Astro page. This allows you to leverage the strengths of different frameworks. +- **Passing Props and Children:** Pass props and children to framework components from Astro components. Use named slots to organize children. +- **Anti-pattern: Mixing Astro and Framework Components:** Do not import `.astro` components directly into UI framework components (e.g., `.jsx` or `.svelte`). Use slots to pass static content from Astro components to framework components. + +### 2.2 Data Fetching + +- **`fetch` API:** Use the `fetch` API to fetch data from external sources or internal APIs. Handle errors appropriately. +- **Content Collections API:** Use the Content Collections API to manage structured content like blog posts or documentation. Define schemas for your content types. +- **CMS Integration:** Integrate with a headless CMS like Hygraph, Contentful, or Strapi to manage content. Use the CMS's API to fetch data and display it on your website. +- **GraphQL:** When using a CMS that supports GraphQL (like Hygraph), use GraphQL queries to fetch only the data you need. This can improve performance and reduce data transfer. + +### 2.3 Routing + +- **File-Based Routing:** Use Astro's file-based routing to create routes automatically. Follow a consistent naming convention for your pages. +- **Dynamic Routes:** Use dynamic routes (e.g., `[slug].astro`) to handle variable segments in the URL. Access the route parameters using `Astro.params`. +- **Nested Routes:** Use nested folders within `src/pages/` to create nested routes. +- **Anti-pattern: Overly Complex Routing:** Avoid creating overly complex routing structures. Keep your routes simple and intuitive. + +### 2.4 Styling + +- **Scoped CSS:** Use scoped CSS within Astro components to avoid style conflicts. This ensures that component styles are isolated. +- **Global Styles:** Use global styles for site-wide styling. Store global styles in `src/styles/global.css`. +- **CSS Frameworks:** Use CSS frameworks like Tailwind CSS, Bootstrap, or Materialize to speed up development and maintain a consistent look and feel. Install the appropriate integration for your chosen CSS framework. +- **CSS Modules:** Consider using CSS Modules for component-specific styling. CSS Modules generate unique class names to avoid naming collisions. + +## 3. Performance Considerations + +### 3.1 Minimizing JavaScript + +- **Static-First Architecture:** Embrace Astro's static-first architecture to minimize JavaScript and improve performance. Render as much as possible as static HTML. +- **Island Architecture:** Only hydrate interactive components. Avoid over-hydrating components. +- **Code Splitting:** Use code splitting to break up your JavaScript into smaller chunks. This can improve initial load times. +- **Lazy Loading:** Use lazy loading for images and other assets that are not immediately visible. This can improve initial load times. + +### 3.2 Image Optimization + +- **Astro's `` Component:** Use Astro's built-in `` component to optimize images. The `` component automatically optimizes images, resizes them, and generates responsive versions. +- **Image Formats:** Use modern image formats like WebP to reduce file sizes. Use AVIF for even better compression, where supported. +- **Compression:** Compress images to reduce file sizes. Use tools like ImageOptim or TinyPNG. +- **Lazy Loading:** Use lazy loading for images that are not immediately visible. +- **Responsive Images:** Use responsive images to serve different image sizes based on the user's screen size. + +### 3.3 Font Optimization + +- **Web Font Formats:** Use modern web font formats like WOFF2. These formats offer better compression and performance. +- **Font Loading:** Use font loading strategies to avoid blocking rendering. Consider using `font-display: swap`. +- **Preloading:** Preload important fonts to improve loading times. + +### 3.4 Caching + +- **Browser Caching:** Configure browser caching to cache static assets like images, fonts, and JavaScript files. This can improve performance for returning users. +- **CDN:** Use a Content Delivery Network (CDN) to serve static assets from geographically distributed servers. This can improve performance for users around the world. + +## 4. Security Best Practices + +### 4.1 Input Validation + +- **Validate User Input:** Validate all user input to prevent injection attacks and other security vulnerabilities. Use server-side validation in addition to client-side validation. +- **Sanitize User Input:** Sanitize user input to remove potentially malicious code. Use a library like DOMPurify to sanitize HTML. + +### 4.2 Cross-Site Scripting (XSS) + +- **Escape Output:** Escape all output to prevent XSS attacks. Use Astro's built-in escaping mechanisms or a library like Handlebars.js. +- **Content Security Policy (CSP):** Implement a Content Security Policy (CSP) to control the resources that the browser is allowed to load. This can help prevent XSS attacks. + +### 4.3 Cross-Site Request Forgery (CSRF) + +- **CSRF Tokens:** Use CSRF tokens to protect against CSRF attacks. Generate a unique CSRF token for each user session and include it in all forms. + +### 4.4 Authentication and Authorization + +- **Secure Authentication:** Use secure authentication mechanisms to protect user accounts. Use a library like Passport.js or Auth0. +- **Authorization:** Implement authorization to control access to resources. Use roles and permissions to define what users are allowed to do. + +### 4.5 Dependency Management + +- **Keep Dependencies Up-to-Date:** Keep your dependencies up-to-date to patch security vulnerabilities. Use a tool like `npm audit` or `yarn audit` to identify and fix vulnerabilities. +- **Lock Dependencies:** Lock your dependencies to prevent unexpected changes. Use `package-lock.json` or `yarn.lock`. + +### 4.6 Environment Variables + +- **Store Secrets Securely:** Store secrets like API keys and database passwords in environment variables. Do not hardcode secrets in your code. +- **Avoid Committing `.env`:** Never commit your `.env` file to version control. Use a `.gitignore` file to exclude it. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +- **Test Individual Components:** Write unit tests to test individual components in isolation. Use a testing framework like Jest or Mocha. +- **Mock Dependencies:** Mock dependencies to isolate components during testing. + +### 5.2 Integration Testing + +- **Test Component Interactions:** Write integration tests to test how components interact with each other. Use a testing framework like Cypress or Playwright. + +### 5.3 End-to-End Testing + +- **Test User Flows:** Write end-to-end tests to test complete user flows. Use a testing framework like Cypress or Playwright. + +### 5.4 Accessibility Testing + +- **Automated Accessibility Testing:** Use automated accessibility testing tools to identify accessibility issues. Use a tool like axe-core. +- **Manual Accessibility Testing:** Perform manual accessibility testing to ensure that your website is accessible to users with disabilities. Use a screen reader to test your website. + +## 6. Common Pitfalls and Gotchas + +- **Over-Hydration:** Avoid over-hydrating components. Only hydrate interactive components. +- **Incorrect Client Directives:** Use the correct client directives for your components. Using the wrong client directive can lead to performance issues or unexpected behavior. +- **Global Scope Pollution:** Avoid polluting the global scope with JavaScript variables. Use modules to encapsulate your code. +- **Missing `alt` Attributes:** Always include `alt` attributes for images. The `alt` attribute is important for accessibility and SEO. +- **Hardcoded Secrets:** Never hardcode secrets in your code. Store secrets in environment variables. +- **Insecure Dependencies:** Keep your dependencies up-to-date to patch security vulnerabilities. +- **Not handling errors:** Always handle errors gracefully, especially when fetching data or interacting with external APIs. + +## 7. Tooling and Environment + +### 7.1 Code Editor + +- **Visual Studio Code:** Use Visual Studio Code as your code editor. VS Code has excellent support for Astro and other web development technologies. +- **Astro VS Code Extension:** Install the Astro VS Code extension for syntax highlighting, code completion, and other features. +- **ESLint and Prettier:** Install ESLint and Prettier for code linting and formatting. + +### 7.2 Package Manager + +- **npm, Yarn, or pnpm:** Use npm, Yarn, or pnpm as your package manager. Choose the package manager that best suits your needs. + +### 7.3 Build Tool + +- **Astro's Built-in Build Tool:** Use Astro's built-in build tool to build your website. The build tool automatically optimizes your code and assets. + +### 7.4 Version Control + +- **Git:** Use Git for version control. Git allows you to track changes to your code and collaborate with other developers. +- **GitHub, GitLab, or Bitbucket:** Use GitHub, GitLab, or Bitbucket to host your Git repository. These platforms provide a centralized location for managing and collaborating on code. + +### 7.5 Deployment + +- **Netlify, Vercel, or Cloudflare Pages:** Use Netlify, Vercel, or Cloudflare Pages to deploy your website. These platforms provide easy-to-use deployment workflows and CDN integration. + +By following these best practices and coding standards, you can ensure that your Astro projects are maintainable, performant, and secure. \ No newline at end of file diff --git a/.cursor/rules/asyncio.mdc b/.cursor/rules/asyncio.mdc new file mode 100644 index 000000000..f697b60c0 --- /dev/null +++ b/.cursor/rules/asyncio.mdc @@ -0,0 +1,371 @@ +--- +description: This rule provides comprehensive guidelines and best practices for utilizing the asyncio library in Python, covering code organization, performance, security, testing, and common pitfalls. +globs: *.py +--- +# asyncio Best Practices and Coding Standards + +This document outlines comprehensive best practices for using the `asyncio` library in Python. It covers various aspects of asyncio development, including code organization, common patterns, performance considerations, security, testing, common pitfalls, and tooling. + +## Library Information: + +- Name: asyncio +- Tags: python, async, standard-library + +## 1. Code Organization and Structure + +Effective code organization is crucial for maintainability and scalability when working with asyncio. + +### 1.1 Directory Structure Best Practices + +A well-defined directory structure helps in organizing different parts of your asyncio application. + + +project_root/ +├── src/ +│ ├── __init__.py +│ ├── main.py # Entry point of the application +│ ├── modules/ +│ │ ├── __init__.py +│ │ ├── networking.py # Handles networking tasks +│ │ ├── processing.py # Data processing tasks +│ │ └── utils.py # Utility functions +│ ├── config.py # Configuration settings +├── tests/ +│ ├── __init__.py +│ ├── test_networking.py +│ ├── test_processing.py +│ └── conftest.py # Fixtures and configuration for pytest +├── requirements.txt # Project dependencies +├── pyproject.toml # Project metadata and build system +├── README.md # Project documentation + + +### 1.2 File Naming Conventions + +Consistent file naming conventions enhance readability and maintainability. + +- Use descriptive names that reflect the module's purpose. +- Prefer lowercase with underscores (snake_case) for file names (e.g., `async_utils.py`, `data_handler.py`). +- Test files should follow the `test_*.py` pattern for pytest compatibility. + +### 1.3 Module Organization + +Divide your code into logical modules based on functionality. + +- Group related functions and classes within the same module. +- Use `__init__.py` files to make directories into Python packages. +- Employ relative imports for internal modules (`from . import utils`). +- Use absolute imports for external libraries (`import aiohttp`). + +python +# src/modules/networking.py +import asyncio +import aiohttp + +async def fetch_data(url: str) -> str: + async with aiohttp.ClientSession() as session: + async with session.get(url) as response: + return await response.text() + + +### 1.4 Component Architecture Recommendations + +Consider using a layered architecture to separate concerns. + +- **Presentation Layer**: Handles user interface or API endpoints. +- **Service Layer**: Contains business logic and orchestrates tasks. +- **Data Access Layer**: Manages data persistence and retrieval. +- **Infrastructure Layer**: Provides supporting services like logging and configuration. + +### 1.5 Code Splitting Strategies + +Split large modules into smaller, more manageable files. + +- Use functions and classes to encapsulate specific tasks. +- Consider splitting modules based on logical boundaries (e.g., one module for database interactions, another for API calls). +- Refactor complex coroutines into smaller, reusable coroutines. + +## 2. Common Patterns and Anti-patterns + +Understanding common patterns and anti-patterns helps in writing efficient and maintainable asyncio code. + +### 2.1 Design Patterns Specific to asyncio + +- **Producer-Consumer**: Use `asyncio.Queue` to manage tasks between producers and consumers. +- **Worker Pool**: Create a pool of worker coroutines to process tasks concurrently. +- **Pub-Sub**: Implement a publish-subscribe pattern using queues or custom event handling. + +#### Producer-Consumer Example + +python +import asyncio + +async def producer(queue: asyncio.Queue, data: list): + for item in data: + await queue.put(item) + print(f"Produced: {item}") + await queue.put(None) # Signal end of production + +async def consumer(queue: asyncio.Queue): + while True: + item = await queue.get() + if item is None: + break + print(f"Consumed: {item}") + queue.task_done() + +async def main(): + queue = asyncio.Queue() + data = [1, 2, 3, 4, 5] + producer_task = asyncio.create_task(producer(queue, data)) + consumer_task = asyncio.create_task(consumer(queue)) + + await asyncio.gather(producer_task, consumer_task) + +if __name__ == "__main__": + asyncio.run(main()) + + +### 2.2 Recommended Approaches for Common Tasks + +- **Making HTTP Requests**: Use `aiohttp` for non-blocking HTTP requests. +- **Reading/Writing Files**: Use `aiofiles` for asynchronous file I/O. +- **Database Operations**: Utilize async database drivers like `aiopg` or `asyncpg`. +- **Task Scheduling**: Use `asyncio.create_task` or `asyncio.gather` to manage concurrent tasks. + +### 2.3 Anti-patterns and Code Smells to Avoid + +- **Blocking Calls**: Avoid using blocking functions like `time.sleep` or `requests` in coroutines. Use `asyncio.sleep` and `aiohttp` instead. +- **Long-Running Coroutines**: Break down long-running coroutines into smaller, awaitable chunks to avoid blocking the event loop. +- **Ignoring Exceptions**: Always handle exceptions properly to prevent unexpected crashes. +- **Over-using Locks**: Excessive use of locks can reduce concurrency. Consider using queues or other synchronization primitives. +- **Unnecessary Context Switching**: Minimize context switches by optimizing code and reducing unnecessary `await` calls. + +### 2.4 State Management Best Practices + +- **Immutable Data**: Prefer immutable data structures to avoid race conditions. +- **Thread-Safe Data Structures**: Use thread-safe data structures from the `queue` or `collections` modules when sharing data between coroutines. +- **Avoid Global State**: Minimize the use of global state to reduce complexity and potential conflicts. + +### 2.5 Error Handling Patterns + +- **Try-Except Blocks**: Use `try-except` blocks to catch and handle exceptions within coroutines. +- **`asyncio.gather(..., return_exceptions=True)`**: Use `return_exceptions=True` in `asyncio.gather` to prevent one exception from canceling other tasks. +- **Logging Errors**: Log exceptions with detailed information for debugging purposes. +- **Graceful Shutdown**: Implement a mechanism to gracefully shut down the application and release resources. + +python +import asyncio +import logging + +async def my_coroutine(value): + try: + if value < 0: + raise ValueError("Value must be non-negative") + await asyncio.sleep(1) + return value * 2 + except ValueError as e: + logging.error(f"Error processing {value}: {e}") + return None + +async def main(): + results = await asyncio.gather( + my_coroutine(5), my_coroutine(-1), my_coroutine(10), return_exceptions=True + ) + print(f"Results: {results}") + +if __name__ == "__main__": + logging.basicConfig(level=logging.ERROR) + asyncio.run(main()) + + +## 3. Performance Considerations + +Optimizing performance is critical for asyncio applications. + +### 3.1 Optimization Techniques + +- **Minimize I/O Operations**: Reduce the number of I/O operations by batching requests or caching data. +- **Use Efficient Data Structures**: Choose appropriate data structures for specific tasks (e.g., dictionaries for fast lookups). +- **Avoid Unnecessary Copying**: Minimize copying data to reduce memory usage and improve performance. +- **Profile Your Code**: Use profiling tools to identify performance bottlenecks. +- **Use `uvloop`**: Consider using `uvloop`, a fast, drop-in replacement for the default asyncio event loop. + +python +try: + import uvloop + asyncio.set_event_loop_policy(uvloop.EventLoopPolicy()) + print("Using uvloop") +except ImportError: + print("uvloop not installed, using default asyncio loop") + + +### 3.2 Memory Management + +- **Resource Management**: Properly release resources (e.g., file handles, database connections) when they are no longer needed. +- **Use Generators**: Use generators to process large datasets in chunks. +- **Limit Object Creation**: Reduce the creation of unnecessary objects to minimize memory overhead. + +### 3.3 Lazy Loading Strategies + +- **Import on Demand**: Import modules only when they are needed to reduce startup time. +- **Load Data Lazily**: Load data only when it is accessed to reduce initial memory usage. + +## 4. Security Best Practices + +Securing asyncio applications is essential for protecting against vulnerabilities. + +### 4.1 Common Vulnerabilities and Prevention + +- **Injection Attacks**: Sanitize user inputs to prevent SQL injection, command injection, and other injection attacks. +- **Cross-Site Scripting (XSS)**: Encode user-generated content to prevent XSS attacks. +- **Denial of Service (DoS)**: Implement rate limiting and input validation to prevent DoS attacks. +- **Man-in-the-Middle (MitM) Attacks**: Use TLS/SSL for secure communication to prevent MitM attacks. + +### 4.2 Input Validation + +- **Validate All Inputs**: Validate all user inputs and data received from external sources. +- **Use Regular Expressions**: Use regular expressions to validate input formats. +- **Limit Input Length**: Restrict the length of input fields to prevent buffer overflows. + +### 4.3 Authentication and Authorization + +- **Use Strong Authentication**: Implement strong authentication mechanisms (e.g., multi-factor authentication). +- **Implement Authorization**: Implement authorization checks to ensure users only have access to authorized resources. +- **Store Passwords Securely**: Hash passwords using strong hashing algorithms (e.g., bcrypt or Argon2). +- **Use JWTs**: Employ JSON Web Tokens (JWTs) for stateless authentication. + +### 4.4 Data Protection Strategies + +- **Encrypt Sensitive Data**: Encrypt sensitive data at rest and in transit. +- **Use Secure Protocols**: Use secure protocols (e.g., HTTPS, SSH) for communication. +- **Regularly Audit Security**: Conduct regular security audits to identify and address vulnerabilities. + +### 4.5 Secure API Communication + +- **Use HTTPS**: Always use HTTPS for API communication. +- **Validate API Responses**: Validate API responses to ensure data integrity. +- **Implement Rate Limiting**: Implement rate limiting to prevent abuse. + +## 5. Testing Approaches + +Testing is crucial for ensuring the reliability of asyncio applications. + +### 5.1 Unit Testing Strategies + +- **Test Coroutines in Isolation**: Use `asyncio.run` or `pytest-asyncio` to test coroutines in isolation. +- **Mock External Dependencies**: Use mocking libraries like `unittest.mock` to mock external dependencies. +- **Assert Expected Outcomes**: Use assertions to verify expected outcomes and error conditions. + +### 5.2 Integration Testing + +- **Test Interactions Between Components**: Test interactions between different components of the application. +- **Use Real Dependencies**: Use real dependencies (e.g., databases, APIs) in a controlled environment. + +### 5.3 End-to-End Testing + +- **Simulate Real User Scenarios**: Simulate real user scenarios to test the entire application flow. +- **Use Test Automation Frameworks**: Use test automation frameworks like Selenium or Playwright. + +### 5.4 Test Organization + +- **Organize Tests by Module**: Organize tests into separate files that correspond to the application modules. +- **Use Descriptive Test Names**: Use descriptive test names that clearly indicate what is being tested. + +### 5.5 Mocking and Stubbing Techniques + +- **Mock Asynchronous Functions**: Use `asyncio.iscoroutinefunction` to check if a function is a coroutine before mocking it. +- **Patch External Dependencies**: Use `unittest.mock.patch` to replace external dependencies with mock objects. +- **Use Asynchronous Mocks**: Use asynchronous mocks to simulate asynchronous behavior. + +python +import asyncio +import unittest.mock +import pytest + +async def fetch_data(url: str) -> str: + # This is just a placeholder; in real code, it would use aiohttp + await asyncio.sleep(0.1) # Simulate I/O delay + return f"Data from {url}" + +@pytest.mark.asyncio +async def test_fetch_data(): + with unittest.mock.patch("__main__.fetch_data") as mock_fetch_data: + mock_fetch_data.return_value = "Mocked data" + result = await fetch_data("http://example.com") + assert result == "Mocked data" + mock_fetch_data.assert_called_once_with("http://example.com") + + +## 6. Common Pitfalls and Gotchas + +Being aware of common pitfalls helps in avoiding mistakes when using asyncio. + +### 6.1 Frequent Mistakes + +- **Mixing Synchronous and Asynchronous Code**: Avoid mixing synchronous and asynchronous code in the same coroutine. +- **Forgetting to Await**: Ensure that you `await` all awaitable objects. +- **Blocking the Event Loop**: Avoid blocking the event loop with long-running synchronous operations. +- **Ignoring Task Cancellation**: Handle task cancellation properly to prevent resource leaks. +- **Not Handling Exceptions**: Always handle exceptions properly to prevent unexpected crashes. + +### 6.2 Edge Cases + +- **Task Timeouts**: Implement task timeouts to prevent tasks from running indefinitely. +- **Resource Limits**: Set resource limits (e.g., maximum number of connections) to prevent resource exhaustion. +- **Signal Handling**: Handle signals (e.g., SIGINT, SIGTERM) to gracefully shut down the application. + +### 6.3 Version-Specific Issues + +- **asyncio API Changes**: Be aware of API changes between different versions of asyncio. +- **Python 3.7+**: Use Python 3.7 or later to take advantage of the latest asyncio features (e.g., `asyncio.run`). + +### 6.4 Compatibility Concerns + +- **Third-Party Libraries**: Ensure that third-party libraries are compatible with asyncio. +- **Event Loop Implementations**: Be aware of compatibility issues between different event loop implementations (e.g., `uvloop`). + +### 6.5 Debugging Strategies + +- **Enable Debug Mode**: Enable asyncio debug mode to get more detailed information about tasks and coroutines. +- **Use Logging**: Use logging to track the execution flow and identify potential issues. +- **Use Debuggers**: Use debuggers like `pdb` or IDE-integrated debuggers to step through code and inspect variables. +- **Inspect Task State**: Inspect the state of tasks using `asyncio.Task.all_tasks()` to identify stuck or failing tasks. + +## 7. Tooling and Environment + +Using the right tools and environment can greatly improve the development experience. + +### 7.1 Recommended Development Tools + +- **IDE**: Use an IDE like VS Code, PyCharm, or Sublime Text with Python support. +- **Linters**: Use linters like `flake8` or `pylint` to enforce coding standards. +- **Formatters**: Use formatters like `black` or `autopep8` to automatically format code. +- **Debuggers**: Use debuggers like `pdb` or IDE-integrated debuggers to step through code and inspect variables. + +### 7.2 Build Configuration + +- **Use `pyproject.toml`**: Configure the project using a `pyproject.toml` file to specify build dependencies and settings. +- **Specify Dependencies**: Specify project dependencies in `requirements.txt` or `pyproject.toml`. +- **Use Virtual Environments**: Use virtual environments to isolate project dependencies. + +### 7.3 Linting and Formatting + +- **Configure Linters**: Configure linters to enforce coding standards (e.g., PEP 8). +- **Configure Formatters**: Configure formatters to automatically format code. +- **Use Pre-commit Hooks**: Use pre-commit hooks to automatically run linters and formatters before committing code. + +### 7.4 Deployment Best Practices + +- **Use a Production-Ready Event Loop**: Use a production-ready event loop like `uvloop`. +- **Use a Process Manager**: Use a process manager like systemd or Supervisor to manage the application process. +- **Use a Load Balancer**: Use a load balancer to distribute traffic across multiple instances of the application. +- **Monitor Application Health**: Monitor the application's health and performance using metrics and alerts. + +### 7.5 CI/CD Integration + +- **Automate Tests**: Automate unit, integration, and end-to-end tests in the CI/CD pipeline. +- **Automate Linting and Formatting**: Automate linting and formatting in the CI/CD pipeline. +- **Automate Deployment**: Automate deployment to production environments. + +By adhering to these best practices, you can build robust, efficient, and maintainable asyncio applications in Python. \ No newline at end of file diff --git a/.cursor/rules/auth0.mdc b/.cursor/rules/auth0.mdc new file mode 100644 index 000000000..df322aaca --- /dev/null +++ b/.cursor/rules/auth0.mdc @@ -0,0 +1,195 @@ +--- +description: This rule provides comprehensive guidance for Auth0 library usage, covering code organization, security, performance, testing, and common pitfalls. Adhering to these guidelines ensures secure, efficient, and maintainable Auth0 integrations. +globs: *.js,*.jsx,*.ts,*.tsx,*.html,*.css,*.scss,*.py,*.java,*.go +--- +--- + +# Auth0 Best Practices and Coding Standards + +This document outlines best practices for working with Auth0 to ensure secure, efficient, and maintainable authentication and authorization in your applications. + +## Library Information: + +- Name: Auth0 +- Tags: authentication, security, identity, SaaS + +## 1. Code Organization and Structure: + +- **Directory Structure:** + - Adopt a modular structure. For example: + + src/ + auth/ + auth0.config.js # Auth0 configuration + auth0.service.js # Authentication and authorization logic + components/ + login.jsx # Login component + profile.jsx # User profile component + utils/ + auth.helper.js # Helper functions for authentication + components/ + ... other application components + +- **File Naming Conventions:** + - Use descriptive names. Example: `auth0.config.js`, `AuthService.ts`, `LoginButton.jsx`. + - Follow a consistent naming convention (e.g., camelCase for JavaScript, PascalCase for React components). +- **Module Organization:** + - Group related Auth0 logic into modules. Avoid placing all Auth0-related code in a single, monolithic file. + - Use dependency injection to decouple Auth0 modules from other parts of your application. +- **Component Architecture (if applicable, e.g., React, Angular, Vue):** + - Create dedicated components for authentication-related UI elements (login buttons, profile displays, etc.). + - Separate concerns: keep UI components focused on rendering and delegate authentication logic to a separate service or module. +- **Code Splitting:** + - For large applications, consider lazy-loading Auth0-related code to improve initial load time. + - Implement code splitting at the route level, loading authentication modules only when necessary. + +## 2. Common Patterns and Anti-patterns: + +- **Design Patterns:** + - **Adapter Pattern:** Use an adapter to abstract the underlying Auth0 SDK and provide a consistent API for your application. + - **Singleton Pattern:** Ensure that only one instance of the Auth0 client is created. + - **Observer Pattern:** Use an observer pattern to notify components when the authentication state changes. +- **Recommended Approaches for Common Tasks:** + - **Authentication:** Use the Auth0 SDK to handle authentication flows (login, logout, password reset). + - **Authorization:** Implement role-based access control (RBAC) using Auth0's authorization features. + - **User Profile Management:** Leverage Auth0's user profile management features to store and retrieve user information. + - **Token Handling:** Securely store and manage access tokens and refresh tokens. Consider using HttpOnly cookies or secure storage mechanisms. +- **Anti-patterns and Code Smells:** + - **Hardcoding Credentials:** Never hardcode Auth0 credentials (Client ID, Client Secret) directly in your code. Use environment variables or secure configuration files. + - **Exposing Sensitive Data:** Avoid exposing sensitive user data (e.g., access tokens) in the client-side code or URLs. + - **Ignoring Error Handling:** Implement robust error handling to catch and handle authentication errors gracefully. + - **Overly Complex Rules/Actions:** Keep Auth0 Rules and Actions simple and focused. Delegate complex logic to external services. + - **Storing Secrets in Rules Code:** Don't store sensitive keys or credentials directly in the Rule code. Use the rule settings and the configuration object instead. +- **State Management:** + - Utilize context API, Redux, Zustand, or similar state management libraries to manage the user's authentication state (e.g., isLoggedIn, user profile). + - Ensure the authentication state is persisted across page reloads (e.g., using localStorage or sessionStorage). +- **Error Handling:** + - Use try-catch blocks to handle potential errors during authentication and authorization. + - Log errors to a central logging service for monitoring and debugging. + - Display user-friendly error messages to the user. + +## 3. Performance Considerations: + +- **Optimization Techniques:** + - **Caching:** Cache user profiles and access tokens to reduce the number of calls to Auth0. + - **Token Reuse:** Reuse existing access tokens whenever possible. + - **Minimize Data Transfer:** Only request the necessary user profile information from Auth0. +- **Memory Management:** + - Release resources (e.g., timers, event listeners) when they are no longer needed. + - Avoid creating unnecessary objects during authentication and authorization. +- **Rendering Optimization (if applicable):** + - Use techniques like memoization and virtualization to optimize the rendering of authentication-related UI components. +- **Bundle Size Optimization:** + - Use tree shaking to remove unused Auth0 code from the final bundle. + - Consider using a smaller alternative to the full Auth0 SDK if possible. +- **Lazy Loading:** + - Lazy load Auth0 modules and components to improve initial page load time. + +## 4. Security Best Practices: + +- **Common Vulnerabilities and Prevention:** + - **Cross-Site Scripting (XSS):** Sanitize user inputs to prevent XSS attacks. + - **Cross-Site Request Forgery (CSRF):** Protect against CSRF attacks by using anti-CSRF tokens. + - **Token Theft:** Securely store and manage access tokens to prevent token theft. + - **Injection Attacks:** Prevent injection attacks by validating and sanitizing all user inputs. +- **Input Validation:** + - Validate all user inputs to prevent malicious data from being processed. + - Use a validation library to simplify the input validation process. +- **Authentication and Authorization:** + - Use the Auth0 SDK for authentication and authorization. + - Implement role-based access control (RBAC) using Auth0's authorization features. + - Enforce multi-factor authentication (MFA) to enhance security. + - Regularly review security settings, enable breach detection features, and utilize logging to monitor authentication events for any suspicious activities. +- **Data Protection:** + - Encrypt sensitive data at rest and in transit. + - Use HTTPS for all communications. + - Protect API keys and other sensitive credentials. +- **Secure API Communication:** + - Always use HTTPS for all communications to protect sensitive data during transmission. + - Store sensitive information, like API keys, in the Auth0 rule settings instead of hardcoding them directly in your code to prevent exposure. + - Avoid sending the entire context object to external services. Send only a subset of the less sensitive attributes from the context object when and where necessary. In a similar fashion, avoid passing any aspect of the auth0 object outside of a rule. + +## 5. Testing Approaches: + +- **Unit Testing:** + - Write unit tests for Auth0-related modules and components. + - Mock the Auth0 SDK to isolate the code being tested. + - Test different authentication scenarios (successful login, failed login, password reset). +- **Integration Testing:** + - Integrate Auth0 with your application and test the entire authentication flow. + - Use a test Auth0 tenant for integration testing. + - Verify that authentication and authorization work as expected. +- **End-to-End Testing:** + - Use end-to-end testing frameworks (e.g., Cypress, Selenium) to test the entire application from the user's perspective. + - Simulate user interactions and verify that authentication works correctly. +- **Test Organization:** + - Organize tests into separate directories based on the modules or components being tested. + - Use descriptive test names to clearly indicate the purpose of each test. +- **Mocking and Stubbing:** + - Use mocking libraries (e.g., Jest, Mocha) to mock the Auth0 SDK and other external dependencies. + - Use stubbing to replace external dependencies with predefined values. + +## 6. Common Pitfalls and Gotchas: + +- **Frequent Mistakes:** + - Forgetting to configure the callback URL in the Auth0 dashboard. + - Incorrectly configuring the allowed logout URLs. + - Failing to handle authentication errors gracefully. + - Storing sensitive data in client-side code. + - Not enabling multi-factor authentication (MFA). +- **Edge Cases:** + - Handling expired access tokens. + - Dealing with network connectivity issues. + - Supporting different browsers and devices. + - Managing user sessions. +- **Version-Specific Issues:** + - Be aware of breaking changes in Auth0 SDK updates. + - Refer to the Auth0 documentation for migration guides. +- **Compatibility Concerns:** + - Ensure that the Auth0 SDK is compatible with the other technologies used in your application. + - Test the integration with different browsers and devices. +- **Debugging:** + - Use the Auth0 logs to troubleshoot authentication issues. + - Use browser developer tools to inspect network requests and responses. + - Set breakpoints in your code to debug authentication logic. + +## 7. Tooling and Environment: + +- **Recommended Tools:** + - Auth0 CLI: For managing Auth0 tenants and applications. + - Auth0 Management API: For programmatically managing Auth0 resources. + - Auth0 SDKs: For integrating Auth0 into your applications. + - Postman or Insomnia: For testing API endpoints. +- **Build Configuration:** + - Use a build tool (e.g., Webpack, Parcel, Rollup) to bundle your application. + - Configure the build tool to use tree shaking to remove unused Auth0 code. + - Minify the code to reduce the bundle size. +- **Linting and Formatting:** + - Use a linter (e.g., ESLint, JSHint) to enforce code style and prevent errors. + - Use a code formatter (e.g., Prettier) to automatically format your code. +- **Deployment:** + - Deploy your Auth0 configuration to production using the Auth0 CLI or Management API. + - Use environment variables to configure your application for different environments. + - Monitor the application for errors and performance issues. +- **CI/CD Integration:** + - Integrate Auth0 deployment into your CI/CD pipeline. + - Use automated tests to verify the integration before deploying to production. + +## Additional Best Practices: + +* **Use Identity Industry Standards:** Use OAuth 2.0 and OpenID Connect for interoperability and compliance. +* **Regularly Review Security Settings:** Enable breach detection and anomaly detection features. Configure rate limits to protect against brute force attacks. +* **Store Configuration Values in the Dashboard:** If your Actions, Rules, Hooks, custom database scripts, or Webtasks require configuration values (such as credentials or API keys), store them in the Auth0 Dashboard. This makes migrating configuration between tenants easier. +* **Add Auth0 Public IP Addresses to Allow List:** If your Actions, Rules, Hooks, custom database scripts, or Webtasks call a service in your intranet or behind another firewall, be sure to add the Auth0 public IP addresses to the Allow List. This lets requests from those IP addresses through. +* **Run Tenant Configuration Checks:** The Auth0 Support Center provides a configuration checker tool. Run the configuration checker periodically during development and again before you launch. To run the configuration check, go to Auth0 Support Center > Tenants, select the gear icon, and choose Run Production Check. +* **Contextual bypass for Multi-Factor Authentication (MFA):** As a recommended best practice, use of allowRememberBrowser or context.authentication should be the only options considered for contextual bypass when using out-of-box MFA. +* **Passwordless Connections:** Enable passwordless login using email or SMS for improved user experience. +* **Custom Domains:** Use a custom domain (example: auth.yourcompany.com) for branding and consistent user experience. +* **Regularly Review User Accounts and Roles:** Remove inactive accounts and update permissions as needed. +* **Software Updates:** Keep your Auth0 SDKs and libraries up to date to ensure you benefit from the latest security patches and features. +* **Continuous Monitoring:** Continuously monitor authentication logs and respond to any anomalies or security incidents. +* **Always use HTTPS:** Always use HTTPS, not HTTP, when making calls to external services or when executing redirect as part of your rule implementation. +* **Check if an email is verified:** Check to see if a user's email address is verified in a rule +* **Check for exact string matches:** For rules that determine access control based on a particular string, such as an email domain, check for an exact string match instead of checking for a substring match. + +By following these best practices, you can build secure, scalable, and maintainable applications with Auth0. \ No newline at end of file diff --git a/.cursor/rules/autogen.mdc b/.cursor/rules/autogen.mdc new file mode 100644 index 000000000..476d5a57e --- /dev/null +++ b/.cursor/rules/autogen.mdc @@ -0,0 +1,166 @@ +--- +description: Provides coding standards, best practices, and guidance specific to the AutoGen library, covering aspects from code structure to security and performance. +globs: *.py +--- +- Best practice 1: Employ Modular Design. Utilize AutoGen's multi-agent framework to create modular agents with specific roles. This enhances code reusability and simplifies the orchestration of complex workflows. Example: Create separate agents for data fetching, code execution, and response formatting. +- Best practice 2: Define Customizable Workflows. Leverage AutoGen's capabilities to define customizable workflows that integrate LLMs, human inputs, and tools. This allows for flexible agent interactions and enhances the performance of LLM applications. Implement a workflow manager to define and control agent interactions. +- Best practice 3: Ensure Code Quality and Consistency. Implement automatic code formatting and linting features provided by AutoGen to ensure consistent coding styles and practices. This reduces human error in code generation and maintains high code quality. Integrate linters like `pylint` or `flake8` in your CI/CD pipeline. +- Best practice 4: Integrate Human Feedback. Incorporate a Human Proxy Agent to facilitate human feedback in the workflow, ensuring that the AI systems remain aligned with user expectations and can adapt based on real-world input. Use AutoGen's `HumanProxyAgent` to gather and incorporate human feedback in the loop. +- Best practice 5: Use UV for dependency management for faster and more reliable installations. +- Best practice 6: Always use the latest stable version of Python (currently 3.12) for better performance and security. +- Best practice 7: Prefer using classes over standalone functions to encapsulate state and behavior, particularly for agents and tools. +- Best practice 8: Separate configuration from code using environment variables and configuration files to allow easy modification without altering the code base + +## Library Information: +- Name: autogen +- Tags: ai, ml, llm, python, agent-framework, multi-agent + +## Comprehensive Guide to AutoGen Best Practices + +### 1. Code Organization and Structure: + - **Directory Structure Best Practices:** + - `src/`: Contains the core AutoGen application code. + - `src/agents/`: Holds individual agent implementations (e.g., `code_writer_agent.py`, `data_analyst_agent.py`). + - `src/workflows/`: Defines workflows or agent interaction patterns (e.g., `code_generation_workflow.py`). + - `src/tools/`: Contains custom tools and utilities used by agents (e.g., `web_scraper.py`, `api_client.py`). + - `config/`: Stores configuration files (e.g., `agent_config.yaml`, `llm_config.json`). + - `tests/`: Contains unit and integration tests. + - `docs/`: Holds documentation. + - `examples/`: Example scripts and notebooks showcasing AutoGen's capabilities. + - **File Naming Conventions:** + - Use descriptive names with snake_case (e.g., `code_executor_agent.py`, `data_processing_tool.py`). + - Configuration files should use `.yaml` or `.json` extensions (e.g., `agent_config.yaml`). + - Test files should be named `test_.py` (e.g., `test_code_executor.py`). + - **Module Organization Best Practices:** + - Group related functionalities into modules (e.g., `agents`, `workflows`, `tools`). + - Use `__init__.py` files to define packages and control imports. + - Implement clear and concise module-level documentation. + - **Component Architecture Recommendations:** + - Separate agents, tools, and workflows into distinct components. + - Use interfaces or abstract base classes to define contracts between components. + - Implement a central registry or dependency injection mechanism to manage component dependencies. + - **Code Splitting Strategies:** + - Split large agent implementations into smaller, more manageable classes or functions. + - Use lazy loading or dynamic imports to load components only when needed. + - Implement a plugin system to allow external components to be added to the application. + +### 2. Common Patterns and Anti-patterns: + - **Design Patterns:** + - **Factory Pattern:** Create agents and tools dynamically based on configuration or user input. + - **Strategy Pattern:** Implement different agent behaviors or tool execution strategies based on the current context. + - **Observer Pattern:** Allow agents to subscribe to events or notifications from other agents or tools. + - **Recommended Approaches for Common Tasks:** + - **Code Generation:** Use AutoGen's code generation capabilities to automate boilerplate code creation or data transformation tasks. Example: Automatically generate data validation functions based on schema definitions. + - **Data Analysis:** Implement agents that can analyze data, generate visualizations, and extract insights. Example: Create an agent that analyzes website traffic data and identifies potential areas for improvement. + - **Workflow Orchestration:** Define workflows that automate complex multi-step tasks involving multiple agents and tools. Example: Implement a workflow that automatically generates, tests, and deploys code changes. + - **Anti-patterns and Code Smells:** + - **God Agent:** Avoid creating a single agent that performs too many tasks. Decompose complex tasks into smaller, more manageable agents. + - **Tight Coupling:** Minimize dependencies between agents and tools. Use interfaces or abstract base classes to decouple components. + - **Code Duplication:** Extract common functionalities into reusable functions or classes. + - **State Management Best Practices:** + - Use a central state management system to store and manage the application's state. + - Implement versioning or snapshotting to track state changes over time. + - Use immutable data structures to prevent unintended state modifications. + - **Error Handling Patterns:** + - Implement robust error handling mechanisms to catch and handle exceptions. + - Use logging to record errors and debug issues. + - Implement retry logic to handle transient errors. + +### 3. Performance Considerations: + - **Optimization Techniques:** + - **Caching:** Cache frequently accessed data to reduce latency and improve throughput. + - **Parallelization:** Use asynchronous programming or multithreading to parallelize tasks and improve performance. + - **Code Optimization:** Profile and optimize code to reduce execution time. + - **Memory Management:** + - Use generators or iterators to process large datasets in chunks. + - Release resources as soon as they are no longer needed. + - Avoid creating unnecessary copies of data. + - **Bundle Size Optimization (if applicable):** N/A for a Python Library, but if creating web apps with autogen, use tree-shaking to remove unused code and minimize bundle size. + - **Lazy Loading:** Load modules and components only when needed to reduce startup time and memory usage. + +### 4. Security Best Practices: + - **Common Vulnerabilities:** + - **Prompt Injection:** Sanitize user inputs to prevent malicious code from being injected into agent prompts. + - **Unauthorized Access:** Implement access controls to restrict access to sensitive data and functionalities. + - **Data Leaks:** Protect sensitive data from being exposed in logs or error messages. + - **Input Validation:** + - Validate all user inputs to ensure that they conform to the expected format and range. + - Use allow lists to restrict the types of inputs that are allowed. + - Sanitize inputs to remove potentially harmful characters or code. + - **Authentication and Authorization:** + - Implement authentication mechanisms to verify the identity of users or agents. + - Use authorization policies to control access to resources and functionalities. + - Use role-based access control (RBAC) to simplify access management. + - **Data Protection:** + - Encrypt sensitive data at rest and in transit. + - Use secure storage mechanisms to protect data from unauthorized access. + - Implement data masking or anonymization techniques to protect sensitive data in logs or error messages. + - **Secure API Communication:** + - Use HTTPS to encrypt API communication. + - Authenticate and authorize API requests. + - Validate API responses to prevent data tampering. + +### 5. Testing Approaches: + - **Unit Testing:** + - Write unit tests for individual agents, tools, and workflows. + - Use mocking or stubbing to isolate components during testing. + - Test different scenarios and edge cases. + - **Integration Testing:** + - Write integration tests to verify the interaction between different components. + - Test the integration with external systems or APIs. + - Use test data that is representative of real-world data. + - **End-to-End Testing:** + - Write end-to-end tests to verify the overall functionality of the application. + - Simulate user interactions and verify that the application behaves as expected. + - Use automated testing frameworks to automate the testing process. + - **Test Organization:** + - Organize tests into separate directories or files based on the component being tested. + - Use descriptive names for test functions or classes. + - Follow a consistent testing convention. + - **Mocking and Stubbing:** + - Use mocking to replace external dependencies with mock objects. + - Use stubbing to replace complex functionalities with simple stubs. + - Use mocking frameworks to simplify the mocking process. + +### 6. Common Pitfalls and Gotchas: + - **Frequent Mistakes:** + - Not properly configuring LLM parameters (e.g., temperature, max tokens). + - Overly complex prompts that confuse the LLM. + - Ignoring error messages and logs. + - Not validating user inputs. + - **Edge Cases:** + - Handling unexpected LLM responses. + - Dealing with rate limits or API outages. + - Handling large datasets or complex workflows. + - **Version-Specific Issues:** + - Be aware of breaking changes in AutoGen updates. + - Check the release notes for known issues or workarounds. + - **Compatibility Concerns:** + - Ensure that AutoGen is compatible with other libraries or frameworks being used in the project. + - Check for conflicts between different versions of AutoGen. + - **Debugging Strategies:** + - Use logging to track the execution flow and identify errors. + - Use a debugger to step through the code and inspect variables. + - Use print statements to output intermediate results. + +### 7. Tooling and Environment: + - **Recommended Development Tools:** + - Python IDE (e.g., VS Code, PyCharm). + - Version control system (e.g., Git). + - Dependency management tool (e.g., pip, uv). + - **Build Configuration:** + - Use a build system to automate the build process. + - Configure the build system to run linters and tests. + - Use a virtual environment to isolate dependencies. + - **Linting and Formatting:** + - Use linters to enforce coding style guidelines. + - Use formatters to automatically format code. + - Integrate linters and formatters into the development workflow. + - **Deployment:** + - Use a deployment platform to deploy the application to a production environment. + - Configure the deployment platform to automatically scale the application. + - Use a monitoring system to track the application's performance. + - **CI/CD Integration:** + - Use a CI/CD pipeline to automate the build, test, and deployment process. + - Integrate the CI/CD pipeline with the version control system. + - Use automated testing to ensure code quality. \ No newline at end of file diff --git a/.cursor/rules/aws-amplify.mdc b/.cursor/rules/aws-amplify.mdc new file mode 100644 index 000000000..8c642c4b5 --- /dev/null +++ b/.cursor/rules/aws-amplify.mdc @@ -0,0 +1,286 @@ +--- +description: This rule provides best practices and coding standards for aws-amplify projects, covering code organization, common patterns, performance, security, testing, pitfalls, and tooling. It aims to guide developers in building robust, scalable, and maintainable aws-amplify applications. +globs: *.js,*.jsx,*.ts,*.tsx +--- +- # aws-amplify Best Practices + + This document outlines best practices and coding standards for developing applications using the aws-amplify library. It aims to guide developers in creating robust, scalable, and maintainable applications. + +- ## 1. Code Organization and Structure + + - ### Directory Structure + + - **src/**: Contains all the application source code. + - **components/**: Reusable UI components. + - `ComponentName/`: Each component in its own directory. + - `ComponentName.tsx`: The component file. + - `ComponentName.module.css`: Component-specific styles (using CSS Modules). + - `ComponentName.test.tsx`: Unit tests for the component. + - `index.ts`: (Optional) Exports the component for easier imports. + - **pages/**: Defines the application's routes/pages (especially important for Next.js and similar frameworks). + - `index.tsx`: The home page. + - `[routeName].tsx`: Dynamic routes. + - **services/**: Contains business logic, data fetching, and interactions with aws-amplify services. + - `authService.ts`: Authentication-related functions. + - `dataService.ts`: Data fetching and manipulation (e.g., interacting with AppSync). + - `storageService.ts`: Storage-related functions (e.g., interacting with S3). + - **models/**: Defines data models used in the application. This often corresponds to your Amplify DataStore or GraphQL schema definitions. + - `Todo.ts`: Example data model for a `Todo` item. + - **utils/**: Utility functions and helpers. + - `apiClient.ts`: A wrapper around the `Amplify` object for making API requests. + - `dateUtils.ts`: Date formatting functions. + - **config/**: Application configuration files. + - `aws-exports.js`: Automatically generated aws-amplify configuration. + - **hooks/**: Custom React hooks for reusable logic. + - `useAuth.ts`: A hook for managing authentication state. + - **amplify/**: Generated directory by Amplify CLI, containing backend definitions. + - **backend/**: Code and configurations that define the AWS resources. + - `api/`: GraphQL API definitions (schema.graphql, resolvers, etc.). + - `auth/`: Authentication configuration (Cognito User Pool, Identity Pool). + - `storage/`: Storage configurations (S3 bucket). + - `function/`: Lambda functions. + + - ### File Naming Conventions + + - Components: `ComponentName.tsx` + - Styles: `ComponentName.module.css` + - Services: `serviceName.ts` + - Models: `ModelName.ts` + - Hooks: `useHookName.ts` + - Tests: `FileName.test.tsx` or `FileName.spec.tsx` + + - ### Module Organization + + - Group related functionalities into modules. + - Use clear and descriptive names for modules. + - Avoid circular dependencies between modules. + - Export only necessary functions and variables from each module. + + - ### Component Architecture + + - **Presentational Components:** Responsible for how things look. Receive data and callbacks via props. + - **Container Components:** Concerned with how things work. Fetch data, manage state, and pass data/callbacks to presentational components. + - Use composition over inheritance. + - Keep components small and focused on a single responsibility. + - Use React Hooks for managing state and side effects. + + - ### Code Splitting + + - Use dynamic imports (`React.lazy`) for components that are not immediately needed. + - Split routes into separate bundles. + - Leverage webpack, Parcel, or other bundlers to automatically split code. + +- ## 2. Common Patterns and Anti-patterns + + - ### Design Patterns + + - **Provider Pattern:** Use React Context to provide aws-amplify client and authentication state to the entire application. + - **Hook Composition:** Create custom hooks that encapsulate aws-amplify logic for reusability. + - **Repository Pattern:** Abstract data access logic behind a repository interface. + + - ### Recommended Approaches + + - **Authentication:** Use `Amplify.configure` to initialize Amplify with the configuration from `aws-exports.js`. + - Implement sign-in, sign-up, sign-out using `Auth` API. + - Utilize Higher Order Components or custom Hooks to protect routes based on authentication status. + - **Data Fetching:** Use `DataStore` or GraphQL APIs for data fetching. Consider using pagination for large datasets. + - **Storage:** Use `Storage` API for uploading and retrieving files from S3. + + - ### Anti-patterns and Code Smells + + - **Tight Coupling:** Avoid directly using `Amplify` APIs in components. Abstract them into services or hooks. + - **Over-fetching:** Fetch only the necessary data from APIs. Use GraphQL fragments to specify required fields. + - **Ignoring Errors:** Always handle errors when interacting with aws-amplify services. + - **Storing Sensitive Data in Client:** Never store API keys or secrets in the client-side code. Use Lambda functions with environment variables. + - **Over-reliance on Amplify UI Components:** While useful for rapid prototyping, customize or replace Amplify UI components for complex designs and accessibility requirements. + + - ### State Management + + - **Component State:** Use `useState` or `useReducer` for local component state. + - **Context API:** Use React Context for application-wide state (e.g., authentication status, user data). + - **Redux/Zustand/Recoil:** Consider using a state management library for complex applications with global state dependencies. + - **Amplify DataStore:** Excellent solution for offline-first applications needing real-time data synchronization with the cloud. Use its built-in mechanisms for local persistence and conflict resolution. + + - ### Error Handling + + - Use `try...catch` blocks to handle errors when calling aws-amplify APIs. + - Display user-friendly error messages. + - Log errors for debugging purposes. + - Implement retry mechanisms for transient errors. + +- ## 3. Performance Considerations + + - ### Optimization Techniques + + - **Caching:** Cache API responses to reduce network requests. Use `Cache` API for storing data in the browser. + - **Lazy Loading:** Load components and data only when needed. + - **Debouncing and Throttling:** Limit the rate at which functions are executed (e.g., event handlers). + - **Memoization:** Cache the results of expensive function calls using `useMemo` or `React.memo`. + - **Image Optimization:** Optimize images before uploading them to S3. Use responsive images with different sizes for different devices. + + - ### Memory Management + + - Avoid memory leaks by cleaning up subscriptions and timers in `useEffect` hooks. + - Release unused resources. + + - ### Rendering Optimization + + - Use `React.memo` to prevent unnecessary re-renders of components. + - Implement shouldComponentUpdate (if using class components) for fine-grained control over rendering. + - Virtualize long lists to render only visible items. + + - ### Bundle Size Optimization + + - Use code splitting to reduce the initial bundle size. + - Remove unused code and dependencies. + - Minify and compress code. + - Analyze bundle size with tools like `webpack-bundle-analyzer` or `Source Map Explorer`. + + - ### Lazy Loading + + - Use `React.lazy` and `Suspense` for lazy-loading components. + - Implement lazy loading for images and other assets. + +- ## 4. Security Best Practices + + - ### Common Vulnerabilities + + - **Cross-Site Scripting (XSS):** Sanitize user input to prevent XSS attacks. + - **Cross-Site Request Forgery (CSRF):** Implement CSRF protection mechanisms. + - **Injection Attacks:** Protect against SQL injection, NoSQL injection, and command injection attacks. + - **Authentication and Authorization Flaws:** Secure authentication and authorization mechanisms. + + - ### Input Validation + + - Validate all user input on both the client and server sides. + - Use strong validation rules to prevent invalid data from being stored in the database. + + - ### Authentication and Authorization + + - Use Amazon Cognito for authentication and authorization. + - Implement role-based access control (RBAC) to restrict access to resources. + - Secure API endpoints with authentication and authorization checks. + - Follow the principle of least privilege when granting permissions. + + - ### Data Protection + + - Encrypt sensitive data at rest and in transit. + - Use HTTPS for all communication. + - Implement data masking and redaction techniques to protect sensitive data from unauthorized access. + - Regularly back up data. + + - ### Secure API Communication + + - Use HTTPS for all API requests. + - Validate API responses. + - Implement rate limiting to prevent abuse. + +- ## 5. Testing Approaches + + - ### Unit Testing + + - Test individual components and functions in isolation. + - Use mocking and stubbing to isolate components from external dependencies. + - Write tests that cover all possible scenarios and edge cases. + - Use testing frameworks like Jest and React Testing Library. + + - ### Integration Testing + + - Test the interaction between different components and modules. + - Test the integration with aws-amplify services. + - Use integration testing frameworks like Cypress or Playwright. + + - ### End-to-End Testing + + - Test the entire application flow from start to finish. + - Simulate real user interactions. + - Use end-to-end testing frameworks like Cypress or Selenium. + + - ### Test Organization + + - Keep tests close to the code they are testing. + - Use descriptive names for tests. + - Organize tests into logical groups. + + - ### Mocking and Stubbing + + - Use mocking libraries like Jest to mock aws-amplify APIs and services. + - Use stubbing to replace complex dependencies with simple implementations for testing purposes. + +- ## 6. Common Pitfalls and Gotchas + + - ### Frequent Mistakes + + - Misconfiguring aws-amplify. + - Ignoring error messages. + - Not handling edge cases. + - Over-complicating code. + - Not testing thoroughly. + + - ### Edge Cases + + - Handling offline scenarios. + - Dealing with slow network connections. + - Handling unexpected API responses. + + - ### Version-Specific Issues + + - Be aware of breaking changes between aws-amplify versions. + - Consult the aws-amplify documentation and release notes for migration guides. + + - ### Compatibility Concerns + + - Ensure compatibility between aws-amplify and other libraries and frameworks used in the project. + - Check for known compatibility issues and workarounds. + + - ### Debugging Strategies + + - Use browser developer tools for debugging client-side code. + - Use CloudWatch logs for debugging server-side code (Lambda functions). + - Use the aws-amplify CLI to diagnose configuration issues. + +- ## 7. Tooling and Environment + + - ### Recommended Tools + + - Visual Studio Code (VS Code) or other preferred IDE + - aws-amplify CLI + - Node.js and npm or yarn + - Git for version control + - Jest and React Testing Library for unit testing + - Cypress or Playwright for end-to-end testing + + - ### Build Configuration + + - Use a build tool like webpack, Parcel, or esbuild. + - Configure the build tool to optimize code for production. + - Use environment variables for configuration settings. + + - ### Linting and Formatting + + - Use ESLint and Prettier to enforce code style and prevent errors. + - Configure ESLint and Prettier to automatically format code on save. + + - ### Deployment + + - Use aws-amplify Console for deploying the application. + - Configure CI/CD pipelines to automatically deploy changes to production. + - Use environment variables for configuration settings. + + - ### CI/CD Integration + + - Integrate with CI/CD tools like GitHub Actions, CircleCI, or Travis CI. + - Automate the build, test, and deployment processes. + - Use environment variables for configuration settings. + +- **Code-First Approach with TypeScript:** Embrace the code-first approach with AWS Amplify Gen 2, using TypeScript for both frontend and backend development to improve developer productivity and catch errors early. Leverage AWS CDK for defining infrastructure as code. + +- **Single Responsibility Principle:** Adhere to the Single Responsibility Principle for AWS Lambda functions to improve code maintainability and testability. + +- **Avoid Function Chaining:** Reduce complexity and improve debugging by avoiding function chaining in Lambda functions. If code reuse is needed, utilize separate TypeScript files for shared logic. + +- **Start with Templates:** Accelerate development by using pre-built templates instead of starting projects from scratch. + +- **Utilize Cloud Sandboxes:** Leverage per-developer cloud sandbox environments for isolated testing and iteration without interfering with other team members. + +- **Take advantage of Amplify's opinionated nature** When you read the docs, you will start to understand how opinionated Amplify is. Whether you choose Amplify or not should include this aspect of its design, and you shouldn't choose this tool if it doesn't align with the direction your team is taking with infrastructure. It is best used as 'Glue' for an All-AWS roadmap. \ No newline at end of file diff --git a/.cursor/rules/aws-cli.mdc b/.cursor/rules/aws-cli.mdc new file mode 100644 index 000000000..1d33952c3 --- /dev/null +++ b/.cursor/rules/aws-cli.mdc @@ -0,0 +1,349 @@ +--- +description: This rule provides comprehensive guidance on best practices for developing and managing AWS resources using the AWS CLI, covering code organization, security, performance, and testing. +globs: *.sh,*.py,*.yml,*.yaml,*.tf,*.json,*.md +--- +# AWS CLI Best Practices and Coding Standards + +This document provides guidelines and recommendations for developing and managing AWS resources using the AWS CLI. Following these practices enhances code quality, security, and operational efficiency. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure + +A well-organized directory structure promotes maintainability and readability. + + +project-root/ +├── scripts/ # Shell scripts for common AWS CLI operations +│ ├── create-s3-bucket.sh +│ ├── deploy-lambda.sh +│ └── ... +├── modules/ # Reusable modules (e.g., Python scripts) +│ ├── s3_utils.py # S3 utility functions +│ ├── iam_utils.py # IAM utility functions +│ └── ... +├── config/ # Configuration files +│ ├── aws_config.yml # AWS CLI configuration +│ └── ... +├── terraform/ # Infrastructure as Code (IaC) using Terraform (optional) +│ ├── main.tf +│ ├── variables.tf +│ └── ... +├── tests/ # Test scripts +│ ├── test_s3_utils.py +│ └── ... +├── README.md # Project documentation +└── .gitignore # Git ignore file + + +### 1.2. File Naming Conventions + +* **Scripts:** Use descriptive names (e.g., `create-ec2-instance.sh`). Suffix with `.sh` for shell scripts. +* **Modules:** Use descriptive names (e.g., `s3_utils.py`). Suffix with `.py` for Python modules. +* **Configuration:** Use `.yml` or `.yaml` for YAML configuration files (e.g., `aws_config.yml`). +* **Terraform:** Follow Terraform conventions (`main.tf`, `variables.tf`). +* **Tests:** Prefix test files with `test_` (e.g., `test_s3_utils.py`). + +### 1.3. Module Organization + +* **Purpose-Driven Modules:** Group related functionalities into modules (e.g., `s3_utils.py` for S3 operations). +* **Avoid Circular Dependencies:** Design modules to minimize inter-dependencies. +* **Clear Interfaces:** Define clear function signatures and API contracts for each module. +* **Documentation:** Include docstrings to explain the purpose and usage of each function within a module. + +### 1.4. Component Architecture + +For larger projects, consider a component-based architecture. Each component encapsulates specific AWS-related functionalities. This approach promotes modularity, testability, and reusability. + +Example: + + +components/ +├── s3/ +│ ├── s3_component.py +│ ├── ... +├── iam/ +│ ├── iam_component.py +│ ├── ... +└── ... + + +### 1.5. Code Splitting + +* **Break Down Complex Scripts:** Split large, complex scripts into smaller, more manageable functions or modules. +* **Configuration-Driven Scripts:** Use configuration files to drive script behavior, making them more flexible and reusable. +* **Parameterization:** Parameterize scripts to accept input variables, reducing hardcoding and improving adaptability. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +* **Facade:** Create a facade to simplify complex AWS CLI operations. +* **Adapter:** Use adapters to translate between different data formats (e.g., JSON and YAML). +* **Strategy:** Implement different strategies for error handling or retry mechanisms. +* **Command Pattern:** Encapsulate AWS CLI commands as objects, facilitating command queuing, logging, and undo operations. + +### 2.2. Recommended Approaches + +* **Configuration Management:** Use environment variables, configuration files, or AWS Systems Manager Parameter Store to manage AWS credentials and configurations. +* **Idempotent Scripts:** Design scripts to be idempotent, meaning that running them multiple times has the same effect as running them once. +* **Logging:** Implement comprehensive logging to track script execution and identify issues. +* **Error Handling:** Use appropriate error handling mechanisms to gracefully handle exceptions. +* **Progress Indicators:** Add progress indicators to provide feedback during long-running operations. + +### 2.3. Anti-patterns + +* **Hardcoding Credentials:** Avoid hardcoding AWS credentials directly in scripts. +* **Lack of Error Handling:** Failing to handle exceptions can lead to unexpected script termination. +* **Overly Complex Scripts:** Overly complex scripts are difficult to maintain and debug. +* **Ignoring Security Best Practices:** Neglecting security best practices can lead to vulnerabilities. +* **Not using IaC:** Managing infrastructure manually instead of using Infrastructure as Code (IaC) tools such as CloudFormation or Terraform. + +### 2.4. State Management + +* **AWS Systems Manager Parameter Store:** Use Parameter Store to store and manage configuration data and secrets. +* **DynamoDB:** Use DynamoDB to store application state data. +* **S3:** Store state data in S3 buckets, ensuring proper access control. +* **CloudFormation Stack Outputs:** If using CloudFormation, use stack outputs to manage state information. +* **Terraform State:** If using Terraform, properly manage the Terraform state file (e.g., using S3 backend with DynamoDB lock). Never commit the state file to version control. + +### 2.5. Error Handling + +* **Try-Except Blocks:** Use `try-except` blocks in Python to handle exceptions. +* **Error Codes:** Check for specific error codes returned by AWS CLI commands. +* **Retry Mechanisms:** Implement retry mechanisms with exponential backoff for transient errors. +* **Alerting:** Set up alerting to notify administrators of critical errors. +* **Consistent Error Messages:** Provide clear, informative error messages that help in debugging. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +* **Parallel Execution:** Use parallel processing to execute multiple AWS CLI commands concurrently. +* **Caching:** Cache frequently accessed data to reduce API calls. +* **Pagination:** Use pagination to retrieve large datasets in smaller chunks. +* **Filtering:** Filter data on the server-side to reduce the amount of data transferred. +* **Asynchronous Operations:** Utilize asynchronous operations when possible to avoid blocking the main thread of execution. + +### 3.2. Memory Management + +* **Large Datasets:** Avoid loading large datasets into memory at once. Use iterators or generators to process data in chunks. +* **Resource Cleanup:** Ensure that resources are properly released after use to prevent memory leaks. +* **Efficient Data Structures:** Use efficient data structures to minimize memory consumption. + +### 3.3. Rendering Optimization (If applicable) + +N/A - The AWS CLI is not a rendering library. + +### 3.4. Bundle Size Optimization (If applicable) + +N/A - Bundle size optimization is not directly relevant to the AWS CLI itself, but it might be relevant for tools built on top of it that are distributed as standalone applications. + +### 3.5. Lazy Loading + +* **Import Modules on Demand:** Import modules only when they are needed to reduce startup time. +* **Load Configuration As Needed:** Load configuration data only when it is required. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities + +* **Credential Exposure:** Exposing AWS credentials in code or configuration files. +* **Insufficient Access Control:** Granting excessive permissions to IAM roles or users. +* **SQL Injection (if applicable):** If constructing SQL queries dynamically, protect against SQL injection. +* **Cross-Site Scripting (XSS) (if applicable):** If the CLI interacts with web interfaces, sanitize inputs to prevent XSS. +* **Command Injection:** If constructing shell commands dynamically from user inputs, sanitize inputs to prevent command injection. + +### 4.2. Input Validation + +* **Sanitize Inputs:** Sanitize all user inputs to prevent malicious code injection. +* **Validate Data Types:** Validate that input data conforms to expected data types. +* **Limit Input Lengths:** Limit the length of input strings to prevent buffer overflows. +* **Regular Expressions:** Use regular expressions to validate input patterns. +* **Whitelist Allowed Values:** If applicable, use a whitelist of allowed values rather than a blacklist. + +### 4.3. Authentication and Authorization + +* **IAM Roles:** Use IAM roles to grant permissions to AWS resources. +* **Least Privilege:** Follow the principle of least privilege when granting permissions. +* **Multi-Factor Authentication (MFA):** Enable MFA for all IAM users. +* **Credential Rotation:** Rotate AWS credentials regularly. +* **IAM Policies:** Implement granular IAM policies to control access to resources. + +### 4.4. Data Protection + +* **Encryption:** Encrypt sensitive data at rest and in transit. +* **Access Control Lists (ACLs):** Use ACLs to control access to S3 buckets and objects. +* **Data Masking:** Mask sensitive data to prevent unauthorized access. +* **Regular Backups:** Create regular backups of critical data. +* **Secure Logging:** Ensure that logs do not contain sensitive information and are stored securely. + +### 4.5. Secure API Communication + +* **HTTPS:** Use HTTPS for all API communication. +* **TLS:** Enforce TLS 1.2 or higher for secure connections. +* **API Gateway:** Use API Gateway to secure and manage API endpoints. +* **Rate Limiting:** Implement rate limiting to prevent denial-of-service attacks. +* **WAF (Web Application Firewall):** Use WAF to protect against common web exploits. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +* **Mock AWS CLI Calls:** Use mocking libraries to simulate AWS CLI calls and verify expected behavior. +* **Test Individual Functions:** Write unit tests for each function or module. +* **Boundary Conditions:** Test boundary conditions and edge cases. +* **Assertion Libraries:** Use assertion libraries to verify expected results. + +### 5.2. Integration Testing + +* **Test End-to-End Workflows:** Test complete workflows that involve multiple AWS services. +* **Use Real AWS Resources:** Use real AWS resources in a test environment. +* **Automated Test Execution:** Automate integration tests as part of the CI/CD pipeline. +* **Verify Resource Creation and Deletion:** Ensure that resources are created and deleted correctly during testing. + +### 5.3. End-to-End Testing + +* **Simulate Real-World Scenarios:** Simulate real-world scenarios to test the application's overall behavior. +* **Automated Test Scripts:** Create automated test scripts to verify functionality. +* **Monitor System Behavior:** Monitor system behavior during end-to-end tests. + +### 5.4. Test Organization + +* **Dedicated Test Directory:** Store test files in a dedicated directory (e.g., `tests/`). +* **Test Naming Conventions:** Follow consistent test naming conventions. +* **Test Suites:** Group related tests into test suites. +* **Test Reports:** Generate test reports to track test results. + +### 5.5. Mocking and Stubbing + +* **Mock AWS CLI:** Use mocking libraries to simulate AWS CLI behavior. +* **Stub External Dependencies:** Stub external dependencies to isolate components during testing. +* **Control Test Data:** Use controlled test data to ensure predictable test results. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +* **Incorrect IAM Permissions:** Granting insufficient or excessive IAM permissions. +* **Missing Configuration:** Failing to configure the AWS CLI properly. +* **Hardcoding Values:** Hardcoding values instead of using configuration files or environment variables. +* **Ignoring Error Messages:** Ignoring error messages can lead to unexpected behavior. +* **Not Using Pagination:** Retrieving incomplete datasets due to lack of pagination. + +### 6.2. Edge Cases + +* **Rate Limiting:** Handling API rate limits gracefully. +* **Service Outages:** Handling service outages or transient errors. +* **Data Consistency:** Ensuring data consistency across multiple AWS services. +* **Concurrency Issues:** Managing concurrency issues when multiple scripts access the same resources. + +### 6.3. Version-Specific Issues + +* **API Changes:** Handling API changes across different AWS CLI versions. +* **Deprecated Features:** Avoiding deprecated features. +* **Compatibility Issues:** Addressing compatibility issues between different AWS CLI versions and AWS services. + +### 6.4. Compatibility Concerns + +* **Operating Systems:** Ensuring compatibility across different operating systems (e.g., Linux, Windows, macOS). +* **Programming Languages:** Addressing compatibility issues with different programming languages. +* **Third-Party Libraries:** Managing dependencies on third-party libraries. + +### 6.5. Debugging Strategies + +* **Logging:** Use comprehensive logging to track script execution. +* **Debugging Tools:** Use debugging tools to step through code and inspect variables. +* **Error Messages:** Analyze error messages carefully to identify the root cause of issues. +* **Verbose Mode:** Use verbose mode to print detailed information about AWS CLI commands. +* **AWS CloudTrail:** Use AWS CloudTrail to track API calls and identify potential issues. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +* **AWS CLI:** The core command-line interface for interacting with AWS services. +* **Text Editor/IDE:** A suitable text editor or IDE for writing scripts (e.g., VS Code, Sublime Text, PyCharm). +* **Version Control:** Git for version control. +* **Shell:** Bash, Zsh, or PowerShell for running scripts. +* **Python (if applicable):** Python for scripting and automation. +* **jq:** A command-line JSON processor. +* **yq:** A command-line YAML processor. + +### 7.2. Build Configuration + +* **Makefile:** Use Makefiles to automate common build tasks. +* **Build Scripts:** Create build scripts to package and deploy applications. +* **Dependency Management:** Use appropriate dependency management tools (e.g., pip for Python). + +### 7.3. Linting and Formatting + +* **Shellcheck:** Use Shellcheck to lint shell scripts. +* **Pylint:** Use Pylint to lint Python code. +* **Black:** Use Black to format Python code. +* **YAML Lint:** Use a YAML linter to validate YAML files. + +### 7.4. Deployment + +* **AWS CloudFormation:** Deploy infrastructure as code using AWS CloudFormation. +* **AWS CodeDeploy:** Automate application deployments using AWS CodeDeploy. +* **AWS Lambda:** Deploy serverless functions using AWS Lambda. +* **Docker:** Containerize applications using Docker. + +### 7.5. CI/CD Integration + +* **AWS CodePipeline:** Use AWS CodePipeline to automate the CI/CD process. +* **Jenkins:** Integrate with Jenkins for continuous integration. +* **GitHub Actions:** Use GitHub Actions for CI/CD. +* **Automated Testing:** Integrate automated testing into the CI/CD pipeline. + +## Infrastructure as Code (IaC) + +* Treat your infrastructure as code using tools like AWS CloudFormation or Terraform. This allows for version control, consistent deployments, and easier management of infrastructure changes. + +## Continuous Integration/Continuous Deployment (CI/CD) + +* Implement CI/CD pipelines using AWS CodePipeline, ensuring that all code changes are automatically tested and deployed. This includes defining clear stages for building, testing, and deploying applications, and incorporating security checks throughout the pipeline. + +## Security + +* Employ least-privilege access controls using IAM, enforce multi-factor authentication (MFA), and regularly review permissions. Security should be integrated at every stage of the development and deployment process. + +## Automate tasks + +* Automate repetitive tasks to increase efficiency and reduce errors. + +## Version control + +* Ensure robust version control for all code and configuration files. + +## Prioritize testing + +* Integrate comprehensive testing at every stage to ensure code quality. + +## Maintain and adapt + +* Regularly update and adapt the pipeline to evolving requirements. + +## Monitor and analyze + +* Continuously monitor and analyze pipeline performance and outcomes. + +## IAM Best Practices: + +* Require human users to use federation with an identity provider to access AWS using temporary credentials +* Require workloads to use temporary credentials with IAM roles to access AWS +* Require multi-factor authentication (MFA) +* Update access keys when needed for use cases that require long-term credentials +* Follow best practices to protect your root user credentials +* Apply least-privilege permissions +* Get started with AWS managed policies and move toward least-privilege permissions +* Use IAM Access Analyzer to generate least-privilege policies based on access activity +* Regularly review and remove unused users, roles, permissions, policies, and credentials +* Use conditions in IAM policies to further restrict access +* Verify public and cross-account access to resources with IAM Access Analyzer +* Use IAM Access Analyzer to validate your IAM policies to ensure secure and functional permissions +* Establish permissions guardrails across multiple accounts +* Use permissions boundaries to delegate permissions management within an account + +By adhering to these best practices, you can develop robust, secure, and efficient AWS CLI applications. \ No newline at end of file diff --git a/.cursor/rules/aws-dynamodb.mdc b/.cursor/rules/aws-dynamodb.mdc new file mode 100644 index 000000000..f5c33c623 --- /dev/null +++ b/.cursor/rules/aws-dynamodb.mdc @@ -0,0 +1,182 @@ +--- +description: This rule provides best practices and coding standards for developing applications using AWS DynamoDB. It covers aspects like schema design, performance optimization, security, and testing. +globs: *.js,*.ts,*.jsx,*.tsx,*.py +--- +- **Minimize the number of tables; prefer single-table design.** Having fewer tables keeps things more scalable, requires less permissions management, and reduces overhead for your DynamoDB application. Consider using a single table with appropriate use of primary and secondary indexes. +- **Identify query patterns before schema design.** Understand the application's access patterns before designing the DynamoDB schema. Knowing how the data will be queried is crucial for optimizing performance and cost. +- **Use sort order for related items and distribute queries to avoid hot spots.** Group related items together by utilizing sort keys and design data keys to distribute traffic evenly across partitions. +- **Prefer queries over scans for efficiency.** DynamoDB queries are more efficient and less costly than scan operations. Always aim to use queries that filter based on partition and sort keys. +- **Use caching to reduce read/write costs.** Implement caching strategies, such as AWS ElastiCache or DynamoDB Accelerator (DAX), to minimize DynamoDB read/write costs. Use appropriate invalidation logic. +- **Validate data integrity in the application layer.** Since DynamoDB doesn't enforce relationship data or integrity constraints, it's essential to perform strict data validations in the application or business layer. + +## 1. Code Organization and Structure + +- **Directory Structure Best Practices:** + - `src/`: Contains the application's source code. + - `src/models/`: Defines data models representing DynamoDB entities. + - `src/services/`: Implements business logic and interacts with DynamoDB. + - `src/repositories/`: Handles DynamoDB data access operations. + - `src/utils/`: Contains utility functions (e.g., data validation, error handling). + - `tests/`: Includes unit, integration, and end-to-end tests. + - `config/`: Stores configuration files (e.g., DynamoDB table names, AWS region). +- **File Naming Conventions:** + - Use descriptive names for files and directories. + - Model files: `user.model.js` or `user.model.ts` + - Service files: `user.service.js` or `user.service.ts` + - Repository files: `user.repository.js` or `user.repository.ts` + - Test files: `user.service.test.js` or `user.service.test.ts` +- **Module Organization Best Practices:** + - Encapsulate DynamoDB interactions within dedicated modules (e.g., repositories). + - Use dependency injection to manage dependencies (e.g., DynamoDB client). + - Follow the single responsibility principle for modules. +- **Component Architecture Recommendations:** + - Adopt a layered architecture (e.g., presentation, application, domain, infrastructure). + - Decouple components to improve maintainability and testability. + - Use interfaces to define contracts between components. +- **Code Splitting Strategies:** + - Bundle only the necessary aws-sdk/client-dynamodb code by using tree shaking and only importing the necessary modules. + - Implement lazy loading for less frequently used features. + +## 2. Common Patterns and Anti-patterns + +- **Design Patterns:** + - **Repository Pattern:** Abstract DynamoDB data access logic. + - **Data Mapper Pattern:** Transform data between application and DynamoDB formats. + - **Single Table Design Pattern:** Minimize tables by using composite keys and attributes to represent different entity types and relationships. + - **Composite Keys Pattern:** Combine attributes to create partition and sort keys. +- **Recommended Approaches for Common Tasks:** + - **Creating Items:** Use the `PutItem` operation. + - **Reading Items:** Use the `GetItem` operation for single items and `Query` for sets. + - **Updating Items:** Use the `UpdateItem` operation for granular updates. + - **Deleting Items:** Use the `DeleteItem` operation. + - **Querying Data:** Utilize indexes (GSI/LSI) to optimize query performance. Use projection expressions to return only required attributes. + - **Scanning Data (Avoid when possible):** If scanning is unavoidable, use `Limit` and `ExclusiveStartKey` for pagination and consider parallel scans for large datasets. +- **Anti-patterns and Code Smells:** + - **Over-fetching Data:** Avoid retrieving unnecessary attributes. + - **Excessive Scanning:** Design schemas and queries to minimize scans. + - **Hardcoding Table Names:** Use configuration files or environment variables. + - **Ignoring Error Handling:** Implement robust error handling with retries and logging. + - **Insufficient Data Validation:** Validate data before writing to DynamoDB. + - **Hot Partitioning:** Design partitions to distribute data evenly and prevent overload of specific partitions. +- **State Management Best Practices:** + - Use DynamoDB to persist application state (e.g., user sessions, feature flags). + - Implement atomic counters and conditional updates to manage concurrent access. +- **Error Handling Patterns:** + - Use try-catch blocks to handle DynamoDB errors. + - Implement retry mechanisms for transient errors (e.g., throttling). + - Log errors with sufficient context for debugging. + - Use custom exception classes to categorize DynamoDB errors. + +## 3. Performance Considerations + +- **Optimization Techniques:** + - **Minimize Read Capacity Units (RCUs) and Write Capacity Units (WCUs):** Optimize queries and writes to consume fewer RCUs and WCUs. Use projection expressions to return only the attributes you need. + - **Use Batch Operations:** Use `BatchGetItem` and `BatchWriteItem` for bulk operations. + - **Optimize Data Sizes:** Keep item sizes small to reduce storage costs and improve performance. If items exceed the size limit, consider storing larger attributes in S3 and referencing them in DynamoDB. + - **Use Parallel Scans (with caution):** Use parallel scans to speed up full table scans, but be aware of the increased RCUs consumption. + - **Optimize Index Usage:** Use indexes effectively to support query patterns. Be mindful of the cost implications of GSI writes. + - **Leverage DynamoDB Accelerator (DAX):** Use DAX for in-memory caching to reduce latency and RCU consumption. +- **Memory Management Considerations:** + - Avoid loading large datasets into memory. + - Use pagination to process data in chunks. + - Release resources promptly after use. +- **Bundle Size Optimization Strategies:** + - Use code splitting to reduce the initial bundle size. + - Remove unused dependencies. + - Optimize image sizes (if applicable). +- **Lazy Loading Strategies:** + - Load data on demand when needed. + - Use placeholders for content that is not immediately visible. + +## 4. Security Best Practices + +- **Common Vulnerabilities and Prevention:** + - **Injection Attacks:** Prevent NoSQL injection by validating and sanitizing user inputs. + - **Unauthorized Access:** Implement proper authentication and authorization mechanisms. + - **Data Exposure:** Encrypt sensitive data at rest and in transit. +- **Input Validation Best Practices:** + - Validate data types, formats, and ranges. + - Sanitize inputs to prevent injection attacks. + - Use input validation libraries (e.g., Joi, Validator.js). +- **Authentication and Authorization Patterns:** + - Use AWS Identity and Access Management (IAM) roles and policies to control access to DynamoDB resources. + - Implement fine-grained access control using conditional IAM policies. + - Consider using AWS Cognito for user authentication and authorization. +- **Data Protection Strategies:** + - Enable encryption at rest and in transit for DynamoDB tables. + - Use AWS Key Management Service (KMS) to manage encryption keys. + - Implement data masking and tokenization for sensitive data. + - Comply with data privacy regulations (e.g., GDPR, CCPA). +- **Secure API Communication:** + - Use HTTPS for all API communication. + - Implement API authentication and authorization. + - Protect against cross-site scripting (XSS) and cross-site request forgery (CSRF) attacks. + +## 5. Testing Approaches + +- **Unit Testing Strategies:** + - Mock DynamoDB client using libraries like `aws-sdk-mock` or `jest-mock-extended`. + - Test individual functions and modules in isolation. + - Verify that functions correctly interact with the mocked DynamoDB client. +- **Integration Testing Approaches:** + - Test the integration between different modules and components. + - Use a local DynamoDB instance (e.g., DynamoDB Local, Docker image) for integration tests. + - Verify that data is correctly written to and read from DynamoDB. +- **End-to-end Testing Recommendations:** + - Test the entire application flow from the user interface to DynamoDB. + - Use a staging environment that mirrors the production environment. + - Verify that the application meets all functional and non-functional requirements. +- **Test Organization Best Practices:** + - Organize tests by module or component. + - Use descriptive names for test cases. + - Follow the AAA (Arrange, Act, Assert) pattern. +- **Mocking and Stubbing Techniques:** + - Use mocking libraries to create mock objects for DynamoDB client. + - Use stubbing to replace real dependencies with controlled test values. + - Verify that mocked methods are called with the expected parameters. + +## 6. Common Pitfalls and Gotchas + +- **Frequent Mistakes:** + - Forgetting to handle pagination when querying large datasets. + - Not understanding DynamoDB capacity units (RCUs and WCUs) and throttling. + - Using inefficient query patterns. + - Neglecting to implement proper error handling. + - Designing schemas that lead to hot partitions. +- **Edge Cases:** + - Handling large item sizes exceeding DynamoDB's limits. + - Dealing with eventual consistency when reading data. + - Managing concurrent updates to the same item. +- **Version-Specific Issues:** + - Be aware of breaking changes in aws-sdk/client-dynamodb versions. + - Consult the AWS documentation and release notes for migration guides. +- **Compatibility Concerns:** + - Ensure compatibility between aws-sdk/client-dynamodb and other libraries. +- **Debugging Strategies:** + - Use AWS CloudWatch Logs to monitor DynamoDB operations. + - Enable DynamoDB Streams to capture changes to DynamoDB tables. + - Use the AWS X-Ray service for distributed tracing. + +## 7. Tooling and Environment + +- **Recommended Development Tools:** + - AWS CLI + - AWS SDK for Javascript/Typescript + - DynamoDB Local (for local development) + - NoSQL Workbench for DynamoDB (for data modeling). +- **Build Configuration Best Practices:** + - Use a build tool (e.g., webpack, Parcel, esbuild) to bundle and optimize code. + - Configure environment variables for DynamoDB table names, AWS region, and credentials. + - Use `.env` files or AWS Systems Manager Parameter Store to manage configuration data. +- **Linting and Formatting Recommendations:** + - Use ESLint or TSLint for linting. + - Use Prettier for code formatting. + - Configure linting and formatting rules to enforce consistent code style. +- **Deployment Best Practices:** + - Use infrastructure-as-code tools (e.g., AWS CloudFormation, AWS CDK, Terraform) to provision DynamoDB resources. + - Implement blue/green deployments or canary deployments to minimize downtime. + - Automate deployments using CI/CD pipelines. +- **CI/CD Integration Strategies:** + - Use CI/CD tools (e.g., Jenkins, GitLab CI, CircleCI, AWS CodePipeline) to automate the build, test, and deployment process. + - Run unit and integration tests in the CI/CD pipeline. + - Deploy code to staging environments for testing and validation before deploying to production. \ No newline at end of file diff --git a/.cursor/rules/aws-ecs.mdc b/.cursor/rules/aws-ecs.mdc new file mode 100644 index 000000000..0ce4ef1fc --- /dev/null +++ b/.cursor/rules/aws-ecs.mdc @@ -0,0 +1,210 @@ +--- +description: This rule covers best practices for developing, deploying, and maintaining applications using AWS Elastic Container Service (ECS). It includes guidance on code organization, performance, security, testing, and common pitfalls. +globs: *.tf,*.yml,*.yaml,*.json,*.sh,*.dockerfile +--- +- **General Principles:** + - **Infrastructure as Code (IaC):** Treat your ECS infrastructure as code. Use tools like Terraform, AWS CloudFormation, or AWS CDK to define and manage your ECS clusters, task definitions, and services. + - **Configuration Management:** Externalize configuration from your application code. Use environment variables, AWS Secrets Manager, or AWS Systems Manager Parameter Store to manage configuration data. + - **Idempotency:** Ensure that your deployment scripts and automation are idempotent. This means that running the same script multiple times should have the same result as running it once. + - **Automation:** Automate all aspects of your ECS deployment, from building container images to deploying services and scaling resources. + +- **1. Code Organization and Structure:** + - **1.1 Directory Structure:** + - **Project Root:** The base directory containing all project-related files. + - `infrastructure/`: Contains IaC code (Terraform, CloudFormation) for ECS clusters, task definitions, services, VPC, etc. + - `application/`: Contains source code for your containerized application. + - `src/`: Application source code. + - `tests/`: Unit, integration, and end-to-end tests. + - `Dockerfile`: Dockerfile for building the application container image. + - `scripts/`: Contains deployment, build, and other utility scripts. + - `docs/`: Documentation for the project. + - `README.md`: Project README file. + + - **1.2 File Naming Conventions:** + - **Terraform:** `main.tf`, `variables.tf`, `outputs.tf`, `.tf` + - **CloudFormation:** `.yml` or `.yaml` + - **Docker:** `Dockerfile` (no extension), `.dockerignore` + - **Scripts:** `.sh` (Bash), `.py` (Python) + - **Configuration:** `config.yml`, `config.json`, `.env` + + - **1.3 Module Organization:** + - **Terraform Modules:** Create reusable Terraform modules for common ECS infrastructure components (e.g., ECS cluster, task definition, load balancer). + - **Application Modules:** Organize your application code into logical modules based on functionality (e.g., authentication, API, database access). + - Use separate git repositories for independent services/applications, and ECS services accordingly. This avoids monolithic deployments and allows independent scaling and versioning. + + - **1.4 Component Architecture:** + - **Microservices:** Design your application as a collection of microservices, each deployed as an independent ECS service. + - **API Gateway:** Use an API gateway (e.g., Amazon API Gateway) to route requests to different ECS services. + - **Message Queue:** Use a message queue (e.g., Amazon SQS, Amazon MQ) for asynchronous communication between services. + - **Database:** Use a managed database service (e.g., Amazon RDS, Amazon DynamoDB) for data persistence. Ensure proper security and access control configurations. + + - **1.5 Code Splitting:** + - Break down large applications or services into smaller, more manageable components that can be deployed independently. + - Optimize container images to only include code required for each specific component. + - Utilize ECS services to deploy each component separately, enabling independent scaling and upgrades. + +- **2. Common Patterns and Anti-patterns:** + - **2.1 Design Patterns:** + - **Sidecar:** Use sidecar containers (e.g., for logging, monitoring, or service mesh) to add functionality to your main application container without modifying the application code. Use a separate container in the same task definition. + - **Backend for Frontend (BFF):** Create a BFF layer that provides a specific API for each client application, improving performance and security. + - **Strangler Fig:** Gradually migrate a legacy application to ECS by introducing new microservices and slowly replacing the old functionality. + - **Aggregator Pattern:** Use an aggregator service to combine data from multiple backend services into a single response. + + - **2.2 Recommended Approaches:** + - **Health Checks:** Implement robust health checks for your containers and configure ECS to use them. Use both `HEALTHCHECK` in the `Dockerfile` for initial container health and ECS health checks (ELB health checks) for service availability. + - **Service Discovery:** Use service discovery (e.g., AWS Cloud Map) to enable services to find each other dynamically. + - **Load Balancing:** Use a load balancer (e.g., Application Load Balancer, Network Load Balancer) to distribute traffic across multiple ECS tasks. + - **Auto Scaling:** Configure auto scaling to automatically adjust the number of ECS tasks based on demand. Use CloudWatch metrics (CPU, memory, request count) as scaling triggers. + - **Immutable Infrastructure:** Avoid making changes to running containers. Instead, redeploy a new container image with the changes. + - **Use ECS Exec for debugging** avoid SSH into running EC2 instances + + - **2.3 Anti-patterns:** + - **Hardcoding Configuration:** Avoid hardcoding configuration values in your application code. + - **Large Container Images:** Keep your container images small by using multi-stage builds and removing unnecessary dependencies. + - **Ignoring Security Best Practices:** Neglecting security best practices can lead to vulnerabilities. Always follow security best practices for container images, task definitions, and IAM roles. + - **Manual Scaling:** Manually scaling ECS tasks is inefficient and error-prone. Use auto scaling instead. + - **Monolithic Containers:** Avoid creating single containers that run multiple applications. Break them down into smaller, single-purpose containers. + + - **2.4 State Management:** + - **Stateless Applications:** Design your applications to be stateless whenever possible. Store state in a database or other external storage service. + - **Persistent Storage:** Use persistent storage volumes (e.g., Amazon EFS, Amazon EBS) for stateful applications. + - **Container Lifecycle:** Understand the container lifecycle and how ECS manages container restarts and replacements. + + - **2.5 Error Handling:** + - **Logging:** Implement comprehensive logging and monitoring for your applications. Log to stdout/stderr, and use a logging driver (e.g., FireLens) to ship logs to a central logging service (e.g., CloudWatch Logs, Splunk). + - **Exception Handling:** Implement proper exception handling in your application code. + - **Retry Logic:** Implement retry logic for transient errors. + - **Dead Letter Queues:** Use dead letter queues (DLQs) to handle messages that cannot be processed after multiple retries. + +- **3. Performance Considerations:** + - **3.1 Optimization Techniques:** + - **Resource Allocation:** Properly size your ECS tasks based on the application's resource requirements (CPU, memory). Monitor resource utilization and adjust task sizes as needed. + - **Container Image Optimization:** Optimize container images by minimizing size, using efficient base images, and leveraging layer caching. Use tools like `docker image prune` to remove unused images. + - **Load Balancing:** Configure load balancing algorithms and connection draining to optimize traffic distribution and minimize downtime. + - **Caching:** Implement caching at various levels (e.g., application, CDN) to reduce latency and improve performance. + - **Connection Pooling:** Reuse database connections to minimize overhead. + + - **3.2 Memory Management:** + - **Memory Limits:** Set appropriate memory limits for your containers. Monitor memory usage and adjust limits to prevent out-of-memory errors. + - **Garbage Collection:** Optimize garbage collection settings for your application's runtime environment. + - **Memory Leaks:** Identify and fix memory leaks in your application code. + + - **3.3 Bundle Size Optimization:** + - **Code Splitting:** Split your application code into smaller bundles to reduce the initial load time. Utilize dynamic imports to only load code when it's needed. + - **Tree Shaking:** Remove unused code from your application bundle. + - **Minification:** Minify your application code to reduce its size. + - **Compression:** Compress your application code to reduce its size. + + - **3.4 Lazy Loading:** + - Load resources (e.g., images, data) only when they are needed. + - Use lazy-loading techniques to improve the initial load time of your application. + +- **4. Security Best Practices:** + - **4.1 Common Vulnerabilities:** + - **Container Image Vulnerabilities:** Unpatched vulnerabilities in container images can be exploited by attackers. Regularly scan container images for vulnerabilities and apply patches. + - **IAM Role Misconfiguration:** Overly permissive IAM roles can allow attackers to access sensitive resources. Follow the principle of least privilege and grant only the necessary permissions. + - **Network Security Misconfiguration:** Misconfigured network security can expose your ECS services to unauthorized access. Use security groups and network ACLs to restrict network access. + - **Secrets Management Vulnerabilities:** Storing secrets in plain text can expose them to attackers. Use a secrets management service (e.g., AWS Secrets Manager) to securely store and manage secrets. + + - **4.2 Input Validation:** + - Validate all input data to prevent injection attacks (e.g., SQL injection, command injection). + - Use a framework or library to perform input validation. + + - **4.3 Authentication and Authorization:** + - Implement authentication and authorization to control access to your ECS services. + - Use a standard authentication protocol (e.g., OAuth 2.0, OpenID Connect). + - Use fine-grained authorization policies to restrict access to specific resources. + + - **4.4 Data Protection:** + - Encrypt sensitive data at rest and in transit. + - Use HTTPS to encrypt data in transit. + - Use AWS KMS to encrypt data at rest. + - Implement data masking and tokenization to protect sensitive data. + + - **4.5 Secure API Communication:** + - Use HTTPS for all API communication. + - Implement authentication and authorization for all API endpoints. + - Validate all input data to prevent injection attacks. + - Protect against Cross-Site Request Forgery (CSRF) attacks. + +- **5. Testing Approaches:** + - **5.1 Unit Testing:** + - Write unit tests for individual components of your application. + - Use a unit testing framework (e.g., JUnit, pytest). + - Mock external dependencies to isolate the component being tested. + + - **5.2 Integration Testing:** + - Write integration tests to verify the interaction between different components of your application. + - Use a test environment that is similar to the production environment. + - Test the integration with external services (e.g., databases, message queues). + + - **5.3 End-to-End Testing:** + - Write end-to-end tests to verify the functionality of the entire application. + - Use an end-to-end testing framework (e.g., Selenium, Cypress). + - Test the application from the user's perspective. + + - **5.4 Test Organization:** + - Organize your tests into a logical directory structure. + - Use meaningful names for your test files and test methods. + - Use a test runner to execute your tests. + + - **5.5 Mocking and Stubbing:** + - Use mocking and stubbing to isolate components during testing. + - Use a mocking framework (e.g., Mockito, EasyMock). + - Create mock objects that simulate the behavior of external dependencies. + +- **6. Common Pitfalls and Gotchas:** + - **6.1 Frequent Mistakes:** + - **Incorrect IAM Permissions:** Granting insufficient or excessive IAM permissions can lead to security vulnerabilities or application failures. + - **Misconfigured Network Settings:** Misconfigured network settings can prevent containers from communicating with each other or with external services. + - **Insufficient Resource Limits:** Setting insufficient resource limits (CPU, memory) can cause containers to crash or become unresponsive. + - **Ignoring Health Checks:** Ignoring health checks can lead to ECS tasks being considered healthy even when they are not. + - **Failing to handle SIGTERM signals:** Applications in containers must gracefully handle SIGTERM signals to allow ECS to shutdown tasks cleanly. + + - **6.2 Edge Cases:** + - **Task Placement Constraints:** Be aware of task placement constraints (e.g., availability zone, instance type) and how they can affect task scheduling. + - **Service Discovery Limitations:** Understand the limitations of service discovery and how it can affect service resolution. + - **Load Balancer Connection Draining:** Be aware of load balancer connection draining and how it can affect application availability during deployments. + + - **6.3 Version-Specific Issues:** + - Stay up-to-date with the latest ECS agent and Docker versions to avoid known issues and security vulnerabilities. + - Check the AWS documentation for any version-specific issues or known bugs. + + - **6.4 Compatibility Concerns:** + - Be aware of compatibility issues between ECS and other AWS services (e.g., VPC, IAM, CloudWatch). + - Test your application with different versions of these services to ensure compatibility. + + - **6.5 Debugging Strategies:** + - **Logging:** Use comprehensive logging to track application behavior and identify errors. + - **Monitoring:** Use monitoring tools (e.g., CloudWatch) to track resource utilization and application performance. + - **Debugging Tools:** Use debugging tools (e.g., Docker exec, AWS Systems Manager Session Manager) to troubleshoot running containers. + - **ECS Exec:** Utilize the ECS Exec feature to directly connect to containers for debugging purposes. + +- **7. Tooling and Environment:** + - **7.1 Recommended Tools:** + - **Terraform:** For infrastructure as code. + - **AWS CLI:** For interacting with AWS services from the command line. + - **Docker:** For building and managing container images. + - **AWS SAM CLI:** For local development of serverless applications. + - **Visual Studio Code (VS Code):** With AWS and Docker extensions for development and debugging. + - **cdk8s:** Define Kubernetes applications and ECS using general purpose programming languages. + + - **7.2 Build Configuration:** + - **Makefile:** Use a Makefile to automate build, test, and deployment tasks. + - **Build Scripts:** Use build scripts to customize the build process. + - **CI/CD Pipeline:** Integrate your build process with a CI/CD pipeline (e.g., AWS CodePipeline, Jenkins). + + - **7.3 Linting and Formatting:** + - **Linters:** Use linters (e.g., ESLint, Pylint) to enforce code style and identify potential errors. + - **Formatters:** Use formatters (e.g., Prettier, Black) to automatically format your code. + - **Editor Integration:** Integrate linters and formatters with your code editor. + + - **7.4 Deployment:** + - **Blue/Green Deployments:** Use blue/green deployments to minimize downtime during deployments. + - **Canary Deployments:** Use canary deployments to gradually roll out new versions of your application. + - **Rolling Updates:** Use rolling updates to gradually update ECS tasks with the new version. + + - **7.5 CI/CD Integration:** + - **Automated Builds:** Automate the build process using a CI/CD pipeline. + - **Automated Tests:** Run automated tests as part of the CI/CD pipeline. + - **Automated Deployments:** Automate the deployment process using a CI/CD pipeline. \ No newline at end of file diff --git a/.cursor/rules/aws-lambda.mdc b/.cursor/rules/aws-lambda.mdc new file mode 100644 index 000000000..ba1019f6d --- /dev/null +++ b/.cursor/rules/aws-lambda.mdc @@ -0,0 +1,220 @@ +--- +description: Comprehensive guide for AWS Lambda development, covering best practices for code organization, performance, security, testing, and common pitfalls. Focuses on building robust, scalable, and secure serverless applications using AWS Lambda. +globs: *.js,*.ts,*.jsx,*.tsx,*.py,*.java,*.go,*.c#,*.cs,*.ps1,*.sh +--- +- **Use Documented APIs Only:** Rely solely on AWS Lambda's documented APIs to ensure compatibility and avoid unexpected behavior. Do not use undocumented or internal APIs. + +- **Single Responsibility Principle:** Design Lambda functions to perform a single, well-defined task. This promotes modularity, testability, and maintainability. + +- **Native AWS Integrations:** Prefer native AWS integrations (e.g., SQS, SNS, Kinesis) for passing data between services instead of custom code, which enhances scalability and reduces complexity. + +- **Local Dependencies:** Store and reference dependencies locally within the Lambda function package to minimize latency and improve cold start times. + +- **Limit Variable Re-initialization:** Initialize variables outside the function handler to leverage execution context reuse and reduce initialization overhead. + +- **Thorough Testing:** Implement a comprehensive testing strategy including unit, integration, and end-to-end tests to ensure function correctness and reliability. + +## 1. Code Organization and Structure: + + - **Directory Structure:** + - Organize code into logical directories based on functionality (e.g., `src`, `tests`, `utils`). + - Separate handler logic from core business logic. + - Example: + + my-lambda-function/ + ├── src/ + │ ├── handler.js # Lambda handler function + │ ├── business-logic.js # Core business logic + │ └── utils.js # Utility functions + ├── tests/ + │ ├── handler.test.js # Unit tests for handler.js + │ └── ... + ├── package.json # Dependencies and scripts + └── ... + + + - **File Naming Conventions:** + - Use descriptive and consistent file names. + - Follow a naming convention (e.g., `kebab-case`, `PascalCase`) consistently. + - Example: `user-service.js`, `UserService.java`. + + - **Module Organization:** + - Break down large functions into smaller, reusable modules. + - Use appropriate module systems (e.g., ES modules in JavaScript, packages in Python, modules in Go). + - Example (JavaScript): + javascript + // src/user-service.js + export function createUser(userData) { ... } + export function getUser(userId) { ... } + + + - **Component Architecture:** + - Design Lambda functions as part of a larger component-based architecture. + - Decouple functions from specific triggers (e.g., API Gateway, SQS) to increase reusability. + - Implement interfaces or contracts to define interactions between components. + + - **Code Splitting Strategies:** + - Use code splitting techniques to reduce Lambda function package size. + - Implement dynamic imports or lazy loading for infrequently used modules (where supported by the runtime). + - Leverage layers for shared dependencies across multiple functions. + +## 2. Common Patterns and Anti-patterns: + + - **Design Patterns:** + - **Event-Driven Architecture:** Design functions to respond to events from various AWS services. + - **Command Pattern:** Encapsulate requests as objects to decouple request processing from the handler. + - **Dependency Injection:** Inject dependencies into functions to improve testability and maintainability. + + - **Recommended Approaches:** + - **Configuration Management:** Use environment variables or AWS Systems Manager Parameter Store for configuration data. + - **Logging:** Implement structured logging with timestamps, request IDs, and relevant context. + - **Idempotency:** Ensure functions are idempotent to handle potential retry scenarios gracefully. + + - **Anti-patterns:** + - **Monolithic Functions:** Avoid creating large, complex functions that perform multiple unrelated tasks. + - **Hardcoding Secrets:** Never hardcode sensitive information (e.g., API keys, passwords) in code. Use AWS Secrets Manager or environment variables. + - **Excessive Dependencies:** Minimize the number of dependencies to reduce package size and improve cold start times. + + - **State Management:** + - Use external services (e.g., DynamoDB, Redis) for persistent state management. + - Leverage Lambda layers for caching frequently accessed data. + - Design functions to be stateless whenever possible. + + - **Error Handling:** + - Implement robust error handling mechanisms, including try-catch blocks and error logging. + - Use custom exceptions for specific error scenarios. + - Return meaningful error messages to the caller. + - Implement dead-letter queues (DLQs) for asynchronous invocations to handle failed events. + +## 3. Performance Considerations: + + - **Optimization Techniques:** + - **Minimize Package Size:** Reduce the size of the deployment package by removing unnecessary files and dependencies. + - **Optimize Dependencies:** Use lightweight dependencies and avoid importing entire libraries when only specific functions are needed. + - **Code Optimization:** Profile and optimize code for performance bottlenecks. + - **Connection Reuse:** Reuse database connections, HTTP clients, and other resources to minimize overhead. + + - **Memory Management:** + - Allocate sufficient memory to Lambda functions based on their needs. + - Monitor memory usage and adjust the memory allocation accordingly. + - Clean up resources (e.g., closing file streams, releasing memory) to avoid memory leaks. + + - **Cold Starts:** + - Minimize cold start times by reducing package size, optimizing dependencies, and using provisioned concurrency. + - Use initilization logic outside the handler to take advantage of container reuse + + - **Bundle Size Optimization:** + - Use tools like Webpack or esbuild to bundle and optimize code for production. + - Minify code and remove unused code (dead code elimination). + + - **Lazy Loading Strategies:** + - Implement lazy loading or dynamic imports for infrequently used modules to reduce initial load time. + +## 4. Security Best Practices: + + - **Common Vulnerabilities:** + - **Injection Attacks:** Prevent SQL injection, command injection, and other injection attacks by validating and sanitizing input. + - **Cross-Site Scripting (XSS):** Protect against XSS vulnerabilities by encoding output properly. + - **Insecure Deserialization:** Avoid deserializing untrusted data to prevent arbitrary code execution. + - **Broken Authentication:** Implement strong authentication and authorization mechanisms. + + - **Input Validation:** + - Validate all input data to prevent malicious or invalid data from being processed. + - Use schema validation libraries to enforce data types and formats. + - Sanitize input data to remove potentially harmful characters or code. + + - **Authentication and Authorization:** + - Use AWS Identity and Access Management (IAM) roles to grant Lambda functions the necessary permissions. + - Implement authentication and authorization mechanisms to protect sensitive resources. + - Use AWS Cognito for user authentication and authorization. + + - **Data Protection:** + - Encrypt sensitive data at rest and in transit. + - Use AWS Key Management Service (KMS) to manage encryption keys. + - Mask or redact sensitive data in logs. + + - **Secure API Communication:** + - Use HTTPS for all API communication. + - Implement API authentication and authorization using API keys, JWT tokens, or other secure mechanisms. + - Protect against common web application attacks (e.g., CSRF, DDoS). + +## 5. Testing Approaches: + + - **Unit Testing:** + - Write unit tests to verify the correctness of individual functions or modules. + - Use mocking or stubbing to isolate units of code from external dependencies. + - Use testing frameworks (e.g., Jest, Mocha, pytest) to automate unit testing. + + - **Integration Testing:** + - Write integration tests to verify the interactions between different components or services. + - Test the integration of Lambda functions with other AWS services (e.g., SQS, DynamoDB). + - Use tools like AWS SAM Local to run integration tests locally. + + - **End-to-End Testing:** + - Write end-to-end tests to verify the entire application flow from end to end. + - Simulate user interactions to test the application's functionality. + - Use tools like Selenium or Cypress to automate end-to-end testing. + + - **Test Organization:** + - Organize tests into separate directories based on the type of test (e.g., unit, integration, e2e). + - Use descriptive test names to clearly identify the purpose of each test. + - Follow a consistent testing strategy across all projects. + + - **Mocking and Stubbing:** + - Use mocking or stubbing to replace external dependencies with controlled test doubles. + - Mock AWS SDK calls to isolate Lambda functions from AWS services. + - Use mocking frameworks (e.g., Jest, Sinon.js) to simplify mocking and stubbing. + +## 6. Common Pitfalls and Gotchas: + + - **Frequent Mistakes:** + - **Incorrect IAM Permissions:** Ensure that Lambda functions have the necessary IAM permissions to access required AWS resources. + - **Timeout Errors:** Increase the Lambda function timeout if it takes longer than expected to complete. + - **Memory Exhaustion:** Allocate sufficient memory to Lambda functions to avoid memory exhaustion errors. + - **Unclosed Connections:** Close database connections, HTTP clients, and other resources properly to avoid resource leaks. + + - **Edge Cases:** + - **Concurrent Invocations:** Handle concurrent invocations gracefully to avoid race conditions or data corruption. + - **Throttling:** Implement retry mechanisms to handle throttling errors from AWS services. + - **Error Retries:** Design functions to be idempotent to handle potential retry scenarios gracefully. + + - **Version-Specific Issues:** + - Be aware of version-specific issues or compatibility concerns when upgrading Lambda function runtimes or dependencies. + - Test Lambda functions thoroughly after upgrading to ensure they still function correctly. + + - **Compatibility Concerns:** + - Ensure that Lambda functions are compatible with the supported runtime environment. + - Use compatible versions of dependencies to avoid compatibility issues. + + - **Debugging Strategies:** + - Use CloudWatch Logs to debug Lambda function execution. + - Use AWS X-Ray to trace and analyze Lambda function performance. + - Use local debugging tools (e.g., AWS SAM Local) to debug Lambda functions locally. + +## 7. Tooling and Environment: + + - **Recommended Tools:** + - **AWS SAM:** Use AWS SAM (Serverless Application Model) to define and deploy Lambda functions and other serverless resources. + - **AWS CLI:** Use the AWS CLI (Command Line Interface) to manage AWS resources from the command line. + - **Terraform/CloudFormation:** Use Infrastructure as Code (IaC) tools like Terraform or CloudFormation to provision and manage AWS infrastructure. + - **IDE:** Use an IDE with support for Lambda development (e.g., VS Code with the AWS Toolkit extension). + + - **Build Configuration:** + - Use build tools (e.g., npm, yarn, Maven, Gradle) to manage dependencies and build Lambda function packages. + - Configure build scripts to automate common tasks (e.g., linting, testing, bundling). + + - **Linting and Formatting:** + - Use linters (e.g., ESLint, Pylint) to enforce coding standards and identify potential code quality issues. + - Use formatters (e.g., Prettier, Black) to automatically format code consistently. + + - **Deployment:** + - Use AWS SAM CLI or other deployment tools to deploy Lambda functions to AWS. + - Implement blue/green deployments or canary deployments to minimize downtime during deployments. + - Automate deployments using CI/CD pipelines. + + - **CI/CD Integration:** + - Integrate Lambda function deployments with CI/CD pipelines to automate testing and deployment. + - Use CI/CD tools (e.g., Jenkins, CircleCI, GitHub Actions) to build, test, and deploy Lambda functions. + +@file aws-lambda-security.mdc +@file aws-lambda-performance.mdc \ No newline at end of file diff --git a/.cursor/rules/aws-rds.mdc b/.cursor/rules/aws-rds.mdc new file mode 100644 index 000000000..6e4ca2642 --- /dev/null +++ b/.cursor/rules/aws-rds.mdc @@ -0,0 +1,324 @@ +--- +description: This rule provides best practices and coding standards for developing applications that interact with AWS RDS. It covers code organization, security, performance, testing, and common pitfalls. +globs: *.sql,*.js,*.py,*.java,*.tf,*.yaml,*.yml +--- +# AWS RDS Best Practices and Coding Standards + +This document outlines best practices for developing applications interacting with AWS Relational Database Service (RDS). It covers code organization, common patterns, performance, security, testing, and deployment. + +## Library Information: + +- Name: aws-rds +- Tags: database, sql, aws, cloud + +## 1. Code Organization and Structure + +### 1.1 Directory Structure + +Organize your project to promote maintainability and scalability. A suggested directory structure is: + + +project-root/ +├── modules/ # Reusable components for RDS interactions +│ ├── db_config.py # Database connection configuration (e.g., using SQLAlchemy) +│ ├── rds_operations.py # CRUD operations (e.g., using boto3) +│ └── query_builder.py # Dynamic SQL query generation +├── models/ # Data models (e.g., SQLAlchemy models, Pydantic schemas) +│ ├── user.py # Example: User model definition +│ └── order.py # Example: Order model definition +├── services/ # Application logic interacting with RDS +│ ├── user_service.py # Example: User management service +│ └── reporting_service.py # Example: Reporting service +├── tests/ # Test suite +│ ├── unit/ # Unit tests for individual components +│ ├── integration/ # Integration tests for RDS interactions +│ └── e2e/ # End-to-end tests +├── config/ # Configuration files +│ ├── settings.py # Application settings (database credentials, etc.) +│ └── logging.yaml # Logging configuration +├── migrations/ # Database migration scripts (e.g., using Alembic) +├── infrastructure/ # Infrastructure-as-Code (IaC) using Terraform or CloudFormation +│ ├── rds.tf # RDS instance and security group definition (Terraform example) +│ └── security_group.tf # RDS Security Group definition (Terraform example) +├── scripts/ # Utility scripts for database management and deployment +├── README.md # Project documentation +├── requirements.txt # Python dependencies (example) +└── .env # Environment variables (for local development, never commit this file!) + + +### 1.2 File Naming Conventions + +- **Python:** Use snake_case for file and variable names (e.g., `rds_operations.py`, `user_id`). +- **JavaScript/TypeScript:** Use camelCase for variable names and PascalCase for class and component names (e.g., `getUser`, `UserComponent`). Generally, kebab-case is recommended for component file names (e.g., `user-component.jsx`). +- **SQL:** Use snake_case for table and column names (e.g., `users`, `user_name`). Use `.sql` as the file extension and use uppercase for SQL keywords (e.g., `SELECT`, `FROM`, `WHERE`). +- **Terraform:** Use snake_case (e.g., `rds_instance.tf`). + +### 1.3 Module Organization + +- Break down large modules into smaller, more manageable files. +- Use clear and descriptive module names. +- Avoid circular dependencies between modules. +- Use dependency injection to decouple modules and improve testability. + +### 1.4 Component Architecture + +- **Layered Architecture:** Separate concerns into distinct layers (e.g., data access layer, business logic layer, presentation layer). +- **Microservices:** For larger applications, consider using a microservices architecture, with each service responsible for a specific domain and interacting with its own RDS instance. +- **Event-Driven Architecture:** Use an event-driven architecture to decouple components and improve scalability. For example, use AWS SNS or SQS to trigger actions based on database changes. + +### 1.5 Code Splitting + +- Split large files into smaller, more focused files. +- Organize files by feature or functionality. +- Use lazy loading for less frequently used modules. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns + +- **Repository Pattern:** Abstract data access logic behind a repository interface. +- **Unit of Work Pattern:** Track changes to entities and persist them as a single unit. +- **Factory Pattern:** Encapsulate object creation logic. +- **Observer Pattern:** React to database changes or events. +- **Singleton Pattern:** Use carefully for database connections. Connection pooling libraries often manage this internally; avoid creating your own Singleton unless necessary for custom connection management logic. + +### 2.2 Recommended Approaches + +- **Connection Pooling:** Use connection pooling to efficiently manage database connections. Popular libraries include SQLAlchemy (Python) and HikariCP (Java). +- **ORM (Object-Relational Mapping):** Use an ORM like SQLAlchemy, Django ORM (Python), Hibernate (Java), or Sequelize (JavaScript) to map database tables to objects and simplify data access. +- **Database Migrations:** Use a migration tool like Alembic (Python) or Flyway (Java) to manage database schema changes. +- **Asynchronous Operations:** Use asynchronous operations to avoid blocking the main thread when interacting with RDS, especially for long-running queries or large data transfers. +- **Parameterized Queries:** Always use parameterized queries to prevent SQL injection attacks. + +### 2.3 Anti-patterns + +- **Writing Raw SQL Directly in Application Code:** Avoid embedding raw SQL queries directly within the application code. This makes the code harder to maintain and more vulnerable to SQL injection attacks. Use an ORM or query builder instead. +- **Ignoring Connection Management:** Failing to properly open and close database connections can lead to resource exhaustion and performance issues. Always use connection pooling and ensure connections are closed promptly. +- **Over-fetching Data:** Avoid retrieving more data than necessary from the database. This can lead to performance problems, especially when dealing with large tables. Use appropriate `WHERE` clauses and select only the required columns. +- **Ignoring Error Handling:** Failing to handle database errors gracefully can lead to unexpected application behavior and data corruption. Implement robust error handling mechanisms and log errors appropriately. +- **Storing Secrets in Code:** Never store database credentials or other sensitive information directly in the application code or configuration files. Use environment variables or a secrets management service like AWS Secrets Manager. +- **Lack of Indexing:** Neglecting to add indexes to frequently queried columns can drastically reduce query performance. Analyze query execution plans to identify missing indexes. + +### 2.4 State Management + +- **Stateless Applications:** Design applications to be stateless to improve scalability and fault tolerance. Use RDS to store persistent data. +- **Caching:** Use caching to reduce the load on RDS. Implement caching at different levels (e.g., application-level caching, database-level caching using Redis or Memcached). + +### 2.5 Error Handling + +- **Try-Except Blocks:** Use `try-except` (Python), `try-catch` (Java/JavaScript) blocks to handle potential exceptions during RDS interactions. +- **Logging:** Log all database errors to a central location for monitoring and debugging. +- **Retry Mechanism:** Implement a retry mechanism with exponential backoff for transient errors (e.g., network connectivity issues). +- **Custom Exceptions:** Define custom exception classes for specific RDS-related errors. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +- **Query Optimization:** Analyze query execution plans and optimize queries for performance. Use indexes, rewrite inefficient queries, and avoid using `SELECT *`. +- **Connection Pooling:** As mentioned earlier, use connection pooling to reuse database connections and reduce overhead. +- **Caching:** Implement caching strategies to reduce the number of database queries. +- **Read Replicas:** Use read replicas to offload read traffic from the primary RDS instance. +- **Provisioned IOPS:** Use provisioned IOPS (IOPS) for workloads that require consistent high performance. +- **Database Tuning:** Tune database parameters (e.g., buffer pool size, query cache size) based on workload characteristics. +- **Partitioning:** Use table partitioning for very large tables to improve query performance and manageability. +- **Stored Procedures:** Use stored procedures for complex operations to reduce network traffic and improve security. + +### 3.2 Memory Management + +- **Appropriate Instance Size:** Choose an RDS instance size that is appropriate for the workload. Consider memory requirements for the database server and application. +- **Monitor Memory Usage:** Monitor memory usage on the RDS instance using CloudWatch metrics and adjust the instance size or database parameters as needed. +- **Avoid Memory Leaks:** Be careful to avoid memory leaks in the application code, especially when dealing with large datasets or database connections. + +### 3.3 Rendering Optimization (If Applicable) + +- This is primarily relevant if you are directly rendering data retrieved from RDS in a user interface. +- **Pagination:** Implement pagination to load data in smaller chunks, improving initial load times. +- **Virtualization:** Use virtualization techniques (e.g., virtual scrolling) to render large lists of data efficiently. +- **Memoization:** Use memoization to cache the results of expensive rendering operations. + +### 3.4 Bundle Size Optimization + +- **Tree Shaking:** Use tree shaking to remove unused code from the JavaScript/TypeScript bundle. +- **Code Splitting:** Split the code into smaller chunks that can be loaded on demand. +- **Minification:** Minify the code to reduce the bundle size. +- **Compression:** Compress the bundle using gzip or Brotli. + +### 3.5 Lazy Loading + +- **Lazy Loading of Modules:** Load modules on demand to reduce the initial load time of the application. +- **Lazy Loading of Data:** Load data only when it is needed (e.g., when a user scrolls down a page). + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities + +- **SQL Injection:** Exploiting vulnerabilities in SQL queries to execute malicious code. +- **Cross-Site Scripting (XSS):** Injecting malicious scripts into web pages served from your application. +- **Cross-Site Request Forgery (CSRF):** Tricking users into performing actions they did not intend to perform. +- **Data Breach:** Unauthorized access to sensitive data stored in RDS. +- **Denial of Service (DoS):** Overwhelming the RDS instance with requests, making it unavailable to legitimate users. + +### 4.2 Input Validation + +- **Whitelist Validation:** Validate input against a whitelist of allowed characters or patterns. +- **Data Type Validation:** Ensure that input data is of the correct data type. +- **Length Validation:** Limit the length of input strings to prevent buffer overflows. +- **Encoding:** Encode input data to prevent HTML injection. +- **Parameterized Queries (again!):** Use parameterized queries to prevent SQL injection attacks. + +### 4.3 Authentication and Authorization + +- **IAM Roles:** Use IAM roles to grant permissions to applications to access RDS. +- **Database Usernames and Passwords:** Use strong database usernames and passwords and store them securely using AWS Secrets Manager. Avoid using the `admin` user for application access. +- **Least Privilege Principle:** Grant only the necessary permissions to database users. +- **Multi-Factor Authentication (MFA):** Enable MFA for database user accounts. +- **Row-Level Security (RLS):** Implement row-level security to restrict access to specific rows in a table. +- **VPC Security Groups:** Utilize VPC security groups to control network access to the RDS instance. + +### 4.4 Data Protection + +- **Encryption at Rest:** Enable encryption at rest for the RDS instance. +- **Encryption in Transit:** Use SSL/TLS to encrypt data in transit between the application and RDS. +- **Data Masking:** Mask sensitive data in the database to protect it from unauthorized access. +- **Data Backup and Recovery:** Implement a data backup and recovery plan to protect against data loss. +- **Regular Auditing:** Audit database activity regularly to identify and investigate suspicious behavior. + +### 4.5 Secure API Communication + +- **HTTPS:** Use HTTPS for all API communication with the application. +- **API Authentication:** Authenticate all API requests using API keys, JWT tokens, or other authentication mechanisms. +- **API Authorization:** Authorize all API requests to ensure that users only have access to the resources they are authorized to access. +- **Rate Limiting:** Implement rate limiting to prevent DoS attacks. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +- **Focus:** Test individual components in isolation (e.g., functions, classes). +- **Mocking:** Use mocking to isolate components from external dependencies, such as RDS. +- **Assertions:** Use assertions to verify that the components are behaving as expected. +- **Test Coverage:** Aim for high test coverage to ensure that all parts of the code are tested. + +### 5.2 Integration Testing + +- **Focus:** Test the interactions between different components, including RDS. +- **Test Databases:** Use test databases for integration tests to avoid affecting production data. +- **Data Setup and Teardown:** Set up test data before each test and tear down the data after each test to ensure that the tests are isolated. +- **Transaction Management:** Use transaction management to rollback changes made during tests. + +### 5.3 End-to-End (E2E) Testing + +- **Focus:** Test the entire application flow, from the user interface to the database. +- **Automated Testing:** Automate E2E tests to ensure that the application is working correctly. +- **Realistic Scenarios:** Use realistic scenarios to simulate user behavior. +- **Browser Automation:** Use browser automation tools (e.g., Selenium, Cypress) to automate E2E tests. + +### 5.4 Test Organization + +- **Separate Test Directory:** Store all tests in a separate directory. +- **Mirror Source Code Structure:** Mirror the source code structure in the test directory to make it easy to find the tests for a specific component. +- **Descriptive Test Names:** Use descriptive test names to make it clear what each test is testing. + +### 5.5 Mocking and Stubbing + +- **Mocking:** Replace external dependencies with mock objects that simulate their behavior. +- **Stubbing:** Replace specific methods or functions of external dependencies with stubs that return predefined values. +- **Mocking Libraries:** Use mocking libraries (e.g., Mockito (Java), pytest-mock (Python), Jest (JavaScript)) to simplify the creation of mock objects and stubs. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +- **Not Using Connection Pooling:** Leading to excessive connection overhead and performance issues. +- **SQL Injection Vulnerabilities:** Due to improper input validation and not using parameterized queries. +- **Over-fetching Data:** Retrieving more data than necessary, leading to performance problems. +- **Ignoring Error Handling:** Leading to unexpected application behavior and data corruption. +- **Not Monitoring RDS Performance:** Failing to monitor RDS performance, leading to undetected performance issues. +- **Underestimating Database Growth:** Not planning for future database growth, leading to performance problems and storage limitations. +- **Using Default Security Settings:** Leaving the RDS instance with default security settings, making it vulnerable to attacks. + +### 6.2 Edge Cases + +- **Handling Large Datasets:** Optimizing queries and using pagination to handle large datasets efficiently. +- **Concurrent Access:** Handling concurrent access to the database using transactions and locking mechanisms. +- **Network Connectivity Issues:** Implementing retry mechanisms and handling network connectivity issues gracefully. +- **Database Failover:** Testing the application's ability to handle database failover seamlessly. +- **Time Zones:** Correctly handling time zones when storing and retrieving dates and times from RDS. + +### 6.3 Version-Specific Issues + +- **Check Release Notes:** Always review the release notes for each new version of the RDS engine to identify any breaking changes or known issues. +- **Compatibility Testing:** Perform compatibility testing to ensure that the application works correctly with the new version of the RDS engine. +- **Deprecation Warnings:** Pay attention to deprecation warnings and update the code accordingly. + +### 6.4 Compatibility Concerns + +- **ORM Compatibility:** Ensure that the ORM is compatible with the specific RDS engine being used. +- **Driver Compatibility:** Use the latest version of the database driver to ensure compatibility with the RDS engine. +- **API Compatibility:** Ensure that the application's API is compatible with the RDS API. + +### 6.5 Debugging Strategies + +- **Logging:** Use logging to trace the execution flow of the application and identify potential issues. +- **Remote Debugging:** Use remote debugging to debug the application running in a production environment. +- **Query Profiling:** Use query profiling tools to analyze the performance of SQL queries. +- **Database Monitoring:** Use database monitoring tools to monitor the health and performance of the RDS instance. +- **CloudWatch Metrics:** Use CloudWatch metrics to monitor CPU utilization, memory usage, disk I/O, and network traffic. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +- **IDE (Integrated Development Environment):** VS Code, IntelliJ IDEA, PyCharm, Eclipse. +- **Database Client:** DBeaver, SQL Developer, pgAdmin. +- **Database Migration Tool:** Alembic, Flyway. +- **Testing Framework:** pytest, unittest (Python), JUnit (Java), Jest (JavaScript). +- **Mocking Library:** Mockito (Java), pytest-mock (Python), Jest (JavaScript). +- **AWS CLI:** The AWS Command Line Interface (CLI) is a unified tool to manage your AWS services. +- **Terraform/CloudFormation:** Infrastructure-as-Code tools to manage RDS infrastructure. + +### 7.2 Build Configuration + +- **Dependency Management:** Use a dependency management tool (e.g., pip (Python), Maven (Java), npm/yarn (JavaScript)) to manage project dependencies. +- **Build Automation:** Use a build automation tool (e.g., Make, Gradle, Maven, npm scripts) to automate the build process. +- **Environment Variables:** Use environment variables to configure the application for different environments (e.g., development, staging, production). + +### 7.3 Linting and Formatting + +- **Linters:** Use linters (e.g., pylint (Python), ESLint (JavaScript), Checkstyle (Java)) to enforce coding standards and identify potential issues. +- **Formatters:** Use formatters (e.g., black (Python), Prettier (JavaScript), Google Java Format (Java)) to automatically format the code. + +### 7.4 Deployment + +- **Infrastructure as Code:** Use Infrastructure as Code (IaC) tools (e.g., Terraform, CloudFormation) to automate the deployment of RDS infrastructure. +- **Containerization:** Use containerization (e.g., Docker) to package the application and its dependencies into a portable container. +- **Orchestration:** Use an orchestration tool (e.g., Kubernetes, ECS) to manage and scale the application. +- **Blue/Green Deployments:** Use blue/green deployments to minimize downtime during deployments. +- **Rolling Deployments:** Use rolling deployments to gradually update the application without disrupting service. + +### 7.5 CI/CD Integration + +- **Continuous Integration (CI):** Integrate the build, test, and linting processes into a CI pipeline to automatically run these checks whenever code is committed. +- **Continuous Deployment (CD):** Automate the deployment process to automatically deploy the application to production whenever code is committed. +- **CI/CD Tools:** Use CI/CD tools (e.g., Jenkins, GitHub Actions, CircleCI, GitLab CI) to manage the CI/CD pipeline. + +## Additional RDS Best Practices (From Exa Search and Citations): + +- **Monitoring Metrics:** Monitor RDS metrics like memory, CPU, storage usage using CloudWatch. Set up alarms for unusual activity. +- **Automatic Backups:** Enable automatic backups with a window during low IOPS periods. +- **Scaling DB Instances:** Scale DB instances vertically as needed to meet demand. +- **Optimize Queries:** Tune queries to minimize resource consumption. +- **AWS Database Drivers:** Use the AWS suite of drivers for faster failover and integration with AWS Secrets Manager and IAM. +- **Regular Updates:** Regularly update statistics and monitor performance metrics. +- **Enough RAM:** Allocate enough RAM so that your working set resides almost completely in memory to minimize disk I/O. +- **Client DNS Caching:** Set a time-to-live (TTL) value of less than 30 seconds if your client application is caching the Domain Name Service (DNS) data of your DB instances. +- **Test Failover:** Test failover for your DB instance to understand how long the process takes for your particular use case. + +## References: + +- [AWS RDS Documentation](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_BestPractices.html) +- [AWS Database Blog](https://aws.amazon.com/blogs/database/best-practices-for-configuring-performance-parameters-for-amazon-rds-for-sql-server/) +- [AWS re:Invent Videos (Links as provided in search results)] + +By adhering to these best practices, developers can build scalable, secure, and performant applications that leverage the power of AWS RDS. \ No newline at end of file diff --git a/.cursor/rules/aws.mdc b/.cursor/rules/aws.mdc new file mode 100644 index 000000000..19eaf03a4 --- /dev/null +++ b/.cursor/rules/aws.mdc @@ -0,0 +1,344 @@ +--- +description: Comprehensive rules and best practices for AWS development using Terraform, covering code organization, security, performance, and testing. Adherence to these guidelines ensures maintainable, secure, and efficient infrastructure code. +globs: *.tf +--- +# AWS Development Best Practices using Terraform + +This document outlines best practices for developing and maintaining AWS infrastructure using Terraform. Following these guidelines will help ensure your code is maintainable, secure, and efficient. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure + +- **Root Module**: The primary entry point for your infrastructure definition. Contains `main.tf`, `variables.tf`, `outputs.tf`, `providers.tf`, `versions.tf` and `locals.tf`. +- **Modules Directory**: Reusable components of your infrastructure. Each module should have its own directory under `modules/`. +- **Examples Directory**: Demonstrates how to use the modules. Each example should be a self-contained Terraform configuration. +- **Scripts Directory**: Custom scripts used by Terraform. +- **Templates Directory**: Templates for files that are read using the `templatefile` function, using the extension `.tftpl`. +- **Files Directory**: Contains static files referenced by Terraform (e.g., startup scripts). +- **Helpers Directory**: Contains utility scripts that are *not* directly called by Terraform. +- **.tfvars files**: Files containing variable definitions. Use `terraform.tfvars` for common values and create environment specific files in the `envs/` directory (e.g., `envs/dev/terraform.tfvars`, `envs/prod/terraform.tfvars`). +- **.gitignore**: Specifies intentionally untracked files that Git should ignore. +- **docs/**: Place any supplemental documentation in a `docs/` subdirectory. +- **CODEOWNERS**: File to document who is responsible for the module. Before any merge request is merged, an owner should approve it. + +Example: + + +├── .terraform/ # Terraform working directory +├── modules/ +│ ├── ec2/ +│ │ ├── main.tf +│ │ ├── variables.tf +│ │ ├── outputs.tf +│ │ └── README.md +│ ├── s3/ +│ │ ├── main.tf +│ │ ├── variables.tf +│ │ ├── outputs.tf +│ │ └── README.md +├── examples/ +│ ├── ec2/ +│ │ ├── main.tf +│ │ ├── variables.tf +│ │ ├── outputs.tf +│ │ └── README.md +│ ├── s3/ +│ │ ├── main.tf +│ │ ├── variables.tf +│ │ ├── outputs.tf +│ │ └── README.md +├── scripts/ +│ ├── configure_instance.sh +├── templates/ +│ ├── user_data.tftpl +├── files/ +│ ├── application.conf +├── helpers/ +│ ├── backup_database.sh +├── terraform.tfvars +├── envs/ +│ ├── dev/ +│ │ ├── terraform.tfvars +│ ├── prod/ +│ │ ├── terraform.tfvars +├── main.tf +├── variables.tf +├── outputs.tf +├── providers.tf +├── versions.tf +├── locals.tf +└── .gitignore + + +### 1.2 File Naming Conventions + +- `main.tf`: Primary entry point for resource definitions. +- `variables.tf`: Variable declarations. +- `outputs.tf`: Output declarations. +- `providers.tf`: Provider configuration. +- `versions.tf`: Terraform and provider version constraints. +- `locals.tf`: Local variable definitions. +- `*.tfvars`: Variable value files. Use `terraform.tfvars` for default values and `env/.tfvars` for environment-specific values. + +### 1.3 Module Organization + +- **Reusable Modules**: Create modules for common infrastructure components (e.g., EC2 instances, S3 buckets, VPCs). +- **Module Composition**: Compose complex infrastructure by combining smaller, reusable modules. +- **Module Versioning**: Use version control (e.g., Git tags) to manage module versions. +- **Module Naming**: Module repositories must use the format `terraform--`. For example: `terraform-aws-ec2` +- **Module sources**: When referencing another module on a public registry, pin the Git commit hash by pointing directly to the underlying Git repo URL. Example: + + terraform + module "lambda" { + source = "github.com/terraform-aws-modules/terraform-aws-lambda.git?ref=e78cdf1f82944897ca6e30d6489f43cf24539374" # --> v4.18.0 + } + + +### 1.4 Component Architecture + +- **Layered Architecture**: Separate infrastructure into logical layers (e.g., network, compute, data). +- **Loose Coupling**: Design components to be loosely coupled, minimizing dependencies between them. +- **Clear Abstractions**: Use modules to create clear abstractions for complex infrastructure components. + +### 1.5 Code Splitting Strategies + +- **Functional Decomposition**: Split code into smaller, manageable functions. +- **Resource Grouping**: Group related resources into separate files or modules. +- **Environment Separation**: Use separate files or modules for environment-specific configurations. +- **Service-Based Separation**: Separate resources by service (e.g., `iam.tf`, `ec2.tf`, `s3.tf`) only if they exceed 150 lines in `main.tf`. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns Specific to AWS + +- **Immutable Infrastructure**: Deploy new infrastructure components instead of modifying existing ones. +- **Infrastructure as Code (IaC)**: Manage infrastructure through code, using tools like Terraform. +- **Blue/Green Deployments**: Deploy new versions of applications alongside existing ones, then switch traffic. +- **Canary Deployments**: Deploy new versions of applications to a small subset of users before rolling them out to everyone. + +### 2.2 Recommended Approaches for Common Tasks + +- **Creating VPCs**: Use the `terraform-aws-modules/vpc/aws` module for creating VPCs. +- **Creating EC2 Instances**: Use the `terraform-aws-modules/ec2-instance/aws` module for creating EC2 instances. +- **Creating S3 Buckets**: Use the `terraform-aws-modules/s3-bucket/aws` module for creating S3 buckets. +- **Managing IAM Roles**: Use the `aws_iam_role` and `aws_iam_policy` resources to manage IAM roles and policies. +- **Using Data Sources**: Utilize data sources (e.g., `aws_ami`, `aws_vpc`) to fetch information about existing AWS resources. +- **Resource Tagging**: Tag all AWS resources for cost allocation, automation, and identification. +- **State File Storage**: Use a remote backend (e.g., S3 with DynamoDB locking) for storing Terraform state. + +### 2.3 Anti-patterns and Code Smells to Avoid + +- **Hardcoding Values**: Avoid hardcoding values in your code. Use variables instead. +- **Storing Secrets in Code**: Never store secrets (e.g., passwords, API keys) in your code. Use AWS Secrets Manager or other secret management solutions. +- **Overly Complex Modules**: Avoid creating modules that are too complex and difficult to understand. +- **Ignoring Errors**: Always handle errors gracefully. +- **Manual Infrastructure Changes**: Avoid making manual changes to infrastructure outside of Terraform. +- **Using embedded resource attributes**: avoid using these embedded resource attributes and instead you should use the unique resource to attach that pseudo-resource. These resource relationships can cause chicken/egg issues that are unique per resource. + +### 2.4 State Management Best Practices + +- **Remote Backend**: Use a remote backend (e.g., S3 with DynamoDB locking) for storing Terraform state. This allows for collaboration and prevents data loss. +- **State Locking**: Enable state locking to prevent concurrent modifications to the state file. +- **State Encryption**: Encrypt the state file at rest and in transit. +- **Workspace Management**: Use Terraform workspaces to manage multiple environments (e.g., development, staging, production). +- **Secure State Access**: Restrict access to the state file to authorized users and systems only. Consider implementing least privilege principles. +- **Versioning**: Implement a mechanism for versioning your state files, which could include frequent backups and using version control systems. +- **Avoid local state**: Never store Terraform state locally. + +### 2.5 Error Handling Patterns + +- **Validation Rules**: Use validation rules within variable definitions to check input values. +- **Error Propagation**: Propagate errors up the call stack to be handled at a higher level. +- **Retry Logic**: Implement retry logic for transient errors (e.g., network errors, API throttling). +- **Logging**: Log errors and warnings to aid in debugging. +- **Custom Error Messages**: Provide clear and informative error messages to users. +- **Panic Handling**: Avoid using `panic()` function that stops the execution, implement error handling. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +- **Efficient Resource Usage**: Use the appropriate instance types and storage classes for your workloads. +- **Caching**: Implement caching to reduce latency and improve performance. +- **Load Balancing**: Use load balancing to distribute traffic across multiple instances. +- **Content Delivery Networks (CDNs)**: Use CDNs to deliver static content closer to users. +- **Database Optimization**: Optimize database queries and indexing for faster performance. +- **Parallelization**: Use `for_each` and `count` to parallelize resource creation and modification. + +### 3.2 Memory Management + +- **Resource Limits**: Set resource limits on EC2 instances and other AWS resources. +- **Memory Profiling**: Use memory profiling tools to identify memory leaks and optimize memory usage. +- **Garbage Collection**: Configure garbage collection settings for your applications. + +### 3.3 Rendering Optimization + +- **Minimize Rendering**: Minimize the amount of data that needs to be rendered. +- **Efficient Templates**: Use efficient templates to generate configuration files. + +### 3.4 Bundle Size Optimization + +- **Code Splitting**: Split code into smaller bundles that can be loaded on demand. +- **Tree Shaking**: Remove unused code from your bundles. +- **Minification**: Minify your code to reduce bundle sizes. +- **Compression**: Compress your bundles to reduce download times. + +### 3.5 Lazy Loading Strategies + +- **Lazy Loading of Resources**: Load resources only when they are needed. +- **Virtualization**: Use virtualization to reduce the number of resources that need to be loaded. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and How to Prevent Them + +- **IAM Misconfiguration**: Prevent IAM misconfiguration by following the principle of least privilege. +- **Unsecured S3 Buckets**: Secure S3 buckets by enabling encryption, access logging, and versioning. +- **Security Group Misconfiguration**: Configure security groups to allow only necessary traffic. +- **SQL Injection**: Prevent SQL injection by using parameterized queries and input validation. +- **Cross-Site Scripting (XSS)**: Prevent XSS by sanitizing user input and output. +- **Cross-Site Request Forgery (CSRF)**: Prevent CSRF by using anti-CSRF tokens. +- **Exposed Secrets**: Avoid exposing secrets by using AWS Secrets Manager and IAM roles. + +### 4.2 Input Validation + +- **Validate User Input**: Validate user input to prevent malicious code from being injected into your application. +- **Use Data Types**: Use appropriate data types for your variables to prevent type-related errors. +- **Regular Expressions**: Use regular expressions to validate complex input patterns. + +### 4.3 Authentication and Authorization Patterns + +- **IAM Roles**: Use IAM roles to grant permissions to AWS resources. +- **Multi-Factor Authentication (MFA)**: Enable MFA for all user accounts. +- **Principle of Least Privilege**: Grant users only the permissions they need to perform their tasks. +- **AWS Cognito**: Use AWS Cognito to manage user authentication and authorization. +- **API Gateway Authorization**: Implement authorization at the API Gateway level to control access to your APIs. + +### 4.4 Data Protection Strategies + +- **Encryption at Rest**: Encrypt data at rest using AWS KMS, S3 encryption, or other encryption solutions. +- **Encryption in Transit**: Encrypt data in transit using HTTPS, TLS, or other encryption protocols. +- **Data Masking**: Mask sensitive data to prevent unauthorized access. +- **Data Redaction**: Redact sensitive data from logs and other output. + +### 4.5 Secure API Communication + +- **HTTPS**: Use HTTPS for all API communication. +- **API Keys**: Use API keys to authenticate API requests. +- **OAuth 2.0**: Use OAuth 2.0 for authorization. +- **Rate Limiting**: Implement rate limiting to prevent abuse. +- **Web Application Firewall (WAF)**: Use a WAF to protect your APIs from common web attacks. + +## 5. Testing Approaches + +### 5.1 Unit Testing Strategies + +- **Test Individual Modules**: Unit test individual Terraform modules to ensure they are working correctly. +- **Mock AWS Resources**: Mock AWS resources to isolate your modules from external dependencies. +- **Test Variable Validation**: Verify variable validation rules work as expected. +- **Test Output Values**: Check that module outputs are correct. + +### 5.2 Integration Testing + +- **Test Module Integration**: Test how modules interact with each other. +- **Deploy to Test Environment**: Deploy your infrastructure to a test environment and verify that it is working correctly. +- **Automated Tests**: Write automated tests to verify that your infrastructure is working correctly. + +### 5.3 End-to-End Testing + +- **Test Full Infrastructure**: Test the entire infrastructure from end to end. +- **Simulate User Behavior**: Simulate user behavior to verify that your infrastructure is working correctly. +- **Monitor Performance**: Monitor performance to identify any performance bottlenecks. + +### 5.4 Test Organization + +- **Separate Test Directory**: Create a separate directory for your tests. +- **Test Naming Conventions**: Use clear naming conventions for your tests. +- **Test Documentation**: Document your tests to explain what they are testing. + +### 5.5 Mocking and Stubbing + +- **Use Mock Objects**: Use mock objects to isolate your code from external dependencies. +- **Use Stub Objects**: Use stub objects to provide canned responses to external dependencies. +- **Terratest**: Utilize Terratest for creating test environments and performing infrastructure tests. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes Developers Make + +- **Forgetting to Enable State Locking**: Enabling state locking is crucial for preventing corruption and ensuring integrity when multiple users work on the same Terraform project. +- **Overlooking IAM Permissions**: Ensuring the Terraform service account has the necessary IAM permissions to create, modify, and delete resources is vital. +- **Incorrect Resource Dependencies**: Managing resource dependencies correctly to prevent errors during creation or deletion. +- **Handling Sensitive Data Insecurely**: Implementing robust measures for managing sensitive data, such as API keys and passwords, by using AWS Secrets Manager or other appropriate services. +- **Neglecting to Monitor Infrastructure**: Setting up comprehensive monitoring and alerting to detect and respond to infrastructure issues promptly. +- **Using hardcoded values**: Always use variables + +### 6.2 Edge Cases to be Aware Of + +- **Resource Limits**: Be aware of AWS resource limits and plan accordingly. +- **Availability Zones**: Understand the implications of availability zones and design your infrastructure to be resilient to AZ failures. +- **Data Consistency**: Understand the consistency models of different AWS services and design your applications accordingly. +- **API Throttling**: Be aware of API throttling limits and implement retry logic. + +### 6.3 Version-Specific Issues + +- **Terraform Version Compatibility**: Ensure that your Terraform code is compatible with the version of Terraform you are using. +- **Provider Version Compatibility**: Ensure that your Terraform providers are compatible with the version of Terraform you are using. +- **AWS API Version Compatibility**: Be aware of changes to the AWS API and update your code accordingly. + +### 6.4 Compatibility Concerns + +- **Backwards Compatibility**: Ensure that your changes are backwards compatible. +- **Cross-Region Compatibility**: Ensure that your infrastructure is compatible across different AWS regions. +- **Cross-Account Compatibility**: Ensure that your infrastructure is compatible across different AWS accounts. + +### 6.5 Debugging Strategies + +- **Terraform Plan**: Use `terraform plan` to preview changes before applying them. +- **Terraform Show**: Use `terraform show` to inspect the current state of your infrastructure. +- **Terraform Graph**: Use `terraform graph` to visualize the dependencies between resources. +- **Terraform Console**: Use `terraform console` to evaluate Terraform expressions. +- **AWS CloudTrail**: Use AWS CloudTrail to audit API calls and identify errors. +- **CloudWatch Logs**: Use CloudWatch Logs to collect and analyze application logs. +- **Enable Debug Logging**: Enable debug logging to get more detailed information about what Terraform is doing. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +- **Terraform CLI**: The Terraform command-line interface. +- **IDE with Terraform Support**: Use an IDE with Terraform support (e.g., VS Code with the Terraform extension). +- **tfenv**: Use tfenv to manage multiple versions of Terraform. +- **AWS CLI**: The AWS command-line interface. +- **jq**: A command-line JSON processor. + +### 7.2 Build Configuration + +- **Makefile**: Use a Makefile to automate common build tasks (e.g., `terraform init`, `terraform plan`, `terraform apply`). +- **.terraformignore**: Create a `.terraformignore` to prevent files from being included in the Terraform package. + +### 7.3 Linting and Formatting + +- **terraform fmt**: Use `terraform fmt` to automatically format your Terraform code. +- **terraform validate**: Use `terraform validate` to validate your Terraform code. +- **tflint**: Use `tflint` to lint your Terraform code. +- **pre-commit hooks**: Enforce code quality standards using pre-commit hooks. + +### 7.4 Deployment Best Practices + +- **Automated Deployments**: Use automated deployments to reduce errors and improve consistency. +- **Infrastructure as Code (IaC)**: Manage infrastructure as code using tools like Terraform. +- **Immutable Infrastructure**: Deploy new infrastructure components instead of modifying existing ones. +- **Blue/Green Deployments**: Deploy new versions of applications alongside existing ones, then switch traffic. +- **Canary Deployments**: Deploy new versions of applications to a small subset of users before rolling them out to everyone. + +### 7.5 CI/CD Integration + +- **Integrate with CI/CD Pipeline**: Integrate Terraform with your CI/CD pipeline. +- **Automated Testing**: Run automated tests as part of your CI/CD pipeline. +- **Automated Deployments**: Automatically deploy your infrastructure as part of your CI/CD pipeline. +- **Version Control**: Store your Terraform code in version control. +- **Terraform Cloud/Enterprise**: Use Terraform Cloud/Enterprise for remote state management and collaboration. + +This document provides a comprehensive set of best practices for AWS development using Terraform. By following these guidelines, you can ensure your code is maintainable, secure, and efficient. \ No newline at end of file diff --git a/.cursor/rules/axios.mdc b/.cursor/rules/axios.mdc new file mode 100644 index 000000000..1e6dfdc40 --- /dev/null +++ b/.cursor/rules/axios.mdc @@ -0,0 +1,185 @@ +--- +description: This rule provides comprehensive guidelines for using Axios effectively, covering best practices, security considerations, and performance optimization. It aims to improve code quality, maintainability, and overall application resilience when working with Axios. +globs: *.js,*.jsx,*.ts,*.tsx +--- +- **Avoid hardcoding URLs**: Store API endpoints in configuration files or environment variables to enhance maintainability and avoid redundancy. +- **Centralize API Logic**: Create dedicated modules or services to encapsulate Axios instances and request logic. This promotes reusability and simplifies debugging. +- **Handle Errors Gracefully**: Implement comprehensive error handling using try-catch blocks, response interceptors, and centralized error logging. Display user-friendly error messages. +- **Use Asynchronous Functions**: Leverage `async/await` or Promises for cleaner and more manageable asynchronous code when making Axios requests. +- **Optimize with Request Interceptors**: Utilize request interceptors for tasks like adding authentication tokens, logging requests, or transforming request data. Response interceptors can be used for error handling or data transformation. +- **Ensure HTTP Requests When Necessary**: Verify that requests are only made when truly needed, preventing unnecessary network traffic and improving performance. Consider caching strategies. + +### 1. Code Organization and Structure: + + - **Directory Structure Best Practices**: + - Create an `api` or `services` directory to house all API-related code. + - Organize API modules based on resources or features (e.g., `api/users.js`, `api/products.js`). + - Separate Axios instance configuration into a dedicated file (e.g., `api/axios-config.js`). + + - **File Naming Conventions**: + - Use descriptive names for API modules (e.g., `userApi.js`, `productService.js`). + - Follow a consistent naming convention (e.g., camelCase for variables, PascalCase for components). + + - **Module Organization**: + - Export individual API functions from each module (e.g., `getUser`, `createUser`). + - Use named exports for better code readability and maintainability. + + - **Component Architecture**: + - Create reusable components for handling data fetching and display. + - Decouple components from specific API calls to improve reusability. + - Utilize custom hooks to encapsulate data fetching logic. + + - **Code Splitting Strategies**: + - Implement lazy loading for API modules that are not immediately required. + - Use dynamic imports to load modules on demand. + - Consider route-based code splitting for larger applications. + +### 2. Common Patterns and Anti-patterns: + + - **Design Patterns Specific to Axios**: + - **Singleton Pattern**: Use a singleton pattern for the Axios instance to ensure a single configuration across the application. + - **Adapter Pattern**: Create an adapter layer to transform API responses into a consistent format. + - **Factory Pattern**: Use a factory pattern to create different Axios instances with specific configurations. + + - **Recommended Approaches for Common Tasks**: + - **Authentication**: Use request interceptors to add authentication headers (e.g., JWT tokens). + - **Error Handling**: Implement a centralized error handling function to log errors and display user-friendly messages. + - **Data Transformation**: Use response interceptors to transform data before it reaches the components. + + - **Anti-patterns and Code Smells to Avoid**: + - **Hardcoding URLs**: Avoid hardcoding API endpoints directly in the code. + - **Duplicated Request Logic**: Don't repeat API request logic across multiple components. + - **Ignoring Errors**: Never ignore errors returned by Axios; always handle them appropriately. + + - **State Management Best Practices**: + - Use a state management library (e.g., Redux, Zustand, Recoil) to manage API data. + - Store API responses in a normalized format to improve performance. + - Utilize selectors to derive data from the store efficiently. + + - **Error Handling Patterns**: + - Implement a global error boundary to catch unexpected errors. + - Display user-friendly error messages based on the error type. + - Log errors to a central logging service for monitoring and debugging. + +### 3. Performance Considerations: + + - **Optimization Techniques**: + - **Caching**: Implement caching mechanisms to avoid redundant API calls. + - **Request Debouncing**: Debounce frequent API requests to reduce server load. + - **Request Cancellation**: Cancel pending requests when the component unmounts or the user navigates away. + + - **Memory Management**: + - Clear Axios instances when they are no longer needed. + - Avoid creating unnecessary Axios instances. + + - **Rendering Optimization**: + - Use memoization techniques to prevent unnecessary re-renders. + - Implement pagination or virtualized lists for large datasets. + + - **Bundle Size Optimization**: + - Use tree shaking to remove unused code from the Axios library. + - Minify and compress the JavaScript bundle. + + - **Lazy Loading Strategies**: + - Lazy load API modules that are not immediately required. + - Use dynamic imports to load modules on demand. + +### 4. Security Best Practices: + + - **Common Vulnerabilities and How to Prevent Them**: + - **Cross-Site Scripting (XSS)**: Sanitize user input to prevent XSS attacks. + - **Cross-Site Request Forgery (CSRF)**: Implement CSRF protection mechanisms. + - **Man-in-the-Middle (MITM) Attacks**: Use HTTPS to encrypt communication between the client and the server. + + - **Input Validation**: + - Validate user input on both the client and server sides. + - Use appropriate validation libraries to prevent injection attacks. + + - **Authentication and Authorization Patterns**: + - Use a secure authentication protocol (e.g., OAuth 2.0, OpenID Connect). + - Implement role-based access control (RBAC) to restrict access to sensitive data. + + - **Data Protection Strategies**: + - Encrypt sensitive data at rest and in transit. + - Use appropriate data masking techniques to protect sensitive information. + + - **Secure API Communication**: + - Use HTTPS for all API communication. + - Implement rate limiting to prevent abuse. + - Monitor API traffic for suspicious activity. + +### 5. Testing Approaches: + + - **Unit Testing Strategies**: + - Mock Axios requests using libraries like `axios-mock-adapter` or `nock`. + - Test individual API functions in isolation. + - Verify that requests are made with the correct parameters and headers. + + - **Integration Testing**: + - Test the interaction between API modules and components. + - Verify that data is fetched and displayed correctly. + - Use a testing framework like Jest or Mocha. + + - **End-to-End Testing**: + - Test the entire application workflow from the user's perspective. + - Use a testing framework like Cypress or Selenium. + + - **Test Organization**: + - Organize tests by module or feature. + - Use descriptive names for test cases. + - Follow a consistent testing style. + + - **Mocking and Stubbing**: + - Use mocking to isolate components from external dependencies. + - Use stubbing to replace API responses with predefined data. + +### 6. Common Pitfalls and Gotchas: + + - **Frequent Mistakes Developers Make**: + - Forgetting to handle errors. + - Using incorrect HTTP methods. + - Hardcoding URLs. + - Not setting appropriate headers. + + - **Edge Cases to Be Aware Of**: + - Network errors. + - Server downtime. + - Rate limiting. + - API versioning. + + - **Version-Specific Issues**: + - Be aware of breaking changes between Axios versions. + - Consult the Axios documentation for specific version information. + + - **Compatibility Concerns**: + - Ensure that Axios is compatible with the target browsers and environments. + - Use polyfills if necessary. + + - **Debugging Strategies**: + - Use the browser's developer tools to inspect network requests. + - Log Axios requests and responses to the console. + - Use a debugging tool like VS Code's debugger. + +### 7. Tooling and Environment: + + - **Recommended Development Tools**: + - VS Code with the ESLint and Prettier extensions. + - Axios DevTools for Chrome and Firefox. + - Postman or Insomnia for testing API endpoints. + + - **Build Configuration**: + - Use a build tool like Webpack or Parcel to bundle the application. + - Configure the build tool to optimize the bundle size. + + - **Linting and Formatting**: + - Use ESLint with a consistent set of rules to enforce code quality. + - Use Prettier to format the code automatically. + + - **Deployment Best Practices**: + - Deploy the application to a secure hosting environment. + - Configure HTTPS for all API communication. + - Monitor the application for errors and performance issues. + + - **CI/CD Integration**: + - Integrate automated testing and deployment into the CI/CD pipeline. + - Use a CI/CD tool like Jenkins, GitLab CI, or CircleCI. \ No newline at end of file diff --git a/.cursor/rules/azure-pipelines.mdc b/.cursor/rules/azure-pipelines.mdc new file mode 100644 index 000000000..ebbb6031c --- /dev/null +++ b/.cursor/rules/azure-pipelines.mdc @@ -0,0 +1,176 @@ +--- +description: Comprehensive guidelines for Azure Pipelines, covering code structure, security, performance, testing, and deployment. Provides actionable advice for developers working with Azure Pipelines YAML definitions to enhance code quality and CI/CD processes. +globs: *.yml,*.yaml +--- +- # General Guidelines + - Utilize YAML pipelines for version control and ease of replication. + - Integrate security practices throughout the pipeline using Azure Key Vault for secret management. + - Implement role-based access controls (RBAC) to limit permissions. + - Establish monitoring and alerting mechanisms for maintaining pipeline health and addressing issues. + - Use Infrastructure as Code (IaC) for consistent deployments. + +- # 1. Code Organization and Structure + - **Directory Structure Best Practices:** + - Organize pipeline definitions into a dedicated directory (e.g., `azure-pipelines`). + - Use subdirectories to categorize pipelines based on application, environment, or team (e.g., `azure-pipelines/app1/dev`, `azure-pipelines/app1/prod`). + - Store reusable templates and scripts in a separate `templates` or `scripts` directory. + - Consider using a `modules` directory for custom modules or extensions related to pipeline functionality. + - **File Naming Conventions:** + - Use descriptive names for pipeline definitions (e.g., `ci-cd-app1-dev.yml`, `deploy-to-staging.yml`). + - Prefix or suffix template files with `template` or `tmpl` (e.g., `build-template.yml`, `deploy-tmpl.yml`). + - Use consistent naming for variables, parameters, and stages within YAML files. + - **Module Organization Best Practices:** + - Break down complex pipelines into smaller, reusable modules using templates. + - Use parameters to configure templates for different scenarios. + - Store modules in a central repository for sharing across projects. + - **Component Architecture Recommendations:** + - Design pipelines with a clear separation of concerns (e.g., build, test, deploy). + - Use stages to represent distinct phases of the CI/CD process. + - Leverage tasks to perform specific actions within each stage. + - Define clear inputs and outputs for each component (stage/task). + - **Code Splitting Strategies:** + - Split large YAML files into smaller, more manageable files using the `extends` keyword. + - Use templates to define reusable pipeline components. + - Implement parameterization to customize pipeline behavior without duplicating code. + +- # 2. Common Patterns and Anti-patterns + - **Design Patterns:** + - **Template Pattern:** Define a base template with placeholders for customization, allowing for consistent pipeline structure with varying configurations. + - **Strategy Pattern:** Use parameters to select different deployment strategies (e.g., blue-green, canary) within a single pipeline. + - **Chain of Responsibility:** Implement a sequence of tasks or stages, where each component handles a specific aspect of the deployment process. + - **Recommended Approaches for Common Tasks:** + - **Secret Management:** Use Azure Key Vault to store sensitive information such as passwords, API keys, and connection strings. Reference secrets in pipelines using variables or tasks. + - **Environment Configuration:** Define environment-specific variables and settings using variable groups. Use conditional execution to apply different configurations based on the target environment. + - **Artifact Management:** Publish and consume artifacts using the `PublishBuildArtifacts` and `DownloadBuildArtifacts` tasks. Use artifact feeds to store and manage dependencies. + - **Rollback Strategies:** Implement rollback mechanisms to revert to a previous version in case of deployment failures. Use deployment slots or blue-green deployments for seamless rollbacks. + - **Anti-patterns and Code Smells:** + - Hardcoding secrets in pipeline definitions. + - Duplicating code across multiple pipelines. + - Overly complex and monolithic YAML files. + - Lack of error handling and logging. + - Insufficient testing and validation. + - **State Management Best Practices:** + - Use Azure DevOps variables or variable groups to store pipeline state. + - Consider using external storage (e.g., Azure Blob Storage, Azure Table Storage) for persistent state. + - Implement idempotency to ensure that pipeline operations can be safely retried. + - **Error Handling Patterns:** + - Use the `try...catch` construct to handle exceptions within pipeline tasks. + - Implement retry mechanisms for transient failures. + - Configure alerts and notifications for pipeline failures. + +- # 3. Performance Considerations + - **Optimization Techniques:** + - **Parallel Execution:** Use parallel jobs to run multiple tasks concurrently, reducing overall pipeline execution time. + - **Caching:** Cache dependencies and build artifacts to speed up subsequent builds. + - **Agent Selection:** Choose the appropriate agent size and type based on the workload requirements. + - **Task Optimization:** Optimize individual tasks to minimize execution time (e.g., using efficient scripts, optimizing database queries). + - **Memory Management:** + - Monitor pipeline memory usage and identify potential memory leaks. + - Use appropriate data structures and algorithms to minimize memory consumption. + - Optimize image sizes for containerized applications. + - **Bundle Size Optimization:** + - Minimize the size of build artifacts by removing unnecessary files and dependencies. + - Use code splitting and tree shaking techniques to reduce bundle size. + - Compress artifacts before publishing. + - **Lazy Loading:** + - Implement lazy loading for large files or datasets to improve pipeline startup time. + +- # 4. Security Best Practices + - **Common Vulnerabilities and Prevention:** + - **Credential Leaks:** Avoid storing secrets in pipeline definitions or source code. Use Azure Key Vault for secure secret management. + - **Command Injection:** Sanitize user inputs and avoid executing untrusted code. Use parameterized tasks and scripts to prevent command injection attacks. + - **Unauthorized Access:** Implement RBAC to restrict access to sensitive resources and operations. + - **Dependency Vulnerabilities:** Regularly scan dependencies for known vulnerabilities and update them to the latest versions. + - **Input Validation:** + - Validate all user inputs to prevent injection attacks and other security vulnerabilities. + - Use regular expressions or other validation techniques to ensure that inputs conform to expected formats. + - Implement input sanitization to remove or escape potentially malicious characters. + - **Authentication and Authorization:** + - Use service principals or managed identities for authentication with Azure resources. + - Implement RBAC to control access to pipelines, resources, and environments. + - Enforce multi-factor authentication (MFA) for privileged accounts. + - **Data Protection:** + - Encrypt sensitive data at rest and in transit. + - Use secure protocols (e.g., HTTPS, TLS) for API communication. + - Mask sensitive information in pipeline logs and outputs. + - **Secure API Communication:** + - Use API keys or access tokens for authentication. + - Implement rate limiting to prevent denial-of-service attacks. + - Validate API responses to prevent data corruption. + +- # 5. Testing Approaches + - **Unit Testing:** + - Write unit tests for individual pipeline tasks and components. + - Use mocking frameworks to isolate components and simulate dependencies. + - Ensure that unit tests cover all critical code paths. + - **Integration Testing:** + - Perform integration tests to verify the interaction between different pipeline stages and tasks. + - Test the integration with external services and resources. + - Use test environments that closely resemble production environments. + - **End-to-end Testing:** + - Conduct end-to-end tests to validate the entire CI/CD pipeline from code commit to deployment. + - Use automated testing frameworks to automate end-to-end tests. + - Test the deployed application in a production-like environment. + - **Test Organization:** + - Organize tests into a dedicated `tests` directory. + - Use a consistent naming convention for test files and methods. + - Group tests based on functionality or component. + - **Mocking and Stubbing:** + - Use mocking frameworks (e.g., Moq, NSubstitute) to create mock objects for dependencies. + - Use stubbing to replace complex dependencies with simple, predictable implementations. + +- # 6. Common Pitfalls and Gotchas + - **Frequent Mistakes:** + - Using incorrect YAML syntax. + - Forgetting to parameterize reusable components. + - Neglecting error handling and logging. + - Insufficient testing. + - Ignoring security best practices. + - **Edge Cases:** + - Handling large files or datasets. + - Dealing with complex dependencies. + - Managing concurrency and race conditions. + - Recovering from catastrophic failures. + - **Version-Specific Issues:** + - Be aware of breaking changes in Azure DevOps updates. + - Test pipelines after upgrading Azure DevOps versions. + - Use version control to track changes to pipeline definitions. + - **Compatibility Concerns:** + - Ensure compatibility between pipeline tasks and agent versions. + - Verify compatibility with external services and resources. + - **Debugging Strategies:** + - Use pipeline logs to identify errors and warnings. + - Enable verbose logging for detailed troubleshooting. + - Use remote debugging to step through pipeline execution. + +- # 7. Tooling and Environment + - **Recommended Development Tools:** + - Visual Studio Code with the Azure Pipelines extension. + - Azure CLI. + - PowerShell. + - YAML linters and validators. + - **Build Configuration Best Practices:** + - Use a consistent build configuration across environments. + - Store build configuration in version control. + - Use environment variables to customize build behavior. + - **Linting and Formatting:** + - Use YAML linters (e.g., yamllint) to enforce consistent formatting and syntax. + - Configure linters to automatically fix formatting issues. + - **Deployment Best Practices:** + - Use deployment slots for zero-downtime deployments. + - Implement rollback mechanisms for failed deployments. + - Monitor deployed applications for performance and errors. + - **CI/CD Integration:** + - Integrate Azure Pipelines with source control systems (e.g., Azure Repos, GitHub). + - Automate build, test, and deployment processes. + - Use triggers to automatically start pipelines on code changes. + +- # Additional Best Practices + - **Infrastructure as Code (IaC):** Use tools like ARM templates or Terraform for consistent deployments. + - **Automated Testing:** Automate builds, run tests, and perform code quality checks with each commit. + - **Secret Management:** Securely manage secrets using Azure Key Vault. + - **Monitoring and Alerting:** Implement robust monitoring and alerting for pipeline health. + - **Role-Based Access Control (RBAC):** Enforce RBAC to limit access to sensitive resources. + - **Regular Audits:** Perform regular security audits to identify and address vulnerabilities. + +By adhering to these guidelines, developers can create robust, secure, and efficient Azure Pipelines for continuous integration and continuous delivery. \ No newline at end of file diff --git a/.cursor/rules/azure.mdc b/.cursor/rules/azure.mdc new file mode 100644 index 000000000..d89a346c1 --- /dev/null +++ b/.cursor/rules/azure.mdc @@ -0,0 +1,347 @@ +--- +description: This rule provides comprehensive best practices for developing Azure applications, covering code organization, security, performance, testing, and common pitfalls. It aims to improve code quality, security posture, and overall efficiency when working with the Azure ecosystem. +globs: *.az,*.tf,*.bicep,*.py,*.js,*.ts,*.json,*.yml,*.yaml,*.ps1,*.sh,*.cs,*.java +--- +# Azure Library Best Practices and Coding Standards + +This document outlines the recommended best practices for developing applications and infrastructure using Azure services. It covers various aspects including code organization, security, performance, testing, and tooling to ensure robust, scalable, and secure solutions. + +## 1. Code Organization and Structure + +A well-organized codebase is crucial for maintainability, scalability, and collaboration. The following guidelines provide a structured approach to organizing your Azure projects. + +### Directory Structure Best Practices + +Adopt a modular and logical directory structure based on the application's architecture and components. + +* **`src/`**: Contains the source code of your application. + * **`modules/`**: Groups related functionalities into independent modules. + * **`components/`**: Contains reusable UI or logical components. + * **`services/`**: Encapsulates external service integrations (e.g., Azure Storage, Cosmos DB). + * **`models/`**: Defines data models and schemas. + * **`utils/`**: Contains utility functions and helper classes. +* **`infra/` or `terraform/` or `bicep/`**: Infrastructure-as-Code definitions (Terraform, Bicep, Azure Resource Manager templates). +* **`config/`**: Configuration files for different environments (development, staging, production). +* **`scripts/`**: Automation scripts for deployment, build processes, etc. +* **`tests/`**: Unit, integration, and end-to-end tests. +* **`docs/`**: Documentation for the project and its components. + +Example: + + +my-azure-project/ +├── src/ +│ ├── modules/ +│ │ ├── user-management/ +│ │ │ ├── ... +│ │ ├── data-processing/ +│ │ │ ├── ... +│ ├── components/ +│ │ ├── button/ +│ │ │ ├── ... +│ ├── services/ +│ │ ├── storage-service.js +│ │ ├── cosmosdb-service.js +│ ├── models/ +│ │ ├── user.js +│ ├── utils/ +│ │ ├── logger.js +├── infra/ +│ ├── main.tf +│ ├── variables.tf +│ ├── outputs.tf +├── config/ +│ ├── development.json +│ ├── production.json +├── scripts/ +│ ├── deploy.sh +│ ├── build.sh +├── tests/ +│ ├── unit/ +│ ├── integration/ +│ ├── e2e/ +├── docs/ +│ ├── README.md +├── .gitignore +├── package.json + + +### File Naming Conventions + +Maintain consistency in file naming to improve readability and searchability. + +* Use descriptive names that reflect the file's purpose. +* Use a consistent case (e.g., camelCase or kebab-case). +* Use appropriate file extensions (e.g., `.js`, `.ts`, `.py`, `.tf`, `.bicep`). +* For React components, use `ComponentName.jsx` or `ComponentName.tsx`. + +Example: + +* `user-service.js` (for a user service module) +* `UserProfile.jsx` (for a user profile component) +* `storage-account.tf` (for a Terraform file defining a storage account) + +### Module Organization + +Divide the application into independent and reusable modules based on functionality. + +* Each module should have a clear responsibility and a well-defined interface. +* Minimize dependencies between modules to promote loose coupling. +* Consider using a module bundler (e.g., Webpack, Parcel) to manage dependencies and optimize the build process. + +Example: + +A `user-management` module could contain components and services related to user authentication, authorization, and profile management. + +### Component Architecture + +For UI-based applications, adopt a component-based architecture (e.g., React, Angular, Vue.js). + +* Divide the UI into reusable components with clear inputs and outputs. +* Follow the principles of single responsibility and separation of concerns. +* Use a component library (e.g., Material UI, Ant Design) to accelerate development and maintain consistency. + +### Code Splitting Strategies + +Improve initial load time by splitting the application into smaller chunks that can be loaded on demand. + +* Use dynamic imports to load modules only when they are needed. +* Configure your module bundler to create separate chunks for different parts of the application. +* Consider using route-based code splitting for single-page applications. + +## 2. Common Patterns and Anti-patterns + +Understanding common patterns and anti-patterns is essential for building efficient, maintainable, and reliable Azure applications. + +### Design Patterns Specific to Azure + +* **Retry Pattern:** Implement retry logic to handle transient failures when interacting with Azure services. +* **Circuit Breaker Pattern:** Prevent cascading failures by temporarily blocking access to a service that is experiencing problems. +* **Queue-Based Load Leveling:** Use queues (e.g., Azure Queue Storage, Azure Service Bus) to decouple components and handle bursts of traffic. +* **Cache-Aside Pattern:** Use a cache (e.g., Azure Cache for Redis) to improve performance by storing frequently accessed data. +* **Gateway Aggregation Pattern:** Expose multiple backend services through a single API gateway (e.g., Azure API Management). + +### Recommended Approaches for Common Tasks + +* **Configuration Management:** Use Azure App Configuration to manage application settings and feature flags. +* **Secret Management:** Use Azure Key Vault to securely store and manage secrets, keys, and certificates. +* **Logging and Monitoring:** Use Azure Monitor to collect and analyze logs, metrics, and traces. +* **Identity and Access Management:** Use Azure Active Directory (Azure AD) for authentication and authorization. +* **Data Storage:** Choose the appropriate Azure storage service based on your data requirements (e.g., Azure Blob Storage for unstructured data, Azure Cosmos DB for NoSQL data, Azure SQL Database for relational data). + +### Anti-Patterns and Code Smells to Avoid + +* **Hardcoding Secrets:** Never hardcode secrets or API keys in your code. Use Azure Key Vault to manage them securely. +* **Ignoring Errors:** Always handle errors gracefully and log relevant information for debugging. +* **Over-Engineering:** Avoid unnecessary complexity and focus on delivering business value. +* **Long-Running Transactions:** Keep transactions short and avoid holding resources for extended periods. +* **Tight Coupling:** Design components and modules to be loosely coupled to improve maintainability and testability. +* **Monolithic Functions:** Avoid creating large, complex functions. Break them down into smaller, more manageable units. + +### State Management Best Practices + +* For serverless functions, design for statelessness whenever possible. Store state in external storage services like Azure Cosmos DB or Azure Storage. +* For web applications, use appropriate state management techniques (e.g., cookies, sessions, local storage, or dedicated state management libraries like Redux or Zustand). +* Consider using distributed caching (e.g., Azure Cache for Redis) to improve performance and scalability. + +### Error Handling Patterns + +* Use try-catch blocks to handle exceptions gracefully. +* Log errors with sufficient detail to aid in debugging. +* Implement a centralized error handling mechanism to catch and handle unhandled exceptions. +* Provide meaningful error messages to the user. +* Use Polly library for implementing resilience policies like retry, circuit breaker, and timeout. + +## 3. Performance Considerations + +Optimizing performance is crucial for delivering a responsive and scalable Azure application. + +### Optimization Techniques + +* **Caching:** Implement caching at various levels (e.g., browser, server, database) to reduce latency and improve performance. +* **Compression:** Enable compression (e.g., Gzip) to reduce the size of HTTP responses. +* **Connection Pooling:** Use connection pooling to reuse database connections and reduce connection overhead. +* **Content Delivery Network (CDN):** Use Azure CDN to cache static assets closer to users. +* **Database Optimization:** Optimize database queries and indexes to improve query performance. +* **Asynchronous Operations:** Use asynchronous operations to avoid blocking the main thread. + +### Memory Management + +* **Dispose Resources:** Dispose of disposable resources (e.g., database connections, file streams) properly to avoid memory leaks. +* **Garbage Collection:** Be aware of garbage collection behavior and avoid creating unnecessary objects. +* **Large Object Heap (LOH):** Be mindful of the LOH and avoid allocating large objects frequently. + +### Rendering Optimization + +* **Virtualization:** Use virtualization techniques (e.g., React Virtualized, Angular CDK Virtual Scroll) to efficiently render large lists of data. +* **Debouncing and Throttling:** Use debouncing and throttling to limit the frequency of event handlers. +* **Lazy Loading:** Load images and other resources only when they are visible on the screen. + +### Bundle Size Optimization + +* **Tree Shaking:** Use tree shaking to remove unused code from your bundles. +* **Code Splitting:** Split your code into smaller chunks that can be loaded on demand. +* **Minification:** Minify your code to reduce its size. +* **Image Optimization:** Optimize images by compressing them and using appropriate formats. + +### Lazy Loading Strategies + +* **Component-Based Lazy Loading:** Load components only when they are needed. +* **Route-Based Lazy Loading:** Load modules associated with specific routes only when those routes are accessed. +* **Image Lazy Loading:** Load images only when they are scrolled into view. + +## 4. Security Best Practices + +Security should be a top priority when developing Azure applications. + +### Common Vulnerabilities and How to Prevent Them + +* **SQL Injection:** Use parameterized queries or an ORM to prevent SQL injection attacks. +* **Cross-Site Scripting (XSS):** Sanitize user input and output to prevent XSS attacks. +* **Cross-Site Request Forgery (CSRF):** Use anti-CSRF tokens to prevent CSRF attacks. +* **Authentication and Authorization Flaws:** Implement robust authentication and authorization mechanisms. +* **Insecure Direct Object References (IDOR):** Validate that users have access to the resources they are requesting. +* **Security Misconfiguration:** Follow security best practices for configuring Azure services. +* **Sensitive Data Exposure:** Protect sensitive data by encrypting it at rest and in transit. + +### Input Validation + +* Validate all user input on both the client and server sides. +* Use a schema validation library (e.g., Joi, Yup) to define and enforce data validation rules. +* Sanitize user input to remove potentially malicious characters. + +### Authentication and Authorization Patterns + +* Use Azure Active Directory (Azure AD) for authentication and authorization. +* Implement role-based access control (RBAC) to restrict access to resources based on user roles. +* Use multi-factor authentication (MFA) to enhance security. +* Follow the principle of least privilege. +* Leverage Managed Identities for Azure resources to securely access other Azure services. + +### Data Protection Strategies + +* Encrypt sensitive data at rest using Azure Storage Service Encryption (SSE) or Azure Disk Encryption. +* Encrypt data in transit using TLS/SSL. +* Use Azure Key Vault to store and manage encryption keys. +* Implement data masking and tokenization to protect sensitive data. + +### Secure API Communication + +* Use HTTPS for all API communication. +* Implement API authentication using API keys, tokens, or OAuth 2.0. +* Use rate limiting to prevent abuse. +* Use Azure API Management to manage and secure your APIs. + +## 5. Testing Approaches + +Thorough testing is essential for ensuring the quality and reliability of your Azure applications. + +### Unit Testing Strategies + +* Write unit tests for individual components and functions. +* Use a unit testing framework (e.g., Jest, Mocha, JUnit, pytest). +* Aim for high code coverage. +* Follow the Arrange-Act-Assert pattern. +* Use mocks and stubs to isolate the code under test. + +### Integration Testing + +* Write integration tests to verify the interaction between different components and services. +* Test the integration with Azure services (e.g., Azure Storage, Cosmos DB). +* Use a testing environment that closely resembles the production environment. + +### End-to-End Testing + +* Write end-to-end tests to verify the entire application workflow. +* Use a testing framework (e.g., Cypress, Selenium, Playwright). +* Automate end-to-end tests as part of your CI/CD pipeline. + +### Test Organization + +* Organize tests into a clear directory structure. +* Use descriptive test names. +* Group related tests together. + +### Mocking and Stubbing + +* Use mocks to simulate the behavior of external dependencies. +* Use stubs to provide predefined responses for API calls. +* Use a mocking framework (e.g., Mockito, Jest Mocks). + +## 6. Common Pitfalls and Gotchas + +Be aware of common pitfalls and gotchas to avoid making mistakes when developing Azure applications. + +### Frequent Mistakes Developers Make + +* **Insufficient Resource Sizing:** Underestimating the required resources can lead to performance issues. +* **Ignoring Azure Service Limits:** Being unaware of Azure service limits can cause unexpected errors. +* **Improper Error Handling:** Inadequate error handling can make it difficult to diagnose and resolve issues. +* **Lack of Monitoring:** Failing to monitor the application's performance and health can lead to undetected problems. +* **Neglecting Security:** Overlooking security best practices can expose the application to vulnerabilities. + +### Edge Cases to Be Aware Of + +* **Network Latency:** Be aware of network latency when interacting with Azure services. +* **Transient Faults:** Implement retry logic to handle transient faults. +* **Resource Contention:** Be aware of resource contention and design the application to handle it gracefully. +* **Data Consistency:** Understand the data consistency models of Azure storage services. + +### Version-Specific Issues + +* Be aware of breaking changes in Azure SDKs and APIs. +* Keep your dependencies up to date. +* Test your application with different versions of Azure services. + +### Compatibility Concerns + +* Ensure that your application is compatible with the target Azure environment. +* Test your application on different browsers and devices. + +### Debugging Strategies + +* Use Azure Monitor to collect and analyze logs, metrics, and traces. +* Use remote debugging to debug your application running in Azure. +* Use logging to track the flow of execution and identify errors. +* Use breakpoints to pause execution and inspect variables. + +## 7. Tooling and Environment + +Choosing the right tools and environment can significantly improve your development experience. + +### Recommended Development Tools + +* **Visual Studio Code:** A popular code editor with excellent support for Azure development. +* **Azure CLI:** A command-line tool for managing Azure resources. +* **Azure PowerShell:** A PowerShell module for managing Azure resources. +* **Terraform:** An infrastructure-as-code tool for provisioning and managing Azure resources. +* **Bicep:** A declarative language for deploying Azure resources. +* **Azure SDKs:** Libraries for interacting with Azure services from various programming languages. + +### Build Configuration + +* Use a build automation tool (e.g., Azure DevOps, Jenkins, GitHub Actions) to automate the build process. +* Configure your build process to run tests, linters, and code formatters. +* Use environment variables to configure your application for different environments. + +### Linting and Formatting + +* Use a linter (e.g., ESLint, Pylint) to enforce code style guidelines. +* Use a code formatter (e.g., Prettier, Black) to automatically format your code. +* Configure your editor to automatically run the linter and formatter on save. + +### Deployment Best Practices + +* Use Infrastructure-as-Code (IaC) to automate the deployment of Azure resources. +* Use a deployment pipeline (e.g., Azure Pipelines, GitHub Actions) to automate the deployment process. +* Use blue-green deployments or canary deployments to minimize downtime. +* Implement rollback strategies to quickly revert to a previous version if necessary. + +### CI/CD Integration + +* Integrate your CI/CD pipeline with Azure DevOps, GitHub Actions, or other CI/CD tools. +* Automate the build, test, and deployment processes. +* Use environment variables to configure your application for different environments. +* Implement automated testing to catch errors early in the development cycle. + +By following these best practices, you can build robust, scalable, secure, and maintainable Azure applications. \ No newline at end of file diff --git a/.cursor/rules/bash.mdc b/.cursor/rules/bash.mdc new file mode 100644 index 000000000..4605cb4d1 --- /dev/null +++ b/.cursor/rules/bash.mdc @@ -0,0 +1,292 @@ +--- +description: This rule enforces best practices and coding standards for Bash scripting to improve code quality, maintainability, and security. It covers naming conventions, formatting, error handling, security, and performance considerations. +globs: *.sh +--- +# Bash Best Practices and Coding Standards + +This document outlines the best practices and coding standards for writing Bash scripts. Following these guidelines will lead to more maintainable, readable, and robust code. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure Best Practices + +For large Bash projects, a well-defined directory structure is crucial. Consider the following structure: + + +project_root/ +├── bin/ # Executable scripts (main entry points) +├── lib/ # Library scripts (reusable functions and modules) +├── config/ # Configuration files (e.g., settings, environment variables) +├── data/ # Data files (e.g., CSV, JSON) +├── log/ # Log files generated by scripts +├── tmp/ # Temporary files used during script execution +├── tests/ # Unit and integration tests +├── docs/ # Documentation (e.g., README, design documents) +└── README.md # Project README + + +### 1.2. File Naming Conventions + +* **Executable Scripts:** Use descriptive names without extensions (e.g., `process_data`, `backup_system`). +* **Library Scripts:** Use `.sh` extension (e.g., `string_utils.sh`, `database_functions.sh`). For bash-specific code use `.bash`. +* **Configuration Files:** Use `.conf` or `.cfg` extensions (e.g., `app.conf`, `settings.cfg`). +* **Test Files:** Use `_test.sh` suffix (e.g., `string_utils_test.sh`). + +### 1.3. Module Organization + +* **Separate Concerns:** Divide your code into logical modules, each responsible for a specific task (e.g., database interaction, string manipulation, system monitoring). +* **Reusable Functions:** Place reusable functions in library scripts (`lib/`) and source them in your main scripts using `source lib/string_utils.sh` or `. lib/string_utils.sh`. +* **Modular Design:** Aim for a modular design where each module can be easily tested and reused in other projects. + +### 1.4. Component Architecture + +* **Functions as Components:** Treat functions as components with well-defined inputs (parameters) and outputs (return values or side effects). +* **Configuration Management:** Use configuration files to store settings that can be easily modified without changing the script's code. +* **Logging:** Implement a consistent logging mechanism to track script execution and errors. + +### 1.5. Code Splitting Strategies + +* **Function Decomposition:** Break down complex tasks into smaller, more manageable functions. +* **File Inclusion:** Use `source` or `.` to include reusable code from other files. +* **External Commands:** Delegate tasks to external commands when appropriate (e.g., `grep`, `sed`, `awk`). +* **Consider a 'main' function:** Wrap the primary logic of the script into a `main` function to improve readability and maintainability. All scripts that are long enough to contain at least one other function should have a `main` function. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +* **Facade:** Create a simplified interface to a complex set of functions or modules. +* **Strategy:** Define a family of algorithms and encapsulate each one in a separate function, making them interchangeable. +* **Template Method:** Define the skeleton of an algorithm in a function, deferring some steps to subfunctions. +* **Chain of Responsibility:** Pass a request along a chain of functions until one of them handles it. +* **Singleton (Carefully):** Use sparingly for global configuration, initialize only once. + +### 2.2. Recommended Approaches for Common Tasks + +* **String Manipulation:** Use Bash's built-in string manipulation features (e.g., parameter expansion, substring extraction) instead of external commands like `sed` or `awk` when possible. +* **File Handling:** Use Bash's built-in file handling commands (e.g., `read`, `write`, `mkdir`, `rm`) for basic operations. For more complex operations, consider `find` with `-exec` or `xargs`. +* **Looping:** Use `for` loops for iterating over lists of items and `while` loops for conditional execution. +* **Conditional Statements:** Use `if`, `elif`, and `else` statements for branching logic. Prefer `[[ ]]` over `[ ]` for string comparisons. +* **Exit Early:** Use `return` or `exit` to exit the script as soon as an error is detected. + +### 2.3. Anti-patterns and Code Smells + +* **Hardcoded Values:** Avoid hardcoding values directly in the script. Use constants or configuration files instead. +* **Unquoted Variables:** Always quote variables to prevent word splitting and globbing issues. +* **Eval:** Avoid using `eval` as it can introduce security vulnerabilities and make the code difficult to understand. +* **Backticks:** Use `$(command)` instead of backticks for command substitution. +* **Relying on Globals:** Minimize the use of global variables to avoid naming conflicts and unexpected side effects. Use `local` within functions. +* **Ignoring Errors:** Always check the return values of commands to handle errors gracefully. Use `set -e` to exit on errors automatically. +* **Over-commenting Obvious Code:** Comments should explain *why* the code is doing something, not *what* the code is doing. +* **Excessive Piping:** While powerful, long pipelines can become unreadable. Consider breaking down complex operations into smaller, more manageable steps with intermediate variables. + +### 2.4. State Management Best Practices + +* **Environment Variables:** Use environment variables to store global configuration settings that can be accessed by all scripts and functions. +* **Temporary Files:** Use temporary files to store intermediate data during script execution. Use `mktemp` to create unique temporary file names and remove them when finished. They should be created under `/tmp` or another location if needed. +* **Persistent Storage:** For persistent state, consider using a simple database (e.g., SQLite) or a key-value store (e.g., Redis). +* **Configuration Files:** Store persistent configuration options in configuration files. + +### 2.5. Error Handling Patterns + +* **`set -e`:** Exit immediately if a command exits with a non-zero status. +* **Check Return Values:** Use `$?` to check the return value of a command and handle errors accordingly. +* **Error Functions:** Create functions to handle common error scenarios (e.g., logging errors, displaying error messages, exiting the script). +* **Signal Handling:** Use `trap` to handle signals (e.g., `SIGINT`, `SIGTERM`) and perform cleanup actions. +* **Informative Error Messages:** Provide clear and informative error messages to help users diagnose problems. The error messages should explain the cause and possibly the recommended solution. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +* **Built-in Commands:** Prefer built-in commands over external commands whenever possible, as they are generally faster. +* **String Operations:** Use Bash's built-in string manipulation features instead of external commands like `sed` or `awk` for simple operations. +* **Avoid Loops:** Minimize the use of loops, as they can be slow in Bash. Consider using `find` with `-exec` or `xargs` for file processing instead. +* **Command Substitution:** Avoid unnecessary command substitution, as it can be expensive. +* **Arrays:** Use arrays to store lists of items instead of strings, as they are more efficient for iteration and manipulation. +* **Parameter Expansion:** Use parameter expansion features like `${var:-default}` to provide default values for variables without using conditional statements. +* **`set -f`:** Disable globbing when not needed to prevent unintended file expansion. + +### 3.2. Memory Management + +* **Limit Data Size:** Avoid loading large files or data structures into memory. +* **Stream Processing:** Use stream processing techniques to process data incrementally instead of loading it all into memory at once. +* **Remove Variables:** Unset or reassign variables to release memory when they are no longer needed. + +### 3.3. Rendering Optimization (If Applicable) + +* **Minimize Output:** Reduce the amount of output generated by the script, as it can slow down execution. +* **Use `printf`:** Use `printf` for formatted output instead of `echo`, as it is generally faster and more portable. + +### 3.4. Bundle Size Optimization (If Applicable) + +* **N/A:** Bash scripts do not typically have bundle sizes in the same way that web applications do. + +### 3.5. Lazy Loading + +* **Source on Demand:** Only source library scripts when they are needed. +* **Conditional Execution:** Use conditional statements to execute code blocks only when certain conditions are met. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities and Prevention + +* **Command Injection:** Avoid using `eval` and carefully sanitize any user input used in commands. +* **Path Injection:** Sanitize path names to prevent malicious users from manipulating file operations. +* **Denial of Service (DoS):** Implement resource limits to prevent scripts from consuming excessive CPU, memory, or disk space. +* **Information Disclosure:** Protect sensitive information (e.g., passwords, API keys) by storing them in environment variables or configuration files with appropriate permissions. +* **Shellshock:** Ensure your Bash version is patched against the Shellshock vulnerability. + +### 4.2. Input Validation + +* **Sanitize User Input:** Carefully sanitize any user input to prevent command injection and other vulnerabilities. +* **Regular Expressions:** Use regular expressions to validate input formats (e.g., email addresses, phone numbers). +* **Whitelist Validation:** Validate input against a whitelist of allowed values. +* **Length Limits:** Enforce length limits on input fields to prevent buffer overflows. + +### 4.3. Authentication and Authorization + +* **Avoid Storing Credentials:** Avoid storing credentials directly in scripts or configuration files. Use environment variables or a secure credential store instead. +* **Principle of Least Privilege:** Run scripts with the minimum necessary privileges. +* **User Authentication:** Implement user authentication using `sudo` or other authentication mechanisms. +* **Role-Based Access Control (RBAC):** Implement RBAC to restrict access to sensitive resources based on user roles. + +### 4.4. Data Protection + +* **Encryption:** Use encryption to protect sensitive data at rest and in transit. +* **Data Masking:** Mask sensitive data in log files and other output. +* **Access Control:** Restrict access to sensitive data to authorized users only. + +### 4.5. Secure API Communication + +* **HTTPS:** Use HTTPS for all API communication to encrypt data in transit. +* **API Keys:** Use API keys to authenticate requests to external APIs. +* **Rate Limiting:** Implement rate limiting to prevent abuse of your API. +* **Input Validation:** Validate all input from external APIs to prevent injection attacks. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +* **Test Individual Functions:** Write unit tests to verify the behavior of individual functions. +* **Mock Dependencies:** Use mocking techniques to isolate functions from their dependencies. +* **Assert Statements:** Use assert statements to verify that the functions return the expected values. +* **Testing Frameworks:** Consider using a bash testing framework like Bats (Bash Automated Testing System). + +### 5.2. Integration Testing + +* **Test Interactions:** Write integration tests to verify the interactions between different modules or components. +* **Real Environments:** Run integration tests in a real or simulated environment. +* **End-to-End Testing:** Write end-to-end tests to verify the entire script from start to finish. + +### 5.3. End-to-End Testing + +* **Real Use Cases:** Emulate real-world use cases to test the script's overall functionality. +* **System-Level Testing:** Verify that the script interacts correctly with the underlying system. + +### 5.4. Test Organization + +* **Separate Test Directory:** Create a separate `tests/` directory to store test files. +* **Naming Convention:** Use a consistent naming convention for test files (e.g., `module_name_test.sh`). +* **Test Suites:** Organize tests into suites based on functionality or module. + +### 5.5. Mocking and Stubbing + +* **Mock External Commands:** Create mock versions of external commands to isolate functions from their dependencies. +* **Stub Functions:** Create stub versions of functions to control their behavior during testing. +* **Environment Variable Mocking:** Mock environment variables to test different configurations. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +* **Forgetting `set -e`:** Not exiting on errors can lead to unexpected behavior. +* **Unquoted Variables:** Forgetting to quote variables can lead to word splitting and globbing issues. +* **Incorrect Variable Scope:** Using global variables when local variables are needed can lead to naming conflicts. +* **Not Checking Return Values:** Not checking return values can lead to errors being ignored. +* **Over-reliance on Globals** Using global variables when local variables are preferable can lead to unintended side effects. +* **Not using Functions** Code becomes difficult to maintain if its not broken into manageable functions. + +### 6.2. Edge Cases + +* **Empty Variables:** Handle empty variables gracefully. +* **Special Characters:** Handle special characters in file names and other input. +* **Large Files:** Handle large files efficiently to prevent memory issues. +* **Race Conditions:** Avoid race conditions when multiple scripts are running concurrently. + +### 6.3. Version-Specific Issues + +* **Bash 3 vs. Bash 4:** Be aware of compatibility issues between different versions of Bash. +* **POSIX Compliance:** If portability is a concern, stick to POSIX-compliant syntax. + +### 6.4. Compatibility Concerns + +* **Operating Systems:** Ensure your scripts are compatible with the target operating systems. +* **External Tools:** Be aware of the dependencies on external tools and ensure they are available on the target systems. + +### 6.5. Debugging Strategies + +* **`set -x`:** Enable tracing to see the commands being executed. +* **`echo` Statements:** Use `echo` statements to print variable values and debug messages. +* **Shellcheck:** Use Shellcheck to identify common errors and warnings. +* **Interactive Debugging:** Use an interactive debugger like `bashdb` to step through the code. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +* **Text Editor:** Use a text editor with syntax highlighting and code completion for Bash (e.g., VS Code, Sublime Text, Vim). +* **Shellcheck:** Use Shellcheck to lint your Bash code. +* **Bats:** Use Bats for unit testing. +* **Bashdb:** Use Bashdb for interactive debugging. +* **Git:** Use Git for version control. + +### 7.2. Build Configuration + +* **Makefile:** Use a Makefile to automate build and test processes. +* **Dependency Management:** Use a dependency management tool like `dep` to manage external dependencies. + +### 7.3. Linting and Formatting + +* **Shellcheck:** Use Shellcheck to lint your Bash code and enforce coding standards. +* **shfmt:** Use shfmt for automatic code formatting. + +### 7.4. Deployment Best Practices + +* **Package Scripts:** Package your scripts and dependencies into a deployable package (e.g., Debian package, RPM package). +* **Configuration Management:** Use configuration management tools (e.g., Ansible, Puppet, Chef) to deploy and configure your scripts on target systems. +* **Automated Deployment:** Use automated deployment pipelines to deploy your scripts quickly and reliably. +* **Idempotent Scripts:** Ensure your deployment scripts are idempotent, meaning they can be run multiple times without causing unintended side effects. + +### 7.5. CI/CD Integration + +* **Continuous Integration (CI):** Integrate your Bash scripts into a CI pipeline to automatically run tests and linting on every commit. +* **Continuous Deployment (CD):** Integrate your Bash scripts into a CD pipeline to automatically deploy your scripts to production. + +## 8. Naming Conventions + +Follow these naming conventions: + +* **Function Names:** Lowercase, with underscores to separate words (e.g., `process_data`, `calculate_total`). +* **Variable Names:** Lowercase, with underscores to separate words (e.g., `input_file`, `output_directory`). Loop variables should be similarly named for the variable you're looping through. +* **Constant Names:** All caps, separated with underscores (e.g., `MAX_RETRIES`, `DEFAULT_TIMEOUT`). Declared at the top of the file. +* **File Names:** Lowercase, with underscores to separate words (e.g., `data_processing.sh`, `configuration_settings.conf`). + +## 9. Style and Formatting + +* **Indentation:** Use 2 spaces for indentation. No tabs. +* **Line Length:** Limit lines to 80 characters. Use line continuation characters (`\`) to break long lines. +* **Spacing:** Use spaces around operators and after commas. +* **Comments:** Write clear and concise comments to explain the purpose and logic of your code. Any function that is not both obvious and short must be commented. Any function in a library must be commented regardless of length or complexity. Use `TODO` comments to mark code that needs to be improved or completed. +* **Quoting:** Always quote variables unless careful unquoted expansion is required. +* **Braces:** Use `${variable}` instead of `$variable` for clarity. +* **Pipelines:** Format pipelines for maximum clarity. Break long pipelines into multiple lines with one pipeline element per line. + +## 10. Exit Codes +* Ensure exit codes are consistent throughout the scripts. +* Use `exit 0` for successful completion. +* Use non-zero exit codes (e.g., 1, 2, 127, etc.) to indicate failure. +* Standardize exit codes for specific error conditions across your project. +* Include custom error messages with meaningful context. + +Following these best practices and coding standards will help you write more maintainable, readable, secure, and efficient Bash scripts. \ No newline at end of file diff --git a/.cursor/rules/beautifulsoup4.mdc b/.cursor/rules/beautifulsoup4.mdc new file mode 100644 index 000000000..261a130fa --- /dev/null +++ b/.cursor/rules/beautifulsoup4.mdc @@ -0,0 +1,5 @@ +--- +description: This rule file outlines best practices for using the beautifulsoup4 library in Python, covering code organization, performance, security, testing, and common pitfalls. +globs: *.py +--- +By following these best practices, you can build robust, efficient, and secure web scrapers using beautifulsoup4. \ No newline at end of file diff --git a/.cursor/rules/behave.mdc b/.cursor/rules/behave.mdc new file mode 100644 index 000000000..4b96f886b --- /dev/null +++ b/.cursor/rules/behave.mdc @@ -0,0 +1,4 @@ +--- +description: This rule file provides comprehensive best practices for writing maintainable and effective Behave BDD tests, including code organization, common patterns, performance considerations, and security. +globs: *.feature +--- diff --git a/.cursor/rules/black.mdc b/.cursor/rules/black.mdc new file mode 100644 index 000000000..c9c5b5ab8 --- /dev/null +++ b/.cursor/rules/black.mdc @@ -0,0 +1,134 @@ +--- +description: Enforces consistent code formatting using Black, the uncompromising Python code formatter, promoting readability and reducing diffs. Covers best practices related to Black's configuration, usage, and integrations. +globs: *.py +--- +- Adopt Black as the primary code formatter for all Python projects. +- Adhere to Black's default style guide for consistent formatting across projects. +- Integrate Black into the development workflow using pre-commit hooks and CI/CD pipelines. +- Configure Black using `pyproject.toml` file for project-specific settings, if needed. +- Use Black's CLI or editor integrations for seamless formatting. + +## Black Code Formatting Best Practices + +Black is the uncompromising Python code formatter. By using it, you agree to cede control over minutiae of hand-formatting. In return, Black gives you speed, determinism, and freedom from `pycodestyle` nagging about formatting. You will save time and mental energy for more important matters. + +### Key Principles + +- **Consistency:** Black enforces a consistent code style, reducing ambiguity and promoting readability. +- **Automation:** Black automates the formatting process, eliminating manual effort and reducing formatting-related code review comments. +- **Integration:** Black integrates seamlessly with popular development tools and workflows. +- **Minimal Configuration:** Black provides sensible defaults and limits configuration options to ensure consistency. + +### 1. Code Organization and Structure + +Black primarily focuses on formatting within files, but it indirectly influences code organization by promoting readability and consistent style. Consider the following: + +- **Directory Structure:** Black doesn't enforce a specific directory structure. However, a well-organized project with clear module boundaries will benefit more from consistent formatting. +- **File Naming Conventions:** Black uses the `.py` extension for python files. Adhering to PEP 8 naming conventions alongside Black is recommended for improved readability (`snake_case` for variables, functions, and modules; `PascalCase` for classes). +- **Module Organization:** Black formats code within modules. Break large modules into smaller, more manageable files to improve readability and maintainability. Organize related functions and classes within modules. +- **Component Architecture:** Black can be used with any component architecture. Focus on creating well-defined components with clear interfaces. Consistent formatting within each component enhances maintainability. +- **Code Splitting Strategies:** Black formats code regardless of the splitting strategy. Break code into smaller functions or classes to improve code readability and organization. + +### 2. Common Patterns and Anti-patterns + +- **Design Patterns:** Black doesn't impose any specific design patterns. However, it encourages clean, readable code that makes it easier to implement design patterns correctly. For example, consistent formatting makes it easier to identify and understand the structure of a Factory pattern. +- **Recommended Approaches:** + - Use Black on all Python files in your project. + - Integrate Black into your pre-commit hooks to automatically format code before committing. + - Use Black's CLI to format entire directories or specific files. + - Configure Black using `pyproject.toml` to customize settings like line length, if necessary. +- **Anti-patterns:** + - Manually formatting code instead of using Black. + - Ignoring Black's formatting suggestions. + - Disabling Black for specific files or sections of code without a strong reason. +- **State Management:** Black doesn't directly address state management, but its consistent formatting promotes code clarity, making state management strategies easier to implement and understand. Consider using established state management patterns like Redux or Context API (if applicable). +- **Error Handling:** Black encourages readable code, making it easier to implement and understand error handling mechanisms. Use `try...except` blocks to handle exceptions gracefully. Consistent formatting of exception handling code improves maintainability. + +### 3. Performance Considerations + +Black's formatting process itself has minimal performance overhead. The main performance considerations are related to the code being formatted, not Black itself. + +- **Optimization Techniques:** Black-formatted code is easier to read and understand, which can help identify performance bottlenecks. Use profiling tools to identify slow code and optimize accordingly. +- **Memory Management:** Black doesn't directly affect memory management. Follow Python's memory management best practices, such as using generators for large datasets and avoiding circular references. +- **Rendering Optimization:** Not applicable as Black is primarily a code formatter and not directly involved in rendering. +- **Bundle Size Optimization:** Not applicable as Black is a code formatter. Bundle size optimization techniques are relevant to web applications or other deployments where code size impacts load times. +- **Lazy Loading:** Not applicable as Black is a code formatter. + +### 4. Security Best Practices + +Black improves code readability, which can help identify potential security vulnerabilities. However, Black itself doesn't directly address security. Focus on the following security best practices: + +- **Common Vulnerabilities:** Black doesn't prevent common vulnerabilities like SQL injection or cross-site scripting (XSS). Use secure coding practices to avoid these vulnerabilities. +- **Input Validation:** Implement robust input validation to prevent injection attacks. Black's consistent formatting makes it easier to review input validation code. +- **Authentication and Authorization:** Use established authentication and authorization patterns to protect your application. Black's consistent formatting can improve the readability of authentication and authorization code. +- **Data Protection:** Use encryption to protect sensitive data. Follow data protection best practices, such as storing passwords securely. +- **Secure API Communication:** Use HTTPS to encrypt API communication. Implement API authentication and authorization to prevent unauthorized access. + +### 5. Testing Approaches + +Black promotes consistent code formatting, making tests easier to write, read, and maintain. + +- **Unit Testing:** Write unit tests for individual functions and classes. Black's consistent formatting makes it easier to write clear and concise unit tests. +- **Integration Testing:** Write integration tests to verify that different components of your application work together correctly. Consistent formatting makes it easier to understand the interactions between components. +- **End-to-end Testing:** Write end-to-end tests to verify that your application works as expected from a user's perspective. Consistent formatting makes it easier to debug end-to-end tests. +- **Test Organization:** Organize your tests into a logical directory structure. Follow a consistent naming convention for test files and functions. +- **Mocking and Stubbing:** Use mocking and stubbing to isolate units of code during testing. Black's consistent formatting makes it easier to understand the code being tested and create appropriate mocks. + +### 6. Common Pitfalls and Gotchas + +- **Frequent Mistakes:** + - Not integrating Black into the development workflow. + - Ignoring Black's formatting suggestions. + - Over-configuring Black and deviating from its default style. +- **Edge Cases:** + - Black may have difficulty formatting very complex or deeply nested expressions. + - Black's formatting may not always be optimal for specific use cases. +- **Version-specific Issues:** + - Ensure that all developers are using the same version of Black to avoid formatting inconsistencies. +- **Compatibility Concerns:** + - Black may have compatibility issues with older versions of Python or other tools. +- **Debugging Strategies:** + - Use Black's CLI to format code and identify formatting issues. + - Use a debugger to step through code and understand its behavior. + +### 7. Tooling and Environment + +- **Recommended Tools:** + - Black: The primary code formatter. + - Pre-commit: A tool for managing pre-commit hooks. + - VS Code, PyCharm, or other Python IDEs with Black integration. +- **Build Configuration:** + - Configure Black in your project's `pyproject.toml` file. + - Integrate Black into your build process to automatically format code. +- **Linting and Formatting:** + - Use Black for formatting and other linters (e.g., Flake8, pylint) for code quality checks. + - Configure your linter to ignore formatting errors reported by Black. +- **Deployment:** + - Ensure that your deployment environment has Black installed. + - Run Black as part of your deployment process to ensure consistent formatting. +- **CI/CD Integration:** + - Integrate Black into your CI/CD pipeline to automatically format code and run tests. + +### Example `.pre-commit-config.yaml` + + +repos: +- repo: https://github.com/psf/black + rev: 24.2.2 # Replace with the latest version + hooks: + - id: black + + +### Example `pyproject.toml` + +toml +[tool.black] +line-length = 88 +target-version = ['py38', 'py39', 'py310'] +include = '\.pyi?$' +exclude = ''' +/(\.(git|hg|mypy_cache|tox|venv)|_build|build|dist)/ # Removed: |/foo/bar/ +''' + + +By following these best practices, you can leverage Black to improve code quality, enhance collaboration, and reduce development time. \ No newline at end of file diff --git a/.cursor/rules/boto3.mdc b/.cursor/rules/boto3.mdc new file mode 100644 index 000000000..6ac9ef102 --- /dev/null +++ b/.cursor/rules/boto3.mdc @@ -0,0 +1,349 @@ +--- +description: This rule file outlines best practices for using the boto3 library, including code organization, security, performance, testing, and common pitfalls. It aims to ensure efficient, secure, and maintainable AWS integrations with boto3. +globs: *.py +--- +--- + +# boto3 Best Practices + +This document outlines best practices for using the boto3 library in Python for interacting with Amazon Web Services (AWS). Following these guidelines will help you write more efficient, secure, and maintainable code. + +## 1. Project Setup and Authentication + +- **Use Sessions**: Always use boto3 sessions for managing configurations and credentials effectively. + python + import boto3 + session = boto3.Session() + s3 = session.client('s3') + +- **Avoid Hard-coding Credentials**: Never hard-code AWS credentials in your code. Use environment variables, AWS CLI configuration, IAM roles, or AWS Secrets Manager. + + - **Environment Variables**: + python + import os + aws_access_key_id = os.environ.get('AWS_ACCESS_KEY_ID') + aws_secret_access_key = os.environ.get('AWS_SECRET_ACCESS_KEY') + s3 = boto3.client('s3', aws_access_key_id=aws_access_key_id, aws_secret_access_key=aws_secret_access_key) + + + - **AWS CLI Configuration**: + Configure your credentials using the AWS CLI (`aws configure`). Boto3 will automatically pick up the credentials. + + - **IAM Roles**: Use IAM roles for EC2 instances or Lambda functions to grant permissions. + + - **AWS Secrets Manager**: Securely store and retrieve secrets (like database passwords or API keys) using AWS Secrets Manager, accessible via boto3. + +## 2. Code Organization and Structure + +- **Directory Structure**: Organize your project with a clear directory structure. + + my_project/ + ├── src/ + │ ├── __init__.py + │ ├── aws/ + │ │ ├── __init__.py + │ │ ├── s3_utils.py + │ │ ├── dynamodb_utils.py + │ ├── models/ + │ │ ├── __init__.py + │ │ ├── user.py + │ ├── main.py + ├── tests/ + │ ├── __init__.py + │ ├── aws/ + │ │ ├── test_s3_utils.py + │ │ ├── test_dynamodb_utils.py + ├── requirements.txt + ├── .env + └── README.md + + +- **File Naming Conventions**: Use descriptive and consistent file names. + - `s3_utils.py`: Contains S3-related utility functions. + - `dynamodb_utils.py`: Contains DynamoDB-related utility functions. + - `user.py`: Defines a User model or data structure. + +- **Module Organization**: Break down your code into logical modules. + - `aws/`: Contains modules for interacting with different AWS services. + - `models/`: Contains data models representing your application's entities. + +- **Component Architecture**: Design your application with reusable components. + - Create separate functions or classes for different AWS operations (e.g., uploading files, querying DynamoDB). + +- **Code Splitting**: Split large files into smaller, more manageable files. + - Group related functions and classes into separate files. + - Use clear and concise names for your files and modules. + +## 3. Common Patterns and Anti-patterns + +- **Design Patterns**: Use appropriate design patterns for boto3 interactions. + + - **Factory Pattern**: Create clients using a factory pattern to abstract client creation. + python + class AWSClientFactory: + def create_client(self, service_name, session=None): + if not session: + session = boto3.Session() + return session.client(service_name) + + factory = AWSClientFactory() + s3 = factory.create_client('s3') + + + - **Strategy Pattern**: Implement different strategies for handling AWS operations (e.g., different retry mechanisms). + +- **Recommended Approaches**: Follow recommended approaches for common tasks. + + - **Uploading Files to S3**: Use `upload_file` or `upload_fileobj` for uploading files to S3. + python + s3.upload_file('my_local_file.txt', 'my_bucket', 'my_s3_key.txt') + + - **Downloading Files from S3**: Use `download_file` or `download_fileobj` for downloading files from S3. + python + s3.download_file('my_bucket', 'my_s3_key.txt', 'my_local_file.txt') + + +- **Anti-patterns**: Avoid common anti-patterns. + + - **Tight Coupling**: Don't tightly couple your application logic with boto3 code. Abstract AWS interactions behind interfaces. + - **Ignoring Errors**: Always handle exceptions when interacting with AWS services. + - **Over-permissioning**: Grant only the necessary permissions to IAM roles. + +- **State Management**: Manage state effectively in your boto3 applications. + + - **Stateless Functions**: Prefer stateless functions that don't rely on global variables. + - **DynamoDB Sessions**: Use DynamoDB for storing session data in web applications. + +- **Error Handling**: Implement robust error handling. + python + from botocore.exceptions import ClientError + + try: + response = s3.get_object(Bucket='my_bucket', Key='my_key') + print(response['Body'].read().decode('utf-8')) + except ClientError as e: + if e.response['Error']['Code'] == 'NoSuchKey': + print('The specified key does not exist.') + else: + print(f'An error occurred: {e}') + + +## 4. Performance Considerations + +- **Optimization Techniques**: Optimize boto3 interactions for performance. + + - **Pagination**: Use pagination to handle large datasets. + python + paginator = s3.get_paginator('list_objects_v2') + pages = paginator.paginate(Bucket='my_bucket') + for page in pages: + for obj in page['Contents']: + print(obj['Key']) + + + - **Batch Operations**: Use batch operations for efficient data processing. + python + import boto3 + + s3 = boto3.resource('s3') + bucket = s3.Bucket('my-bucket') + + delete_keys = [{'Key': 'file1.txt'}, {'Key': 'file2.txt'}] + + response = bucket.delete_objects(Delete={'Objects': delete_keys}) + print(response) + + - **Transfer Acceleration**: For faster uploads over long distances, enable S3 Transfer Acceleration. + python + s3 = boto3.client('s3', + config=boto3.session.Config( + s3={'use_accelerate_endpoint': True} + ) + ) + + +- **Memory Management**: Be mindful of memory usage when working with large files or datasets. + + - **Streaming**: Use streaming operations to process data in chunks. + - **Garbage Collection**: Ensure proper garbage collection to release unused memory. + +- **Bundle Size Optimization**: Reduce bundle size by using only necessary boto3 modules. + +- **Lazy Loading**: Load boto3 modules only when needed to reduce startup time. + +## 5. Security Best Practices + +- **Common Vulnerabilities**: Understand and prevent common vulnerabilities. + + - **Credential Leakage**: Protect your AWS credentials from being exposed in code or logs. + - **Injection Attacks**: Sanitize inputs to prevent injection attacks. + - **Cross-Site Scripting (XSS)**: Protect against XSS attacks when displaying data from AWS. + +- **Input Validation**: Validate all inputs to prevent security issues. + + - **Sanitize Inputs**: Remove or escape potentially harmful characters from inputs. + - **Use Parameterized Queries**: Use parameterized queries to prevent SQL injection attacks when working with databases. + +- **Authentication and Authorization**: Implement proper authentication and authorization mechanisms. + + - **Principle of Least Privilege**: Grant only the necessary permissions to IAM roles and users. + - **Multi-Factor Authentication (MFA)**: Enable MFA for sensitive operations. + +- **Data Protection**: Protect your data at rest and in transit. + + - **Encryption**: Use server-side encryption (SSE-S3) for data security by default. For additional security using AWS Key Management Service (KMS) keys, include encryption parameters + python + response = boto3.client('s3').upload_file( + file_path, + bucket_name, + key, + ExtraArgs={ + 'ServerSideEncryption': 'aws:kms', + 'SSEKMSKeyId': 'your-kms-key-id' + } + ) + + + - **Secure Transport**: Enforce HTTPS-only access by applying a bucket policy. + - **Versioning**: Enable S3 Versioning for recovery. + +- **Secure API Communication**: Ensure secure communication with AWS APIs. + + - **HTTPS**: Always use HTTPS for communication with AWS services. + +## 6. Testing Approaches + +- **Unit Testing**: Write unit tests for your boto3 components. + + - **Mocking**: Use mocking libraries like `moto` to simulate AWS services during testing. + - **Test Cases**: Create comprehensive test cases for different scenarios. + +- **Integration Testing**: Test the integration of your boto3 components with AWS services. + + - **LocalStack**: Use LocalStack to simulate AWS services locally for integration testing. + - **End-to-End Tests**: Write end-to-end tests to verify the complete application flow. + +- **Test Organization**: Organize your tests in a clear and maintainable structure. + + - **Separate Test Files**: Create separate test files for each module or component. + - **Descriptive Test Names**: Use descriptive names for your test functions and classes. + +- **Mocking and Stubbing**: Use mocking and stubbing techniques to isolate components during testing. + + - **Mock Boto3 Clients**: Mock boto3 clients to avoid making real API calls during unit tests. + + python + import boto3 + from moto import mock_aws + import unittest + + class MyTestCase(unittest.TestCase): + + @mock_aws + def test_s3_upload(self): + s3 = boto3.client('s3') + s3.create_bucket(Bucket='mybucket') + s3.put_object(Bucket='mybucket', Key='myfile', Body='my content') + response = s3.get_object(Bucket='mybucket', Key='myfile') + self.assertEqual(response['Body'].read().decode('utf-8'), 'my content') + + if __name__ == '__main__': + unittest.main() + + + +## 7. Common Pitfalls and Gotchas + +- **Frequent Mistakes**: Be aware of common mistakes when using boto3. + + - **Incorrect Region**: Specifying the wrong AWS region. + - **Missing Permissions**: Insufficient IAM permissions. + - **Not Handling Exceptions**: Ignoring exceptions raised by boto3. + +- **Edge Cases**: Consider edge cases when designing your application. + + - **Throttling**: Handle API throttling by implementing retry mechanisms. + +- **Version-Specific Issues**: Be aware of version-specific issues with boto3. + + - **API Changes**: Keep up with API changes and deprecations. + - **Compatibility**: Test your code with different boto3 versions. + +- **Compatibility Concerns**: Address compatibility concerns between boto3 and other technologies. + + - **Python Versions**: Ensure compatibility with different Python versions. + - **Dependency Conflicts**: Resolve dependency conflicts with other libraries. + +- **Debugging**: Use effective debugging strategies for boto3 applications. + + - **Logging**: Implement detailed logging to track API calls and errors. + - **Debugging Tools**: Use debugging tools like `pdb` or IDE debuggers. + +## 8. Tooling and Environment + +- **Recommended Development Tools**: Use recommended development tools. + + - **IDE**: Use a Python IDE like VS Code, PyCharm, or Sublime Text. + - **AWS CLI**: Use the AWS CLI for managing AWS resources. + +- **Build Configuration**: Configure your build process for boto3 projects. + + - **Requirements File**: Use a `requirements.txt` file to manage dependencies. + - **Virtual Environments**: Use virtual environments to isolate dependencies. + +- **Linting and Formatting**: Follow linting and formatting recommendations. + + - **PEP 8**: Adhere to PEP 8 style guidelines. + - **Linters**: Use linters like `flake8` or `pylint` to enforce code quality. + - **Formatters**: Use formatters like `black` to automatically format your code. + +- **Deployment**: Follow deployment best practices. + + - **Infrastructure as Code (IaC)**: Use IaC tools like Terraform or CloudFormation to manage your infrastructure. + - **CI/CD**: Implement CI/CD pipelines for automated deployments. + +- **CI/CD Integration**: Integrate your boto3 projects with CI/CD systems. + + - **Automated Tests**: Run automated tests during the CI/CD process. + - **Deployment Pipelines**: Create deployment pipelines to automate the deployment process. + +## 9. Advanced S3 features + +- S3 Object Lambda: Process and transform the data returned by S3 GET requests on the fly without modifying the stored object. +- S3 Lifecycle policies: Automate the transition of objects to more cost-effective storage classes, or schedule their deletion based on defined rules. +- S3 Event Notifications: Monitor bucket events and trigger actions such as invoking Lambda functions, sending messages via SNS or SQS, when new objects are created. + +For example, you can configure S3 Event Notifications with Boto3 as follows: +python +import boto3 +s3_client = boto3.client('s3') +notification_configuration = { + 'LambdaFunctionConfigurations': [ + { + 'LambdaFunctionArn': 'arn:aws:lambda:us-east-1:123456789012:function:ProcessS3Event', + 'Events': ['s3:ObjectCreated:*'], + 'Filter': { + 'Key': { + 'FilterRules': [ + {'Name': 'suffix', 'Value': '.jpg'} + ] + } + } + } + ] +} +s3_client.put_bucket_notification_configuration( + Bucket='your-bucket-name', + NotificationConfiguration=notification_configuration +) + + +## 10. Best Practices Summary + +- **Security**: Prioritize security by avoiding hard-coded credentials, enforcing the principle of least privilege, and using encryption. +- **Performance**: Optimize performance by using pagination, batch operations, and S3 Transfer Acceleration. +- **Error Handling**: Implement robust error handling with comprehensive logging. +- **Testing**: Write unit tests, integration tests, and end-to-end tests using mocking and LocalStack. +- **Code Organization**: Maintain a clear and modular code structure. + +By following these best practices, you can ensure that your boto3 applications are efficient, secure, and maintainable. \ No newline at end of file diff --git a/.cursor/rules/bottle.mdc b/.cursor/rules/bottle.mdc new file mode 100644 index 000000000..49f1e48a6 --- /dev/null +++ b/.cursor/rules/bottle.mdc @@ -0,0 +1,279 @@ +--- +description: Comprehensive coding standards, best practices, and architectural guidelines for Python backend development with the Bottle microframework, designed to improve code quality, maintainability, and security. +globs: *.py +--- +# Bottle Framework Best Practices + +This document provides comprehensive coding standards, best practices, and architectural guidelines for Python backend development using the Bottle microframework. Adhering to these guidelines will improve code quality, maintainability, security, and overall project success. + +## 1. Code Organization and Structure + +Effective code organization is crucial for maintainability and scalability. Here's how to structure your Bottle applications: + +### 1.1. Directory Structure Best Practices + +A well-organized directory structure makes it easier to navigate and understand the project. + + +my_bottle_app/ +├── app.py # Main application entry point +├── routes/ +│ ├── __init__.py # Make 'routes' a package +│ ├── home.py # Route definitions for the home page +│ ├── api.py # Route definitions for the API +│ └── users.py # Route definitions for user management +├── models/ +│ ├── __init__.py +│ ├── user.py # User model definition +│ └── product.py # Product model definition +├── views/ +│ ├── __init__.py +│ ├── home.tpl # Template for the home page +│ └── user_list.tpl # Template for listing users +├── config.py # Application configuration +├── db.py # Database connection and setup +├── utils/ +│ ├── __init__.py +│ ├── auth.py # Authentication utilities +│ └── helpers.py # General helper functions +├── tests/ +│ ├── __init__.py +│ ├── test_routes.py # Tests for route handlers +│ └── test_models.py # Tests for data models +├── requirements.txt # Project dependencies +└── README.md + + +### 1.2. File Naming Conventions + +* **Python Files:** Use lowercase with underscores (snake_case) for Python file names (e.g., `user_model.py`, `api_routes.py`). +* **Template Files:** Use lowercase with underscores, and the `.tpl` extension (e.g., `product_details.tpl`). +* **Configuration Files:** Use a descriptive name like `config.py`. +* **Test Files:** Prefix test files with `test_` (e.g., `test_user_routes.py`). + +### 1.3. Module Organization + +* **Separate Concerns:** Group related functionality into separate modules (e.g., routes, models, utilities). +* **Use Packages:** Utilize Python packages (directories with `__init__.py` files) to further organize modules into logical groups. +* **Avoid Circular Imports:** Carefully manage dependencies between modules to prevent circular import errors. Structure your code so that lower-level modules (e.g., models, utilities) don't depend on higher-level modules (e.g., routes). + +### 1.4. Component Architecture + +* **MVC (Model-View-Controller):** While Bottle is simple, aim for an MVC-like structure: + * **Models:** Represent data and business logic (e.g., database interactions). + * **Views:** Render the user interface (using Bottle's templating or another engine). + * **Controllers (Routes):** Handle user requests, interact with models, and select the appropriate view. +* **Service Layer (Optional):** For more complex applications, introduce a service layer between routes and models to encapsulate business logic and improve testability. + +### 1.5. Code Splitting Strategies + +* **Functional Decomposition:** Break down complex functions into smaller, more manageable functions with single responsibilities. +* **Route Grouping:** Group related routes into separate modules or route blueprints for better organization. +* **Configuration Management:** Move configuration settings to a separate module for easy management and environment-specific configurations. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +* **Singleton (Carefully):** Use sparingly for resources like database connections. Ensure thread safety if using a singleton in a multithreaded environment. +* **Factory:** Useful for creating model instances or other objects in a controlled manner. +* **Middleware (Plugins):** Bottle's plugin system allows you to create reusable middleware components for tasks like authentication, logging, and request processing. + +### 2.2. Recommended Approaches for Common Tasks + +* **Database Access:** Use an ORM (SQLAlchemy) for database interaction. This provides an abstraction layer, improves code readability, and helps prevent SQL injection vulnerabilities. +* **Templating:** Leverage Bottle's built-in templating engine for simple projects. For more complex templating needs, consider Jinja2 or Mako. +* **Request Handling:** Use Bottle's decorators (`@route`, `@get`, `@post`) for defining routes and handling HTTP requests. +* **Error Handling:** Implement custom error handlers for different HTTP status codes to provide informative error messages to the client. + +### 2.3. Anti-patterns and Code Smells + +* **Fat Routes:** Avoid putting too much logic directly in route handlers. Delegate complex tasks to models, services, or helper functions. +* **Global State:** Minimize the use of global variables. Prefer dependency injection or passing data as arguments. +* **Hardcoding Configuration:** Never hardcode sensitive information like API keys or database credentials. Use environment variables or a configuration file. +* **Ignoring Exceptions:** Don't catch exceptions and do nothing. Log the error and take appropriate action (e.g., return an error response to the client). +* **SQL Injection Vulnerabilities:** Avoid string concatenation when building SQL queries. Use parameterized queries provided by your ORM or database driver. + +### 2.4. State Management + +* **Stateless Architecture:** Prefer a stateless architecture where each request is independent and doesn't rely on server-side session state. This improves scalability. +* **Sessions (if needed):** If session state is required, use a session middleware (e.g., Beaker) or a token-based authentication system (JWT). +* **Cookies:** Use cookies for storing non-sensitive information on the client-side. + +### 2.5. Error Handling + +* **Specific Exception Handling:** Catch specific exceptions rather than generic `Exception` to handle different error scenarios appropriately. +* **Logging:** Use the `logging` module to log errors and other important events. Configure different logging levels (DEBUG, INFO, WARNING, ERROR, CRITICAL) for different environments. +* **Custom Error Pages:** Create custom error pages for different HTTP status codes (404, 500, etc.) to provide a better user experience. +* **Return Informative Error Responses:** When an error occurs, return a JSON response with a clear error message and status code. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +* **Caching:** Implement caching for frequently accessed data to reduce database load and improve response times. Use Bottle's caching mechanism or an external caching system like Redis or Memcached. +* **Database Optimization:** Optimize database queries by using indexes, avoiding unnecessary joins, and selecting only the required columns. +* **Gzip Compression:** Enable Gzip compression for HTTP responses to reduce the amount of data transferred to the client. +* **Asynchronous Operations:** Use asynchronous tasks (e.g., Celery) for long-running operations like sending emails or processing large datasets. This prevents blocking the main thread and improves responsiveness. +* **Profiling:** Use profiling tools to identify performance bottlenecks in your code. + +### 3.2. Memory Management + +* **Avoid Memory Leaks:** Be careful when dealing with large datasets or long-running processes to avoid memory leaks. Use generators or iterators to process data in chunks. +* **Resource Management:** Release resources (e.g., database connections, file handles) as soon as they are no longer needed. Use `try...finally` blocks or context managers (`with` statement) to ensure resources are released even if an exception occurs. + +### 3.3. Rendering Optimization + +* **Template Caching:** Enable template caching to improve rendering performance. Bottle's built-in templating engine supports caching. +* **Minify CSS and JavaScript:** Minify CSS and JavaScript files to reduce their size and improve loading times. +* **Optimize Images:** Optimize images for the web by using appropriate image formats, compression levels, and dimensions. + +### 3.4. Bundle Size Optimization (Applicable if using Bottle to serve static assets) + +* **Code Splitting (For Single Page Apps):** If building a SPA, consider code-splitting your javascript into smaller bundles. +* **Tree Shaking:** Use a bundler (like Webpack or Parcel) that supports tree shaking to remove unused code from your JavaScript bundles. + +### 3.5. Lazy Loading + +* **Lazy Imports:** Import modules only when they are needed. This can reduce startup time. +* **Database Connection on Demand:** Establish database connections only when they are required, rather than at application startup. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities and Prevention + +* **SQL Injection:** Use parameterized queries with an ORM like SQLAlchemy to prevent SQL injection vulnerabilities. +* **Cross-Site Scripting (XSS):** Sanitize user input before displaying it in templates to prevent XSS attacks. Use Bottle's HTML escaping functions or a templating engine that automatically escapes HTML. +* **Cross-Site Request Forgery (CSRF):** Implement CSRF protection by generating and validating CSRF tokens for state-changing requests. Bottle doesn't have built-in CSRF protection, so you'll need to use a library or implement it manually. +* **Authentication Bypass:** Ensure proper authentication and authorization mechanisms are in place to prevent unauthorized access to resources. +* **Denial of Service (DoS):** Implement rate limiting and input validation to prevent DoS attacks. + +### 4.2. Input Validation + +* **Validate All User Input:** Validate all user input on both the client-side and server-side. +* **Use Data Type Validation:** Ensure that input data matches the expected data types. +* **Sanitize Input:** Sanitize input data to remove potentially harmful characters or code. +* **Limit Input Length:** Limit the length of input fields to prevent buffer overflows. + +### 4.3. Authentication and Authorization + +* **Use a Strong Authentication Scheme:** Implement a secure authentication scheme like OAuth 2.0 or JWT. +* **Hash Passwords:** Hash passwords using a strong hashing algorithm (e.g., bcrypt or Argon2) before storing them in the database. Never store passwords in plain text. +* **Implement Role-Based Access Control (RBAC):** Implement RBAC to control access to resources based on user roles. +* **Use HTTPS:** Always use HTTPS to encrypt communication between the client and server. + +### 4.4. Data Protection + +* **Encrypt Sensitive Data:** Encrypt sensitive data (e.g., credit card numbers, social security numbers) both in transit and at rest. +* **Use Environment Variables:** Store sensitive configuration data (e.g., API keys, database credentials) in environment variables rather than hardcoding them in the code. +* **Regularly Update Dependencies:** Keep your dependencies up to date to patch security vulnerabilities. +* **Secure File Uploads:** Validate file types and sizes during upload. Store uploaded files outside the web root and serve them through a controlled endpoint. + +### 4.5. Secure API Communication + +* **API Keys:** Use API keys for authentication and authorization of API clients. +* **Rate Limiting:** Implement rate limiting to prevent abuse of your API. +* **Input Validation:** Thoroughly validate all data received through the API. +* **HTTPS Only:** Require HTTPS for all API communication. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +* **Test Individual Components:** Unit tests should focus on testing individual components (e.g., functions, classes) in isolation. +* **Use a Testing Framework:** Use a testing framework like `pytest` or `unittest` to write and run unit tests. +* **Mock Dependencies:** Mock external dependencies (e.g., database connections, API calls) to isolate the component being tested. +* **Test Edge Cases:** Test edge cases and boundary conditions to ensure that the component handles unexpected input correctly. +* **Follow the Arrange-Act-Assert Pattern:** Structure your tests using the Arrange-Act-Assert pattern to make them more readable and maintainable. + +### 5.2. Integration Testing + +* **Test Interactions Between Components:** Integration tests should focus on testing the interactions between different components of the application. +* **Use a Test Database:** Use a separate test database to avoid affecting the production database. +* **Test API Endpoints:** Test API endpoints to ensure that they return the correct data and status codes. + +### 5.3. End-to-End Testing + +* **Simulate User Interactions:** End-to-end tests should simulate user interactions with the application to ensure that the entire system works correctly. +* **Use a Browser Automation Tool:** Use a browser automation tool like Selenium or Playwright to automate end-to-end tests. +* **Test Common User Flows:** Test common user flows to ensure that the application meets the needs of its users. + +### 5.4. Test Organization + +* **Separate Test Files:** Create separate test files for each module or component. +* **Organize Tests by Feature:** Organize tests into directories based on the feature they are testing. +* **Use Descriptive Test Names:** Use descriptive test names that clearly indicate what the test is verifying. + +### 5.5. Mocking and Stubbing + +* **Use Mocking for External Dependencies:** Use mocking to simulate the behavior of external dependencies (e.g., databases, APIs) in your tests. +* **Use Stubbing for Complex Calculations:** Use stubbing to replace complex calculations with precomputed values. +* **Use a Mocking Library:** Use a mocking library like `unittest.mock` or `pytest-mock` to simplify the process of creating mocks and stubs. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +* **Not Sanitizing User Input:** Failing to sanitize user input can lead to XSS vulnerabilities. +* **Hardcoding Secrets:** Hardcoding sensitive information like API keys or database passwords directly in the code makes them vulnerable to exposure. +* **Incorrect Error Handling:** Improper exception handling can result in unhandled errors and application crashes. +* **Ignoring Security Updates:** Neglecting dependency updates leaves you exposed to known vulnerabilities. + +### 6.2. Edge Cases + +* **Unicode Handling:** Ensure your application handles Unicode characters correctly, especially when dealing with user input or data from external sources. +* **Time Zones:** Be mindful of time zones when dealing with dates and times. Store dates and times in UTC and convert them to the user's local time zone when displaying them. +* **Concurrency Issues:** If your application handles concurrent requests, be aware of potential race conditions and synchronization issues. + +### 6.3. Version-Specific Issues + +* **Bottle Versions:** Stay informed about changes and bug fixes of specific Bottle versions you are using. Check the Bottle changelog for changes that may impact your application. + +### 6.4. Compatibility Concerns + +* **Python Version Compatibility:** Ensure your application is compatible with the Python version you are using. Bottle supports Python 3.6+. +* **Dependency Conflicts:** Be aware of potential dependency conflicts between different libraries. Use a virtual environment to isolate your project's dependencies. + +### 6.5. Debugging Strategies + +* **Use a Debugger:** Use a debugger like `pdb` or the built-in debugger in your IDE to step through your code and inspect variables. +* **Logging:** Use the `logging` module to log errors and other important events. Configure different logging levels for different environments. +* **Error Handling:** Implement custom error handlers to catch exceptions and provide informative error messages. +* **Testing:** Write unit tests and integration tests to catch bugs early in the development process. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +* **IDE:** Use a powerful IDE like Visual Studio Code, PyCharm, or Sublime Text for code editing, debugging, and testing. +* **Virtual Environment Manager:** Use `venv`, `virtualenv`, or `conda` to create virtual environments for isolating project dependencies. +* **Package Manager:** Use `pip` or `conda` to install and manage Python packages. +* **Database Client:** Use a database client like DBeaver or pgAdmin to connect to and manage your databases. + +### 7.2. Build Configuration + +* **Use `requirements.txt`:** Use a `requirements.txt` file to specify your project's dependencies. Generate this file using `pip freeze > requirements.txt`. +* **Specify Versions:** Specify exact versions of your dependencies in `requirements.txt` to ensure consistent builds across different environments. +* **Use a Build Tool (Optional):** For more complex projects, consider using a build tool like Make or Invoke to automate build tasks. + +### 7.3. Linting and Formatting + +* **Use a Linter:** Use a linter like `flake8` or `pylint` to enforce coding style and identify potential errors. +* **Use a Formatter:** Use a code formatter like `black` or `autopep8` to automatically format your code according to PEP 8. +* **Configure Your IDE:** Configure your IDE to automatically run the linter and formatter when you save a file. + +### 7.4. Deployment Best Practices + +* **Use a Production Web Server:** Use a production-ready web server like Gunicorn or uWSGI to serve your application. +* **Use a Reverse Proxy:** Use a reverse proxy like Nginx or Apache to handle static assets, SSL termination, and load balancing. +* **Use a Process Manager:** Use a process manager like Supervisor or systemd to ensure that your application is always running. +* **Monitor Your Application:** Use a monitoring tool like Prometheus or Grafana to monitor your application's performance and identify potential issues. + +### 7.5. CI/CD Integration + +* **Use a CI/CD Pipeline:** Use a CI/CD pipeline to automate the process of building, testing, and deploying your application. +* **Use a CI/CD Tool:** Use a CI/CD tool like Jenkins, GitLab CI, GitHub Actions, or CircleCI to create your CI/CD pipeline. +* **Run Tests Automatically:** Configure your CI/CD pipeline to run tests automatically on every commit. +* **Automate Deployment:** Automate the deployment process so that new versions of your application are deployed automatically when the tests pass. + +By following these best practices, you can build robust, maintainable, and secure Bottle applications that meet the needs of your users and your organization. \ No newline at end of file diff --git a/.cursor/rules/bun.mdc b/.cursor/rules/bun.mdc new file mode 100644 index 000000000..2fbf56e86 --- /dev/null +++ b/.cursor/rules/bun.mdc @@ -0,0 +1,310 @@ +--- +description: This rule provides comprehensive guidance on Bun library coding standards, best practices, and common patterns. It includes performance optimization, security considerations, and testing strategies. +globs: *.js,*.ts,*.jsx,*.tsx,*.bun +--- +# Bun Library Best Practices + +This document outlines the recommended coding standards, best practices, and patterns for developing applications using the Bun library. Following these guidelines ensures maintainability, performance, security, and overall code quality. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure + +A well-defined directory structure is crucial for maintainability. Consider the following structure as a starting point: + + +project-root/ +├── src/ # Source code +│ ├── components/ # Reusable UI components (if applicable) +│ ├── services/ # Business logic and API interactions +│ ├── utils/ # Utility functions +│ ├── types/ # TypeScript type definitions +│ ├── routes/ # API route handlers +│ ├── middleware/ # Middleware functions +│ ├── models/ # Data models +│ ├── config/ # Configuration files +│ ├── index.ts # Entry point for the application +├── tests/ # Unit and integration tests +├── public/ # Static assets (e.g., images, CSS) +├── .env # Environment variables +├── bun.lockb # Lockfile for dependencies +├── package.json # Project metadata and dependencies +├── tsconfig.json # TypeScript configuration +├── README.md # Project documentation + + +* **src/**: Contains the main source code of the application. +* **components/**: Houses reusable UI components (if building a web application). +* **services/**: Encapsulates business logic and interactions with external APIs. +* **utils/**: Contains utility functions used throughout the application. +* **types/**: Stores TypeScript type definitions. +* **routes/**: Defines API route handlers using Bun's built-in HTTP server. +* **middleware/**: Includes middleware functions for request processing. +* **models/**: Defines data models used in the application. +* **config/**: Contains configuration files for different environments. +* **tests/**: Holds unit and integration tests. +* **public/**: Stores static assets like images and CSS files. +* **.env**: Stores environment variables. +* **bun.lockb**: Lockfile ensuring consistent dependency versions. +* **package.json**: Defines project metadata and dependencies. +* **tsconfig.json**: Configures TypeScript compiler options. +* **README.md**: Provides project documentation and instructions. + +### 1.2. File Naming Conventions + +* Use descriptive and consistent file names. +* Prefer camelCase for JavaScript/TypeScript files (e.g., `userService.ts`, `apiHelper.js`). +* Use kebab-case for component directories (e.g., `user-profile`). +* For React components, use PascalCase for the filename (e.g., `UserProfile.tsx`). + +### 1.3. Module Organization + +* Group related functionality into modules. +* Use clear and concise module names. +* Export only the necessary functions and classes from each module. +* Favor explicit imports over global variables. + +### 1.4. Component Architecture (if applicable) + +* If building a web application, adopt a component-based architecture (e.g., using React, SolidJS). +* Divide the UI into small, reusable components. +* Follow the Single Responsibility Principle (SRP) for each component. +* Use a consistent component structure (e.g., a folder for each component containing the component file, styles, and tests). + +### 1.5. Code Splitting Strategies + +* Use dynamic imports (`import()`) to split code into smaller chunks. +* Load only the necessary code for each route or component. +* Consider using a library like `esbuild` (Bun's underlying bundler) or `parcel` to automate code splitting. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +* **Singleton**: Use for managing global resources (e.g., database connections, configuration). +* **Factory**: Use for creating objects without specifying their concrete classes. +* **Observer**: Use for implementing event-driven systems. +* **Middleware**: Use to handle requests and responses in a centralized manner. +* **Dependency Injection**: Use to decouple components and improve testability. + +### 2.2. Recommended Approaches for Common Tasks + +* **API Request Handling**: Utilize `fetch` API provided by Bun for making HTTP requests. +* **File System Operations**: Use `Bun.file()` and other built-in functions for reading and writing files. +* **Process Management**: Leverage `Bun.spawn()` or `Bun.serve()` to manage child processes and servers. +* **Environment Variable Access**: Use `Bun.env` or `process.env` to access environment variables. +* **Logging**: Implement logging using `console.log` or a dedicated logging library like `pino`. + +### 2.3. Anti-patterns and Code Smells + +* **Global State**: Avoid using global variables for application state. Use state management solutions instead. +* **Long Functions**: Break down long functions into smaller, more manageable functions. +* **Duplicated Code**: Extract common logic into reusable functions or modules. +* **Magic Numbers**: Use named constants instead of hardcoded values. +* **Ignoring Errors**: Always handle errors properly using try-catch blocks or error handling middleware. + +### 2.4. State Management Best Practices + +* If your application requires complex state management, consider using a library like Zustand, Valtio, or Jotai. +* Choose a state management solution that fits the complexity of your application. +* Keep state updates predictable and consistent. +* Avoid mutating state directly. + +### 2.5. Error Handling Patterns + +* Use try-catch blocks to handle synchronous errors. +* Use `async/await` with try-catch blocks for asynchronous errors. +* Implement error handling middleware to catch unhandled exceptions. +* Log errors with relevant information (e.g., stack trace, request details). +* Provide informative error messages to the user. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +* **Minimize Dependencies**: Reduce the number of dependencies to decrease bundle size and install time. +* **Code Splitting**: Split code into smaller chunks that can be loaded on demand. +* **Tree Shaking**: Remove unused code during the build process. +* **Caching**: Cache frequently accessed data to reduce latency. +* **Compression**: Compress responses using gzip or Brotli to reduce network traffic. +* **Efficient Algorithms**: Choose the most efficient algorithms for your tasks. + +### 3.2. Memory Management + +* Avoid memory leaks by properly releasing resources. +* Use weak references to avoid circular dependencies. +* Monitor memory usage using tools like `bun --inspect`. +* Be mindful of large data structures and use streams when appropriate. + +### 3.3. Rendering Optimization (if applicable) + +* Use virtualization for large lists or tables. +* Optimize images and other assets. +* Use memoization to avoid unnecessary re-renders. +* Profile rendering performance using browser developer tools. + +### 3.4. Bundle Size Optimization + +* Use a bundler like `esbuild` to minimize bundle size. +* Remove unused code and dependencies. +* Use code splitting to load only the necessary code. +* Consider using a smaller alternative to large libraries. + +### 3.5. Lazy Loading Strategies + +* Use dynamic imports (`import()`) to load modules on demand. +* Implement lazy loading for images and other assets. +* Use a library like `react-lazyload` (if using React) to simplify lazy loading. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities and How to Prevent Them + +* **Cross-Site Scripting (XSS)**: Sanitize user input to prevent malicious scripts from being injected into the page. +* **Cross-Site Request Forgery (CSRF)**: Use CSRF tokens to prevent attackers from forging requests on behalf of authenticated users. +* **SQL Injection**: Use parameterized queries or an ORM to prevent attackers from injecting malicious SQL code. +* **Authentication and Authorization**: Implement robust authentication and authorization mechanisms to protect sensitive data. +* **Denial of Service (DoS)**: Implement rate limiting and other measures to prevent attackers from overwhelming the server. + +### 4.2. Input Validation + +* Validate all user input on both the client and server sides. +* Use a validation library like `zod` or `yup` to define validation schemas. +* Sanitize user input to remove potentially harmful characters. +* Escape user input when displaying it on the page. + +### 4.3. Authentication and Authorization Patterns + +* Use a secure authentication protocol like OAuth 2.0 or OpenID Connect. +* Store passwords securely using a hashing algorithm like bcrypt or Argon2. +* Implement role-based access control (RBAC) to restrict access to sensitive resources. +* Use JSON Web Tokens (JWT) for authentication and authorization. + +### 4.4. Data Protection Strategies + +* Encrypt sensitive data at rest and in transit. +* Use HTTPS to encrypt communication between the client and server. +* Store encryption keys securely using a key management system. +* Regularly back up data to prevent data loss. + +### 4.5. Secure API Communication + +* Use HTTPS for all API communication. +* Implement API authentication using API keys or JWTs. +* Rate limit API requests to prevent abuse. +* Validate API requests and responses. +* Use a firewall to protect the API from unauthorized access. + +## 5. Testing Approaches + +### 5.1. Unit Testing Strategies + +* Write unit tests for individual functions and classes. +* Use a testing framework like Jest or Bun's built-in test runner. +* Aim for high code coverage. +* Use mocks and stubs to isolate units of code. +* Test edge cases and error conditions. + +### 5.2. Integration Testing + +* Write integration tests to verify the interaction between different modules. +* Test the integration of the application with external APIs. +* Use a testing framework like Jest or Mocha. +* Use a testing database or mock API to isolate the tests. + +### 5.3. End-to-End Testing + +* Write end-to-end tests to verify the entire application flow. +* Use a testing framework like Playwright or Cypress. +* Run tests in a browser environment. +* Test the application from the user's perspective. + +### 5.4. Test Organization + +* Create a separate `tests` directory for test files. +* Organize test files in a way that mirrors the source code structure. +* Use descriptive test names. +* Follow a consistent testing style. + +### 5.5. Mocking and Stubbing + +* Use mocks and stubs to isolate units of code during testing. +* Use a mocking library like `jest.mock()` or `sinon`. +* Mock external dependencies to avoid relying on external services. +* Stub functions to control their behavior during testing. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes Developers Make + +* **Not Using Strict Mode**: Always use strict mode (`'use strict'`) to catch common coding errors. +* **Ignoring Error Handling**: Always handle errors properly using try-catch blocks or error handling middleware. +* **Leaking Global Variables**: Avoid creating global variables by using `let` or `const` to declare variables. +* **Not Understanding Asynchronous JavaScript**: Understand how asynchronous JavaScript works to avoid common pitfalls like callback hell. +* **Over-Engineering**: Keep the code simple and avoid unnecessary complexity. + +### 6.2. Edge Cases to Be Aware Of + +* **Handling Null and Undefined Values**: Check for null and undefined values before using them to avoid errors. +* **Integer Overflow**: Be aware of integer overflow and underflow when performing arithmetic operations. +* **Unicode Support**: Properly handle Unicode characters to avoid encoding issues. +* **Time Zone Handling**: Handle time zones correctly to avoid date and time discrepancies. + +### 6.3. Version-Specific Issues + +* Be aware of breaking changes in new versions of Bun. +* Test the application with different versions of Bun to ensure compatibility. +* Use a version manager like `bunx` to manage different Bun versions. + +### 6.4. Compatibility Concerns + +* Ensure the application is compatible with different operating systems and browsers. +* Use polyfills to support older browsers. +* Test the application on different devices to ensure responsiveness. + +### 6.5. Debugging Strategies + +* Use the `bun --inspect` flag to debug the application using Chrome DevTools. +* Use `console.log` statements to print debugging information. +* Use a debugger like VS Code's built-in debugger. +* Use a logging library to log errors and other important events. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +* **VS Code**: A popular code editor with excellent TypeScript support. +* **ESLint**: A linter for identifying and fixing code style issues. +* **Prettier**: A code formatter for automatically formatting code. +* **Jest**: A testing framework for unit and integration testing. +* **Playwright/Cypress**: A testing framework for end-to-end testing. +* **Postman/Insomnia**: API client for testing API endpoints. + +### 7.2. Build Configuration + +* Use a build tool like `esbuild` or `webpack` to bundle the application. +* Configure the build tool to optimize the bundle size and performance. +* Use environment variables to configure the build process for different environments. + +### 7.3. Linting and Formatting + +* Use ESLint to enforce consistent coding style. +* Use Prettier to automatically format code. +* Integrate ESLint and Prettier into the development workflow using VS Code extensions or command-line tools. +* Configure ESLint and Prettier to follow the project's coding style guidelines. + +### 7.4. Deployment Best Practices + +* Use a process manager like `pm2` or `systemd` to manage the application in production. +* Deploy the application to a cloud platform like DigitalOcean, Vercel, or Render. +* Use a CI/CD pipeline to automate the deployment process. +* Monitor the application's performance and health using monitoring tools. + +### 7.5. CI/CD Integration + +* Use a CI/CD platform like GitHub Actions, GitLab CI, or CircleCI to automate the build, test, and deployment process. +* Configure the CI/CD pipeline to run tests, lint code, and build the application on every commit. +* Use environment variables to configure the CI/CD pipeline for different environments. +* Deploy the application to a staging environment before deploying it to production. + +By following these best practices, developers can build high-quality, performant, and secure applications using the Bun library. Remember to adapt these guidelines to the specific needs of your project. \ No newline at end of file diff --git a/.cursor/rules/c-sharp.mdc b/.cursor/rules/c-sharp.mdc new file mode 100644 index 000000000..c31493a4c --- /dev/null +++ b/.cursor/rules/c-sharp.mdc @@ -0,0 +1,368 @@ +--- +description: This rule file provides a comprehensive guide to C# best practices, coding standards, and common patterns for writing maintainable, performant, and secure code. +globs: *.cs +--- +# C# Best Practices and Coding Standards + +This document provides a comprehensive guide to C# best practices, coding standards, and common patterns for writing maintainable, performant, and secure code. It covers various aspects of C# development, including code organization, common patterns, performance considerations, security best practices, testing approaches, common pitfalls, and tooling. + +**Library Information:** +- Name: c-sharp +- Tags: language, microsoft, dotnet, backend + +## 1. Code Organization and Structure + +A well-organized codebase is crucial for maintainability, scalability, and collaboration. Here are some best practices for organizing your C# code: + +### 1.1. Directory Structure Best Practices + +* **Project Root:** Contains the solution file (.sln) and project directories. +* **Project Directory:** Contains the project file (.csproj), source code, and other project-related files. + * `src/`: Contains the main source code. + * `Models/`: Data models and DTOs (Data Transfer Objects). + * `Services/`: Business logic and service classes. + * `Controllers/`: API controllers (if applicable). + * `Repositories/`: Data access logic. + * `Utilities/`: Helper classes and utility functions. + * `Exceptions/`: Custom exception definitions. + * `Interfaces/`: Interface definitions for abstraction. + * `Configuration/`: Configuration-related classes. + * `tests/`: Contains unit tests, integration tests, and end-to-end tests. + * `docs/`: Documentation for the project. + * `build/`: Build scripts and configuration files. + * `Properties/`: Assembly information and project settings. + +Example: + + +MyProject/ +├── MyProject.sln +├── MyProject/ +│ ├── MyProject.csproj +│ ├── src/ +│ │ ├── Models/ +│ │ ├── Services/ +│ │ ├── Controllers/ +│ │ ├── Repositories/ +│ │ ├── Utilities/ +│ │ ├── Exceptions/ +│ │ ├── Interfaces/ +│ │ └── Configuration/ +│ ├── Properties/ +│ │ └── AssemblyInfo.cs +│ └── appsettings.json +├── MyProject.Tests/ +│ ├── MyProject.Tests.csproj +│ └── UnitTests/ +└── README.md + + +### 1.2. File Naming Conventions + +* **Classes:** PascalCase (e.g., `MyClass.cs`) +* **Interfaces:** IPascalCase (e.g., `IMyInterface.cs`) +* **Enums:** PascalCase (e.g., `MyEnum.cs`) +* **Structs:** PascalCase (e.g., `MyStruct.cs`) +* **Delegates:** PascalCase (e.g., `MyDelegate.cs`) +* **Configuration Files:** appsettings.json, config.xml +* **Test Files:** `MyClassTests.cs` + +### 1.3. Module Organization + +* **Namespaces:** Use namespaces to group related classes and interfaces. Follow a consistent naming convention for namespaces (e.g., `CompanyName.ProjectName.ModuleName`). +* **Assemblies:** Divide large projects into multiple assemblies to improve build times, reduce dependencies, and enable code reuse. Consider functional or domain boundaries when creating assemblies. +* **NuGet Packages:** Use NuGet packages to manage dependencies and share code across projects. + +### 1.4. Component Architecture + +* **Layered Architecture:** Separate the application into distinct layers (e.g., presentation, business logic, data access) to promote separation of concerns and testability. +* **Microservices Architecture:** For large and complex applications, consider a microservices architecture where the application is composed of small, independent services. +* **Dependency Injection (DI):** Use DI to manage dependencies between components and improve testability and maintainability. Popular DI containers include Autofac, Ninject, and Microsoft.Extensions.DependencyInjection. + +### 1.5. Code Splitting Strategies + +* **By Feature:** Group code related to a specific feature into a separate module or assembly. +* **By Layer:** Separate code based on the architectural layer (e.g., presentation, business logic, data access). +* **By Responsibility:** Split classes and methods into smaller, more focused units of work. +* **Partial Classes:** Use partial classes to split a large class into multiple files for better organization (use sparingly). + +## 2. Common Patterns and Anti-patterns + +Understanding common design patterns and anti-patterns is essential for writing effective and maintainable C# code. + +### 2.1. Design Patterns + +* **Singleton:** Ensures that a class has only one instance and provides a global point of access to it. +* **Factory:** Provides an interface for creating objects without specifying their concrete classes. +* **Abstract Factory:** Provides an interface for creating families of related objects without specifying their concrete classes. +* **Builder:** Separates the construction of a complex object from its representation, allowing the same construction process to create different representations. +* **Observer:** Defines a one-to-many dependency between objects so that when one object changes state, all its dependents are notified and updated automatically. +* **Strategy:** Defines a family of algorithms, encapsulates each one, and makes them interchangeable. Strategy lets the algorithm vary independently from clients that use it. +* **Dependency Injection (DI):** A technique whereby one object (or static method) supplies the dependencies of another object. This helps to decouple components. +* **Repository:** Mediates between the domain and data mapping layers, acting like an in-memory domain object collection. +* **Unit of Work:** Maintains a list of objects affected by a business transaction and coordinates the writing out of changes. +* **Asynchronous Programming Patterns (TAP, EAP, APM):** Handle asynchronous operations efficiently using async/await. + +### 2.2. Recommended Approaches for Common Tasks + +* **String Manipulation:** Use `StringBuilder` for efficient string concatenation in loops. +* **File I/O:** Use `using` statements or `try-finally` blocks to ensure proper disposal of file resources. +* **Data Access:** Use ORMs like Entity Framework Core or Dapper for simplified data access. +* **Asynchronous Operations:** Use `async` and `await` for non-blocking asynchronous operations. +* **Configuration Management:** Use `Microsoft.Extensions.Configuration` for managing application configuration. +* **Logging:** Use logging frameworks like Serilog or NLog for structured logging. + +### 2.3. Anti-patterns and Code Smells + +* **God Class:** A class that does too much and has too many responsibilities. +* **Long Method:** A method that is too long and complex. +* **Feature Envy:** A method that accesses data from another object more than its own. +* **Shotgun Surgery:** Changes to one part of the code require changes to many other parts. +* **Data Clump:** Groups of data that appear together in multiple places. +* **Primitive Obsession:** Using primitive types instead of creating custom classes for domain concepts. +* **Switch Statements (over Polymorphism):** Using large switch statements instead of leveraging polymorphism. +* **Magic Numbers/Strings:** Hardcoding values directly in the code instead of using constants or configuration settings. +* **Ignoring Exceptions:** Catching exceptions without handling them properly. +* **Empty Catch Blocks:** Catching exceptions and doing nothing. +* **Over-commenting:** Writing excessive comments that are obvious from the code itself. +* **Dead Code:** Code that is never executed. + +### 2.4. State Management Best Practices + +* **Stateless Services:** Design services to be stateless whenever possible to improve scalability and reliability. +* **Session State:** Use session state sparingly and only when necessary. Consider using distributed caching for session state in web applications. +* **Caching:** Use caching to improve performance by storing frequently accessed data in memory. +* **Redux/Flux:** For complex UI applications, consider using a state management library like Redux or Flux. +* **Immutable Data Structures:** Use immutable data structures to simplify state management and prevent unintended side effects. + +### 2.5. Error Handling Patterns + +* **Try-Catch-Finally:** Use `try-catch-finally` blocks to handle exceptions and ensure proper resource cleanup. +* **Exception Filters:** Use exception filters to catch specific exceptions based on certain conditions. +* **Custom Exceptions:** Create custom exception types for specific error conditions in your application. +* **Logging Exceptions:** Log exceptions with sufficient context to aid in debugging. +* **Graceful Degradation:** Handle errors gracefully and provide informative error messages to the user. +* **Throw Early, Catch Late:** Detect errors as early as possible and handle exceptions at a higher level. + +## 3. Performance Considerations + +Optimizing C# code for performance is crucial for creating responsive and efficient applications. + +### 3.1. Optimization Techniques + +* **Avoid Boxing and Unboxing:** Boxing and unboxing value types can be expensive. Use generics to avoid boxing and unboxing. +* **Use Value Types When Appropriate:** Value types (structs) can be more efficient than reference types (classes) for small, immutable data structures. +* **Minimize Object Allocation:** Object allocation can be expensive. Reuse objects whenever possible. +* **Use `StringBuilder` for String Concatenation:** `StringBuilder` is more efficient than string concatenation using the `+` operator, especially in loops. +* **Optimize LINQ Queries:** Use LINQ carefully and avoid unnecessary iterations or computations. Consider using `AsParallel()` for parallel processing of large collections (with caution, as parallelism introduces complexity). +* **Use Asynchronous Programming:** Use `async` and `await` to avoid blocking the main thread and improve responsiveness. +* **Avoid Excessive Locking:** Minimize the use of locks to prevent contention and improve concurrency. +* **Use Lazy Initialization:** Initialize objects only when they are needed to avoid unnecessary initialization overhead. +* **Profile Your Code:** Use profiling tools to identify performance bottlenecks and optimize accordingly. Visual Studio Profiler, dotTrace, and PerfView are good options. + +### 3.2. Memory Management Considerations + +* **Garbage Collection:** Understand how the garbage collector works and avoid creating excessive garbage. +* **Dispose of Resources:** Implement the `IDisposable` interface and use `using` statements to ensure proper disposal of resources (e.g., file streams, database connections). +* **Weak References:** Use weak references to hold references to objects without preventing them from being garbage collected. +* **Object Pooling:** Use object pooling to reuse objects and reduce allocation overhead. +* **Large Object Heap (LOH):** Be aware of the Large Object Heap and avoid allocating large objects unnecessarily. + +### 3.3. Rendering Optimization (If Applicable) + +* **UI Virtualization:** Use UI virtualization to render only the visible items in a large list or grid. +* **Reduce Overdraw:** Minimize the number of times pixels are drawn on top of each other. +* **Batch Rendering:** Batch rendering operations to reduce the number of draw calls. +* **Use Hardware Acceleration:** Use hardware acceleration to offload rendering tasks to the GPU. + +### 3.4. Bundle Size Optimization (If Applicable) + +* **Tree Shaking:** Remove unused code from the bundle. +* **Code Minification:** Minify code to reduce its size. +* **Code Compression:** Compress code using Gzip or Brotli. +* **Image Optimization:** Optimize images to reduce their size without sacrificing quality. + +### 3.5. Lazy Loading Strategies + +* **Lazy Initialization:** Use `Lazy` to initialize objects only when they are accessed. +* **Virtual Proxy:** Use a virtual proxy to load related data only when it is needed. +* **Explicit Loading:** Load related data explicitly using methods like `Include` in Entity Framework Core. + +## 4. Security Best Practices + +Security should be a primary concern in C# development to protect against vulnerabilities and attacks. + +### 4.1. Common Vulnerabilities and Prevention + +* **SQL Injection:** Parameterize database queries to prevent SQL injection attacks. +* **Cross-Site Scripting (XSS):** Encode user input to prevent XSS attacks. +* **Cross-Site Request Forgery (CSRF):** Use anti-forgery tokens to prevent CSRF attacks. +* **Authentication and Authorization Vulnerabilities:** Implement secure authentication and authorization mechanisms. +* **Insecure Direct Object References (IDOR):** Validate user access to objects to prevent IDOR attacks. +* **File Upload Vulnerabilities:** Validate file types and sizes to prevent malicious file uploads. +* **Denial-of-Service (DoS) Attacks:** Implement rate limiting and other measures to prevent DoS attacks. +* **Deserialization Vulnerabilities:** Avoid deserializing untrusted data. +* **Dependency Vulnerabilities:** Regularly update dependencies to patch security vulnerabilities. + +### 4.2. Input Validation + +* **Validate All User Input:** Validate all user input on both the client and server side. +* **Use Regular Expressions:** Use regular expressions to validate input formats. +* **Sanitize Input:** Sanitize input to remove or escape potentially malicious characters. +* **Use Whitelisting:** Use whitelisting to allow only known good input values. +* **Limit Input Length:** Limit the length of input fields to prevent buffer overflows. + +### 4.3. Authentication and Authorization + +* **Use Strong Passwords:** Enforce strong password policies and use password hashing algorithms like bcrypt. +* **Implement Multi-Factor Authentication (MFA):** Use MFA to add an extra layer of security to the authentication process. +* **Use Role-Based Access Control (RBAC):** Use RBAC to control user access to resources based on their roles. +* **Use Claims-Based Authentication:** Use claims-based authentication to represent user identities and permissions. +* **Implement Proper Session Management:** Implement secure session management practices, including session timeouts and secure cookies. +* **OAuth 2.0 and OpenID Connect:** Leverage industry-standard protocols for authentication and authorization. + +### 4.4. Data Protection + +* **Encrypt Sensitive Data:** Encrypt sensitive data at rest and in transit. +* **Use Key Management:** Use a secure key management system to store and manage encryption keys. +* **Protect Connection Strings:** Protect connection strings and other sensitive configuration data. +* **Implement Auditing:** Implement auditing to track user actions and detect security breaches. + +### 4.5. Secure API Communication + +* **Use HTTPS:** Use HTTPS to encrypt communication between the client and server. +* **Implement API Authentication:** Use API keys, JWTs, or other authentication mechanisms to secure API endpoints. +* **Rate Limiting:** Implement rate limiting to prevent API abuse. +* **Input Validation:** Validate all API input to prevent injection attacks. +* **Output Encoding:** Encode API output to prevent XSS attacks. + +## 5. Testing Approaches + +Testing is a critical part of the software development process and helps ensure the quality and reliability of C# applications. + +### 5.1. Unit Testing + +* **Test Individual Units of Code:** Unit tests should focus on testing individual classes, methods, or functions in isolation. +* **Use Mocking Frameworks:** Use mocking frameworks like Moq or NSubstitute to isolate units of code and simulate dependencies. +* **Follow the Arrange-Act-Assert Pattern:** Arrange the test data, act on the code under test, and assert the expected results. +* **Write Clear and Concise Tests:** Write tests that are easy to read and understand. +* **Test Edge Cases and Error Conditions:** Test edge cases and error conditions to ensure that the code handles them properly. +* **Aim for High Code Coverage:** Aim for high code coverage to ensure that most of the code is tested. + +### 5.2. Integration Testing + +* **Test Interactions Between Components:** Integration tests should focus on testing the interactions between different components or modules of the application. +* **Use Real Dependencies or Test Doubles:** Use real dependencies or test doubles to simulate the environment in which the components will operate. +* **Test Data Access Logic:** Test data access logic to ensure that it interacts correctly with the database. +* **Test API Endpoints:** Test API endpoints to ensure that they handle requests and responses correctly. + +### 5.3. End-to-End Testing + +* **Test the Entire Application Flow:** End-to-end tests should focus on testing the entire application flow from start to finish. +* **Use Automation Frameworks:** Use automation frameworks like Selenium or Playwright to automate end-to-end tests. +* **Test User Interfaces:** Test user interfaces to ensure that they are functional and user-friendly. +* **Test Performance and Scalability:** Test performance and scalability to ensure that the application can handle the expected load. + +### 5.4. Test Organization + +* **Create Separate Test Projects:** Create separate test projects for unit tests, integration tests, and end-to-end tests. +* **Organize Tests by Feature or Module:** Organize tests by feature or module to improve maintainability. +* **Use Descriptive Test Names:** Use descriptive test names that clearly indicate what the test is verifying. +* **Use Test Categories:** Use test categories to group related tests together. + +### 5.5. Mocking and Stubbing + +* **Use Mocking Frameworks:** Use mocking frameworks like Moq or NSubstitute to create mock objects and stub dependencies. +* **Create Mock Objects for Dependencies:** Create mock objects for dependencies that are difficult or time-consuming to set up. +* **Stub Method Calls:** Stub method calls to return specific values or throw exceptions. +* **Verify Method Calls:** Verify that methods are called with the expected arguments. + +## 6. Common Pitfalls and Gotchas + +Being aware of common pitfalls and gotchas can help developers avoid mistakes and write more robust C# code. + +### 6.1. Frequent Mistakes + +* **NullReferenceException:** Not handling null values properly. +* **Incorrect Use of Asynchronous Programming:** Blocking the main thread with synchronous operations. +* **Memory Leaks:** Not disposing of resources properly. +* **Concurrency Issues:** Race conditions, deadlocks, and other concurrency issues. +* **Unhandled Exceptions:** Not catching or logging exceptions properly. +* **SQL Injection:** Not parameterizing database queries. +* **XSS Attacks:** Not encoding user input properly. +* **CSRF Attacks:** Not using anti-forgery tokens. +* **Ignoring Code Analysis Warnings:** Ignoring warnings from the compiler or code analysis tools. + +### 6.2. Edge Cases + +* **Empty Collections:** Handling empty collections properly. +* **Zero Values:** Handling zero values properly. +* **Maximum and Minimum Values:** Handling maximum and minimum values properly. +* **Date and Time Zones:** Handling date and time zones correctly. +* **Unicode Characters:** Handling Unicode characters correctly. + +### 6.3. Version-Specific Issues + +* **C# Language Features:** Being aware of new language features and how they affect existing code. +* **.NET Framework vs. .NET Core vs. .NET:** Understanding the differences between the different .NET platforms. +* **Breaking Changes:** Being aware of breaking changes in new versions of the .NET framework. + +### 6.4. Compatibility Concerns + +* **Cross-Platform Compatibility:** Ensuring that the code works correctly on different operating systems. +* **Backward Compatibility:** Ensuring that the code is compatible with older versions of the .NET framework. +* **Interoperability with Other Languages:** Ensuring that the code can interoperate with other languages like C++ or Java. + +### 6.5. Debugging Strategies + +* **Use the Visual Studio Debugger:** Use the Visual Studio debugger to step through code, inspect variables, and set breakpoints. +* **Use Logging:** Use logging to track the execution flow of the code and log important information. +* **Use Unit Tests:** Use unit tests to isolate and debug individual units of code. +* **Use Code Analysis Tools:** Use code analysis tools to identify potential problems in the code. +* **Use Profiling Tools:** Use profiling tools to identify performance bottlenecks. + +## 7. Tooling and Environment + +Choosing the right tools and configuring the development environment properly can significantly improve productivity and code quality. + +### 7.1. Recommended Development Tools + +* **Visual Studio:** The primary IDE for C# development. +* **Visual Studio Code:** A lightweight code editor with C# support. +* **JetBrains Rider:** A cross-platform .NET IDE. +* **NuGet Package Manager:** For managing dependencies. +* **Git:** For version control. +* **Resharper:** Powerful Visual Studio extension for code analysis, refactoring, and navigation. + +### 7.2. Build Configuration + +* **Use MSBuild or dotnet CLI:** Use MSBuild or the dotnet CLI to build the project. +* **Configure Build Configurations:** Configure different build configurations for debug and release builds. +* **Use NuGet Package Restore:** Use NuGet package restore to automatically download dependencies during the build process. +* **Sign Assemblies:** Sign assemblies to prevent tampering. +* **Generate Documentation:** Generate documentation from XML comments. +* **Enable deterministic builds:** Ensure builds are reproducible by enabling deterministic builds in the project file. + +### 7.3. Linting and Formatting + +* **Use StyleCop Analyzers:** Use StyleCop Analyzers to enforce coding style rules. +* **Use Roslyn Analyzers:** Use Roslyn analyzers to detect potential problems in the code. +* **Configure EditorConfig:** Use EditorConfig to define coding style rules for the project. +* **Use Code Formatters:** Use code formatters like the Visual Studio code formatter to automatically format code. + +### 7.4. Deployment Best Practices + +* **Choose a Deployment Target:** Choose a deployment target based on the application's requirements (e.g., Azure App Service, AWS Elastic Beanstalk, Docker). +* **Use a Deployment Pipeline:** Use a deployment pipeline to automate the deployment process. +* **Configure Application Settings:** Configure application settings properly for the deployment environment. +* **Monitor Application Health:** Monitor application health to detect and resolve issues. + +### 7.5. CI/CD Integration + +* **Use a CI/CD Platform:** Use a CI/CD platform like Azure DevOps, GitHub Actions, or Jenkins to automate the build, test, and deployment process. +* **Configure Build Triggers:** Configure build triggers to automatically start builds when code is committed. +* **Run Unit Tests and Integration Tests:** Run unit tests and integration tests as part of the build process. +* **Deploy to Different Environments:** Deploy the application to different environments (e.g., development, staging, production) as part of the deployment pipeline. +* **Automated Code Reviews:** Integrate with code review tools to automate aspects of code review. + +By following these best practices and coding standards, developers can write C# code that is maintainable, performant, secure, and reliable. \ No newline at end of file diff --git a/.cursor/rules/chakra-ui.mdc b/.cursor/rules/chakra-ui.mdc new file mode 100644 index 000000000..e9bba0439 --- /dev/null +++ b/.cursor/rules/chakra-ui.mdc @@ -0,0 +1,207 @@ +--- +description: This rule enforces best practices and coding standards for Chakra UI projects, including accessibility, styling, performance, and security. It aims to provide clear, actionable guidance for developers using Chakra UI. +globs: *.js,*.jsx,*.ts,*.tsx +--- +- **Accessibility**: Ensure all components follow WAI-ARIA standards, including keyboard navigation and screen reader support. + + - Use semantic HTML elements where appropriate (e.g., ` + + + +### 1.5 Code Splitting Strategies + +In htmx, code splitting can refer to different aspects. While htmx itself doesn't require complex JavaScript bundles, consider these: + +* **HTML Components:** Split large pages into smaller, more manageable components. Load these components on demand using htmx requests to improve initial page load time. +* **CSS Files:** Separate your CSS into logical modules (e.g., base styles, component styles, page-specific styles) to avoid loading unnecessary styles on every page. +* **htmx Extensions:** Only include the htmx extensions that you are actively using. Do not include all extensions if only a few are needed. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns Specific to htmx + +* **Progressive Enhancement:** Design your application to work without JavaScript first, then enhance it with htmx. This ensures accessibility and a baseline level of functionality for all users. +* **Hypermedia as the Engine of Application State (HATEOAS):** The core philosophy of htmx. The server should provide all the necessary links and forms for the client to interact with the application. The client simply follows these links and submits the forms; it doesn't need to know the specific URLs or request methods. +* **Server-Side Rendering (SSR):** Focus on rendering HTML on the server. This improves initial page load time, SEO, and accessibility. htmx enhances this by enabling dynamic updates without full page reloads. +* **Component-Based UI:** Design your UI as a collection of reusable components. Each component manages its own state and behavior, making the application easier to understand and maintain. + +### 2.2 Recommended Approaches for Common Tasks + +* **Form Handling:** Use htmx to submit forms and update the UI based on the server's response. Validate user input on both the client and server sides. +* **Pagination:** Implement pagination using htmx to load content in chunks. The server provides links to the next and previous pages, and htmx handles the loading and rendering of the new content. +* **Real-Time Updates:** Use WebSockets or Server-Sent Events (SSE) with htmx to display real-time data. The server pushes updates to the client, and htmx updates the UI accordingly. +* **Dynamic Search:** Implement live search functionality by sending AJAX requests to the server as the user types. Display the search results in real-time using htmx. +* **Modal Dialogs:** Use htmx to load the content of modal dialogs on demand. This avoids loading unnecessary content on initial page load. + +### 2.3 Anti-patterns and Code Smells to Avoid + +* **Over-reliance on JavaScript:** htmx aims to reduce the amount of JavaScript required for web development. Avoid using JavaScript for tasks that can be easily accomplished with htmx attributes. +* **Complex Client-Side Logic:** Keep your client-side logic as simple as possible. Move complex logic to the server-side, where it can be more easily tested and maintained. +* **Mixing htmx with other Front-end frameworks (React, Angular, Vue):** While technically possible, this defeats the purpose of using htmx. Using htmx with other front-end frameworks will likely increase complexity. +* **Ignoring Security Best Practices:** htmx does not automatically protect against security vulnerabilities. Always follow security best practices, such as input validation and output escaping, to prevent XSS attacks. +* **Creating API Endpoints that return JSON to only convert it to HTML:** htmx shines when your endpoints return HTML. Avoid the extra steps needed for JSON parsing on the client side. + +### 2.4 State Management Best Practices + +htmx promotes a server-centric approach to state management. The server maintains the application's state, and the client simply displays it. This simplifies client-side code and reduces the risk of inconsistencies between the client and server. + +* **Server-Side Sessions:** Store user-specific data in server-side sessions. This ensures that sensitive data is not exposed to the client. +* **Hidden Input Fields:** Use hidden input fields to store data that needs to be persisted across requests. This is useful for maintaining state in forms. +* **Cookies:** Use cookies to store small amounts of data on the client-side. Be mindful of cookie size limits and security implications. +* **URL Parameters:** Use URL parameters to represent state that can be shared or bookmarked. This is useful for implementing features like filtering and sorting. +* **`hx-vals`:** Use `hx-vals` for temporary client-side state that is not critical and doesn't need to be persisted on the server. This is suitable for things like the current step in a multi-step form or the visibility state of a component. + +### 2.5 Error Handling Patterns + +* **HTTP Status Codes:** Use appropriate HTTP status codes to indicate the success or failure of a request. For example, use `200 OK` for successful requests, `400 Bad Request` for invalid input, and `500 Internal Server Error` for server-side errors. +* **htmx Error Handling:** htmx provides the `htmx:responseError` event for handling errors. Use this event to display error messages to the user or take other appropriate actions. +* **Server-Side Error Pages:** Configure your server to display custom error pages for different HTTP status codes. This provides a user-friendly experience even when errors occur. +* **Client-Side Error Messages:** Display informative error messages to the user when a request fails. Avoid displaying technical details that could expose sensitive information. +* **Error Logging:** Log errors on the server-side to help diagnose and fix problems. Include relevant information such as the request URL, user ID, and error message. +* **Graceful Degradation:** Even when errors occur, ensure that your application degrades gracefully. Provide alternative functionality or a clear explanation of the problem. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +* **Minimize DOM Updates:** htmx is efficient at updating the DOM, but excessive updates can still impact performance. Use the `hx-swap` attribute to control how content is swapped into the DOM and minimize unnecessary changes. +* **Optimize Server-Side Rendering:** Ensure that your server-side rendering is as efficient as possible. Use caching, compression, and other optimization techniques to reduce the time it takes to generate HTML. +* **Compress Static Assets:** Compress your CSS, JavaScript, and image files to reduce their size and improve loading times. Use tools like Gzip or Brotli for compression. +* **Use a CDN:** Serve your static assets from a Content Delivery Network (CDN) to improve loading times for users around the world. +* **Lazy Load Images:** Load images only when they are visible in the viewport. This reduces initial page load time and improves performance. +* **Throttle and Debounce Events:** When using events like `keyup` or `mousemove` with `hx-trigger`, use the `throttle` and `debounce` modifiers to limit the number of requests sent to the server. + +### 3.2 Memory Management + +* **Avoid Memory Leaks:** Be careful to avoid memory leaks in your JavaScript code. Remove event listeners when they are no longer needed and avoid creating circular references. +* **Minimize DOM Elements:** Reduce the number of DOM elements on your page as much as possible. Complex layouts with many nested elements can impact performance. +* **Use Event Delegation:** Use event delegation to attach event listeners to a parent element instead of attaching them to individual child elements. This reduces the number of event listeners and improves performance. + +### 3.3 Rendering Optimization + +* **Use Efficient CSS Selectors:** Use efficient CSS selectors to minimize the time it takes to apply styles to your elements. Avoid complex selectors that traverse the DOM unnecessarily. +* **Avoid Reflows and Repaints:** Avoid triggering reflows and repaints in your JavaScript code. These operations can be expensive and impact performance. Read properties before making changes to the DOM, and batch your DOM updates. +* **Use Hardware Acceleration:** Take advantage of hardware acceleration to improve rendering performance. Use CSS properties like `transform` and `opacity` to trigger hardware acceleration. + +### 3.4 Bundle Size Optimization + +* **Minimize Dependencies:** Use only the dependencies that you absolutely need. Avoid including large libraries that you only use for a small number of features. +* **Tree Shaking:** Use a bundler that supports tree shaking to remove unused code from your JavaScript bundles. +* **Code Splitting:** Split your code into smaller chunks that can be loaded on demand. This reduces the initial download size and improves performance. + +### 3.5 Lazy Loading Strategies + +* **Lazy Load Images:** Load images only when they are visible in the viewport. This reduces initial page load time and improves performance. Use the `loading="lazy"` attribute on `` elements. +* **Lazy Load Components:** Load components only when they are needed. Use htmx to load the component's HTML when it is first requested. +* **Intersection Observer API:** Use the Intersection Observer API to detect when an element enters the viewport. This can be used to trigger the lazy loading of images or components. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and How to Prevent Them + +* **Cross-Site Scripting (XSS):** Prevent XSS attacks by escaping user input and avoiding the use of `innerHTML`. Use template engines that automatically escape output. +* **Cross-Site Request Forgery (CSRF):** Protect against CSRF attacks by using anti-CSRF tokens in your forms. Most server-side frameworks provide built-in CSRF protection. +* **SQL Injection:** Prevent SQL injection attacks by using parameterized queries or ORMs. Never construct SQL queries by concatenating user input. +* **Authentication and Authorization:** Implement proper authentication and authorization mechanisms to protect your application from unauthorized access. +* **Sensitive Data Exposure:** Avoid exposing sensitive data in URLs or client-side code. Store sensitive data securely on the server-side. + +### 4.2 Input Validation + +* **Client-Side Validation:** Use client-side validation to provide immediate feedback to the user and prevent invalid data from being submitted to the server. However, never rely solely on client-side validation. +* **Server-Side Validation:** Always validate user input on the server-side. This is the only way to ensure that your application is protected from invalid or malicious data. +* **Sanitize User Input:** Sanitize user input to remove or escape potentially harmful characters. This can help prevent XSS and other attacks. + +### 4.3 Authentication and Authorization Patterns + +* **Cookie-Based Authentication:** Use cookies to store authentication tokens. Set the `Secure`, `HttpOnly`, and `SameSite` attributes to protect your cookies from unauthorized access. +* **Session Management:** Use server-side sessions to store user-specific data. This prevents sensitive data from being exposed to the client. +* **Role-Based Access Control (RBAC):** Implement RBAC to control access to different parts of your application based on user roles. This ensures that users can only access the resources that they are authorized to access. + +### 4.4 Data Protection Strategies + +* **Encryption:** Encrypt sensitive data both in transit and at rest. Use HTTPS to encrypt data in transit, and use strong encryption algorithms to encrypt data at rest. +* **Hashing:** Use hashing to store passwords and other sensitive data. Never store passwords in plaintext. +* **Data Masking:** Mask sensitive data in your application's UI to prevent it from being exposed to unauthorized users. For example, you can mask credit card numbers or social security numbers. + +### 4.5 Secure API Communication + +* **HTTPS:** Always use HTTPS to encrypt communication between the client and server. +* **API Authentication:** Require API clients to authenticate themselves before accessing your API. Use API keys, OAuth, or other authentication mechanisms. +* **Rate Limiting:** Implement rate limiting to prevent API abuse. This limits the number of requests that a client can make in a given period of time. +* **Input Validation:** Validate all input to your API to prevent injection attacks and other security vulnerabilities. +* **Output Sanitization:** Sanitize all output from your API to prevent XSS attacks. +* **CORS (Cross-Origin Resource Sharing):** Configure CORS carefully to allow only trusted domains to access your API. + +### 4.6 The Golden Rules of htmx Security (reiterated): + +* **Only call routes you control**: Use relative URLs to avoid executing untrusted code. +* **Always use an auto-escaping template engine**: Prevent XSS attacks by ensuring dynamic content is escaped. +* **Only serve user-generated content inside HTML tags**: Avoid inserting user content in dangerous contexts like script or style tags. +* **Secure your cookies**: Use attributes like Secure, HttpOnly, and SameSite=Lax for authentication cookies. + +## 5. Testing Approaches + +### 5.1 Unit Testing Strategies + +* **Test Server-Side Logic:** Write unit tests for your server-side logic to ensure that it is working correctly. Use a testing framework like pytest (Python), Jest (Node.js), or JUnit (Java). +* **Test htmx Endpoints:** Write tests for your htmx endpoints. This could involve making requests to your endpoints and checking the HTML responses. +* **Mock External Dependencies:** Mock external dependencies, such as databases or APIs, to isolate your code and make your tests more reliable. + +### 5.2 Integration Testing + +* **Test Interactions Between Components:** Write integration tests to ensure that your components are working together correctly. This includes testing the interactions between htmx attributes and JavaScript code. +* **Test Data Flow:** Test the flow of data through your application to ensure that data is being processed and displayed correctly. + +### 5.3 End-to-End Testing + +* **Simulate User Interactions:** Write end-to-end tests to simulate user interactions with your application. This includes testing the entire workflow from start to finish. +* **Test UI Rendering:** Test the rendering of your UI to ensure that it is displaying correctly. This includes testing the layout, styles, and content of your pages. + +### 5.4 Test Organization + +* **Organize Tests by Feature:** Organize your tests by feature to make it easier to find and run tests for specific parts of your application. +* **Use a Consistent Naming Convention:** Use a consistent naming convention for your tests to make them easier to understand and maintain. +* **Keep Tests Independent:** Keep your tests independent of each other to avoid cascading failures. + +### 5.5 Mocking and Stubbing + +* **Use Mocking to Isolate Code:** Use mocking to isolate your code and make your tests more reliable. This involves replacing external dependencies with mock objects that simulate their behavior. +* **Use Stubbing to Control Behavior:** Use stubbing to control the behavior of your mock objects. This allows you to test different scenarios and edge cases. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes Developers Make + +* **Incorrect `hx-target` and `hx-swap` Usage:** Pay close attention to the `hx-target` and `hx-swap` attributes. Incorrect usage can lead to unexpected results and broken UI. +* **Forgetting Server-Side Validation:** Always validate user input on the server-side. Client-side validation is not sufficient for security. +* **Over-complicating Client-Side Logic:** htmx is designed to simplify client-side code. Avoid adding unnecessary complexity. +* **Ignoring Accessibility:** Make sure your htmx application is accessible to all users. Use semantic HTML, provide alternative text for images, and ensure that your application is keyboard-navigable. +* **Not handling edge cases and error conditions**: Ensure you have covered the most common error conditions, such as server errors, network issues, and no results. + +### 6.2 Edge Cases to Be Aware Of + +* **Handling Empty Responses:** Consider how your application should behave when the server returns an empty response. Should the target element be cleared, or should a default message be displayed? +* **Handling Concurrent Requests:** Be aware of the potential for concurrent requests. Use the `hx-sync` attribute to manage synchronization between requests. +* **Handling Browser History:** htmx does not automatically manage browser history. If you need to support browser history, you will need to use JavaScript or a dedicated htmx extension. +* **Unexpected interactions with browser extensions**: Some browser extensions, especially those that modify HTTP headers, can interfere with htmx's behavior. Test your application with common extensions enabled to identify and address any conflicts. + +### 6.3 Version-Specific Issues + +* **Refer to the Official Documentation:** Always refer to the official htmx documentation for the latest information and best practices. +* **Check Release Notes:** Review the release notes for each new version of htmx to identify any breaking changes or known issues. + +### 6.4 Compatibility Concerns + +* **Browser Compatibility:** htmx is compatible with modern browsers. However, older browsers may require polyfills. +* **Server-Side Framework Compatibility:** htmx can be used with any server-side framework that can generate HTML. However, some frameworks may require specific configuration. + +### 6.5 Debugging Strategies + +* **Use Browser Developer Tools:** Use your browser's developer tools to inspect network requests, examine the DOM, and debug JavaScript code. +* **htmx Events:** Listen for htmx events, such as `htmx:beforeRequest`, `htmx:afterRequest`, and `htmx:responseError`, to gain insight into what is happening in your application. +* **Server-Side Logging:** Add logging to your server-side code to help diagnose problems. +* **Simplify the Problem:** Break down complex htmx interactions into smaller, more manageable steps to isolate the source of the problem. +* **Inspect HTTP Headers:** Pay attention to the HTTP headers in both the request and response. Check the `Content-Type`, `HX-Request`, and other relevant headers. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +* **Text Editor or IDE:** Use a text editor or IDE with support for HTML, CSS, and JavaScript. Popular options include VS Code, Sublime Text, and Atom. +* **Browser Developer Tools:** Use your browser's developer tools to inspect network requests, examine the DOM, and debug JavaScript code. +* **htmx Extension for Chrome:** Install the htmx extension for Chrome to enable features like attribute highlighting and event inspection. +* **Server-Side Development Tools:** Use the development tools appropriate for your chosen server-side framework (e.g., Python debugger, Node.js inspector). + +### 7.2 Build Configuration + +* **Use a Task Runner:** Use a task runner like Gulp or Grunt to automate common development tasks, such as minifying CSS and JavaScript, optimizing images, and running tests. +* **Use a Module Bundler:** Use a module bundler like Webpack or Parcel to bundle your JavaScript modules into a single file. +* **Configure Code Linting and Formatting:** Integrate linters (like ESLint, stylelint) and formatters (like Prettier) to automatically enforce consistent coding styles and catch common errors. + +### 7.3 Linting and Formatting + +* **Use a Code Linter:** Use a code linter to automatically detect errors and enforce coding style guidelines. Popular linters include ESLint (JavaScript) and stylelint (CSS). +* **Use a Code Formatter:** Use a code formatter to automatically format your code according to a consistent style. Popular formatters include Prettier and js-beautify. + +### 7.4 Deployment Best Practices + +* **Use a Version Control System:** Use a version control system like Git to track changes to your code. This makes it easier to collaborate with others and revert to previous versions if necessary. +* **Automate Deployment:** Automate your deployment process using a tool like Jenkins, Travis CI, or CircleCI. This makes it easier to deploy your application to production quickly and reliably. +* **Use a Continuous Integration/Continuous Deployment (CI/CD) Pipeline:** Implement a CI/CD pipeline to automatically build, test, and deploy your application whenever changes are made to your codebase. +* **Monitor Your Application:** Monitor your application in production to detect and resolve problems quickly. + +### 7.5 CI/CD Integration + +* **Set Up a CI/CD Pipeline:** Set up a CI/CD pipeline to automatically build, test, and deploy your application. This makes it easier to release new features and bug fixes quickly and reliably. +* **Run Tests Automatically:** Configure your CI/CD pipeline to run your tests automatically whenever changes are made to your codebase. This helps to ensure that your code is working correctly before it is deployed to production. +* **Automate Deployment:** Automate your deployment process using a tool like Jenkins, Travis CI, or CircleCI. This makes it easier to deploy your application to production quickly and reliably. + +By adhering to these best practices and coding standards, you can build htmx applications that are robust, maintainable, secure, and performant. Remember to stay up-to-date with the latest htmx documentation and community resources to ensure that you are using the most effective techniques. \ No newline at end of file diff --git a/.cursor/rules/httpx.mdc b/.cursor/rules/httpx.mdc new file mode 100644 index 000000000..1da0d3038 --- /dev/null +++ b/.cursor/rules/httpx.mdc @@ -0,0 +1,255 @@ +--- +description: This rule provides comprehensive best practices for using the httpx library, covering code organization, performance, security, testing, and common pitfalls. Adhering to these guidelines will improve code quality, maintainability, and security when working with httpx. +globs: *.py +--- +# httpx Best Practices + +This document outlines recommended practices for using the `httpx` library in Python. Following these guidelines will help ensure maintainable, performant, and secure code. + +## Library Information: + +- Name: httpx +- Tags: web, python, http-client, async + +## 1. Code Organization and Structure + +### 1.1 Directory Structure Best Practices + +- **Dedicated Modules:** Organize httpx-related code into dedicated modules or packages. +- **Configuration:** Separate configuration settings (timeouts, headers, proxies) from the core logic. +- **Data Models:** Define data models (e.g., using `dataclasses` or `pydantic`) to represent request and response data. + +Example: + + +my_project/ +├── httpx_client/ +│ ├── __init__.py +│ ├── client.py # httpx client setup, session management +│ ├── utils.py # Utility functions for request/response handling +│ ├── models.py # Data models for requests and responses +│ └── exceptions.py # Custom exceptions +├── ... + + +### 1.2 File Naming Conventions + +- Use descriptive names that reflect the file's purpose (e.g., `api_client.py`, `request_utils.py`). +- Follow PEP 8 conventions (snake_case). + +### 1.3 Module Organization + +- **Layered Architecture:** Consider a layered architecture with modules for: + - **Client Initialization:** Handles httpx client setup (connection pooling, timeouts). + - **Request Building:** Constructs httpx Request objects. + - **Response Handling:** Parses and validates responses, handles errors. +- **Abstraction:** Abstract away direct httpx calls behind service functions or classes. + +### 1.4 Component Architecture + +- **Reusable Components:** Design reusable components for common tasks like: + - **Authentication:** Implementing custom authentication flows. + - **Rate Limiting:** Managing API rate limits. + - **Retry Logic:** Handling transient errors with retry mechanisms. + +### 1.5 Code Splitting Strategies + +- **Feature-Based Splitting:** Split code based on features or API endpoints. +- **Abstraction:** Use interfaces or abstract base classes for httpx clients to facilitate mocking and testing. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns + +- **Factory Pattern:** Use a factory function or class to create httpx clients with different configurations. +- **Strategy Pattern:** Implement different request strategies based on the API requirements. + +### 2.2 Recommended Approaches for Common Tasks + +- **Client Instance Usage:** Always use `httpx.Client()` or `httpx.AsyncClient()` for making requests. This enables connection pooling. +- **Context Managers:** Use the client as a context manager (`with httpx.Client() as client:`) or explicitly close the connection pool (`client.close()`). +- **Timeouts:** Always set timeouts to prevent hanging requests. +- **Error Handling:** Use `try-except` blocks to handle `HTTPStatusError` and `RequestError`. +- **Async Support:** Utilize `httpx.AsyncClient()` for concurrent requests. +- **Configuration Sharing:** Configure default headers, parameters, and authentication at the client level. + +python +import httpx + +with httpx.Client(timeout=10.0, headers={'User-Agent': 'MyApp/1.0'}) as client: + try: + response = client.get('https://example.com/api/data') + response.raise_for_status() # Raise HTTPStatusError for bad responses (4xx or 5xx) + data = response.json() + # Process data + except httpx.HTTPStatusError as e: + print(f"HTTP Error: {e}") + except httpx.RequestError as e: + print(f"Request Error: {e}") + + +### 2.3 Anti-patterns and Code Smells + +- **Global Client Instances:** Avoid creating global httpx client instances without proper management of their lifecycle. +- **Ignoring Errors:** Never ignore exceptions raised by httpx. Always handle them appropriately. +- **Hardcoding URLs:** Avoid hardcoding URLs directly in the code. Use configuration files or environment variables. + +### 2.4 State Management + +- **Statelessness:** Design components to be as stateless as possible. +- **Dependency Injection:** Use dependency injection to provide httpx clients to components that need them. + +### 2.5 Error Handling Patterns + +- **Custom Exceptions:** Define custom exception classes to represent specific httpx-related errors. +- **Logging:** Log errors and warnings with sufficient context. +- **Retry Logic:** Implement retry logic for transient errors. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +- **Connection Pooling:** httpx automatically uses connection pooling when using `httpx.Client()` or `httpx.AsyncClient()`. Ensure you are leveraging this by reusing client instances. +- **HTTP/2:** Enable HTTP/2 support if the server supports it. +- **Keep-Alive:** Ensure keep-alive connections are enabled. +- **Asynchronous Operations:** Use `httpx.AsyncClient()` for concurrent I/O-bound operations. + +### 3.2 Memory Management + +- **Streaming Responses:** Use response streaming (`response.iter_bytes()`, `response.iter_text()`) for large responses to avoid loading the entire response into memory. +- **Chunked Uploads:** Use chunked uploads for large file uploads. + +### 3.3 Bundle Size Optimization + +- **Tree Shaking:** Ensure your build tools perform tree shaking to remove unused httpx code. + +### 3.4 Lazy Loading + +- **On-Demand Initialization:** Initialize httpx clients only when they are needed. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and Prevention + +- **Injection Attacks:** Sanitize user inputs to prevent injection attacks. +- **Man-in-the-Middle Attacks:** Always use HTTPS and verify SSL certificates. +- **Sensitive Data Exposure:** Avoid logging sensitive data (API keys, passwords). + +### 4.2 Input Validation + +- **Validate Request Data:** Validate all data before sending it to the API. +- **Validate Response Data:** Validate responses from the API. + +### 4.3 Authentication and Authorization + +- **Secure Authentication:** Use secure authentication mechanisms (OAuth 2.0, JWT). +- **Least Privilege:** Grant only the necessary permissions to users. + +### 4.4 Data Protection + +- **Encryption:** Encrypt sensitive data at rest and in transit. +- **Data Masking:** Mask sensitive data in logs and error messages. + +### 4.5 Secure API Communication + +- **HTTPS:** Always use HTTPS. +- **TLS 1.3:** Use TLS 1.3 or higher. +- **Certificate Pinning:** Consider certificate pinning for added security (advanced). + +## 5. Testing Approaches + +### 5.1 Unit Testing + +- **Mocking:** Use mocking libraries (e.g., `unittest.mock`, `pytest-mock`) to mock httpx calls. +- **Isolate Components:** Isolate components that use httpx for unit testing. + +### 5.2 Integration Testing + +- **Test Against Real APIs:** Perform integration tests against real APIs (with appropriate mocking or stubbing). + +### 5.3 End-to-End Testing + +- **Simulate User Flows:** Simulate end-to-end user flows that involve httpx calls. + +### 5.4 Test Organization + +- **Separate Test Files:** Create separate test files for each module or component. + +### 5.5 Mocking and Stubbing Techniques + +- **Mock Responses:** Mock httpx responses to simulate different API scenarios. +- **Patch httpx Functions:** Patch httpx functions to control their behavior during tests. +- **RESPX:** Use RESPX for advanced mocking of httpx requests and responses. + +python +import httpx +import pytest +import respx + +@respx.mock +async def test_get_data(): + respx.get("https://example.com/api/data").mock(return_value=httpx.Response(200, json={"key": "value"})) + async with httpx.AsyncClient() as client: + response = await client.get("https://example.com/api/data") + assert response.status_code == 200 + assert response.json() == {"key": "value"} + + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +- **Forgetting to Close Clients:** Always close httpx clients to release resources. +- **Ignoring Timeouts:** Not setting timeouts can lead to hanging requests. +- **Incorrect Error Handling:** Improper error handling can mask critical errors. +- **Misunderstanding Connection Pooling:** Not reusing client instances negates connection pooling benefits. +- **Not Handling Asynchronous Context Correctly**: Using `await` improperly when using `AsyncClient()`. + +### 6.2 Edge Cases + +- **Character Encoding Issues:** Handle character encoding issues when processing text responses. +- **SSL Certificate Errors:** Handle SSL certificate errors gracefully. +- **Network Connectivity Issues:** Implement robust error handling for network connectivity issues. + +### 6.3 Version-Specific Issues + +- **Consult httpx Documentation:** Refer to the httpx documentation for version-specific issues and compatibility notes. + +### 6.4 Compatibility Concerns + +- **Asynchronous Libraries:** Ensure compatibility with other asynchronous libraries (e.g., asyncio). +- **Third-Party Packages:** Be aware of compatibility issues with third-party packages that use httpx. + +### 6.5 Debugging Strategies + +- **Logging:** Use logging to trace httpx requests and responses. +- **Debugging Tools:** Use debugging tools (e.g., pdb, ipdb) to inspect httpx code. +- **Wireshark/tcpdump:** Use network analysis tools (Wireshark, tcpdump) to capture and analyze network traffic. + +## 7. Tooling and Environment + +### 7.1 Recommended Tools + +- **Development Environment:** Use a modern IDE (e.g., VS Code, PyCharm). +- **Dependency Management:** Use a dependency management tool (e.g., pipenv, poetry). +- **Testing Framework:** Use a testing framework (e.g., pytest, unittest). +- **Mocking Libraries:** Use mocking libraries (e.g., `unittest.mock`, `pytest-mock`, RESPX). + +### 7.2 Build Configuration + +- **Reproducible Builds:** Ensure reproducible builds using dependency locking. + +### 7.3 Linting and Formatting + +- **PEP 8 Compliance:** Use a linter (e.g., flake8, pylint) to enforce PEP 8 compliance. +- **Code Formatting:** Use a code formatter (e.g., black, autopep8) to automatically format code. + +### 7.4 Deployment + +- **Containerization:** Use containerization (e.g., Docker) to ensure consistent deployments. +- **Environment Variables:** Configure httpx clients using environment variables. + +### 7.5 CI/CD Integration + +- **Automated Testing:** Integrate automated testing into your CI/CD pipeline. +- **Code Analysis:** Integrate code analysis tools into your CI/CD pipeline. \ No newline at end of file diff --git a/.cursor/rules/huggingface.mdc b/.cursor/rules/huggingface.mdc new file mode 100644 index 000000000..26e0d6d65 --- /dev/null +++ b/.cursor/rules/huggingface.mdc @@ -0,0 +1,342 @@ +--- +description: This rule provides guidelines for best practices when working with the Hugging Face Transformers library, covering code organization, performance, testing, security, and common pitfalls. It emphasizes community standards and maintainability. +globs: *.py +--- +# Hugging Face Transformers Library Best Practices + +This document outlines best practices and coding standards for using the Hugging Face Transformers library. It covers various aspects, including code organization, performance, testing, security, and common pitfalls. + +## Philosophy: Embrace the Single Model File Policy + +- **Single Model File Policy:** Prioritize having all the code for a model's forward pass within a single file. While it might seem to contradict the DRY (Don't Repeat Yourself) principle, this improves readability and accessibility, especially for newcomers. This makes it easier to understand the entire model architecture without navigating through multiple files. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure + +While the Transformers library promotes the single model file policy, good project-level organization is still crucial. A typical project structure might look like this: + + +my_project/ +├── data/ +│ ├── raw/ +│ ├── processed/ +│ └── ... +├── models/ +│ ├── modeling_bert.py # Example: BERT model definition +│ ├── ... +├── scripts/ +│ ├── train.py +│ ├── evaluate.py +│ ├── preprocess.py +│ └── ... +├── utils/ +│ ├── data_utils.py +│ ├── model_utils.py +│ └── ... +├── tests/ +│ ├── test_models.py +│ ├── test_data_utils.py +│ └── ... +├── notebooks/ # optional +│ ├── exploration.ipynb +│ └── ... +├── requirements.txt +├── pyproject.toml +└── README.md + + +- **`data/`:** Stores raw and processed datasets. +- **`models/`:** Contains model definitions (adhering to the single model file policy). +- **`scripts/`:** Holds training, evaluation, and preprocessing scripts. +- **`utils/`:** Includes utility functions for data handling, model management, etc. +- **`tests/`:** Contains unit, integration, and end-to-end tests. +- **`notebooks/`:** (Optional) Jupyter notebooks for experimentation. + +### 1.2 File Naming Conventions + +- Use descriptive and consistent file names. +- For model files, follow the naming convention `modeling_.py` (e.g., `modeling_bert.py`). +- For utility files, use `*_utils.py` (e.g., `data_utils.py`). +- Use lowercase letters and underscores for file names (snake_case). + +### 1.3 Module Organization + +- Keep modules focused and avoid creating overly large modules. Each module should have a clear responsibility. +- Import necessary modules within functions or classes to improve readability and avoid unnecessary dependencies at the module level. +- Use relative imports within the project structure (e.g., `from . import data_utils`) to avoid conflicts and improve portability. + +### 1.4 Component Architecture + +- Design models as modular components. While adhering to the single model file policy, the internal structure of the model can be highly modular. Use classes to encapsulate different parts of the model (e.g., embedding layer, attention mechanism, feedforward network). +- Each component should have a well-defined interface. +- Leverage inheritance when appropriate to share common functionality between similar components. + +### 1.5 Code Splitting Strategies + +- For extremely large models, consider splitting the model definition into multiple files, but maintain a clear entry point that encapsulates the entire model. A primary file should import and orchestrate the other components. +- Consider splitting preprocessing and postprocessing logic into separate modules to improve readability and maintainability. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns Specific to Hugging Face + +- **Configuration-Driven Design:** Use `PretrainedConfig` objects to define model parameters. This allows for easy configuration and sharing of model architectures. +- **Tokenizers and Pipelines:** Leverage the built-in tokenizers and pipelines for efficient text processing. Understand how to customize them for specific tasks. +- **Model Cards:** Create comprehensive model cards that document the model's architecture, training data, evaluation metrics, and intended use cases. This is crucial for reproducibility and transparency. + +### 2.2 Recommended Approaches for Common Tasks + +- **Loading Pre-trained Models:** Use `AutoModel` and `AutoTokenizer` classes to automatically load the appropriate model and tokenizer based on the model name. + + python + from transformers import AutoModelForSequenceClassification, AutoTokenizer + + model_name = "bert-base-uncased" + model = AutoModelForSequenceClassification.from_pretrained(model_name) + tokenizer = AutoTokenizer.from_pretrained(model_name) + + +- **Fine-tuning Models:** Use the `Trainer` class for efficient fine-tuning. Customize the training loop when necessary for advanced control. + + python + from transformers import Trainer, TrainingArguments + + training_args = TrainingArguments( + output_dir="./results", + evaluation_strategy="epoch", + num_train_epochs=3 + ) + + trainer = Trainer( + model=model, + args=training_args, + train_dataset=train_dataset, + eval_dataset=eval_dataset + ) + + trainer.train() + + +- **Inference:** Use pipelines for simple inference tasks or write custom inference loops for more control. + + python + from transformers import pipeline + + classifier = pipeline("sentiment-analysis") + result = classifier("This is a great movie!") + + +### 2.3 Anti-patterns and Code Smells to Avoid + +- **Ignoring `model_max_length`:** When initializing tokenizers and models, be mindful of the `model_max_length` parameter. Ensure that input sequences are properly truncated or padded to avoid errors or performance issues. +- **Hardcoding Paths:** Avoid hardcoding file paths. Use relative paths or environment variables to make the code more portable. +- **Lack of Documentation:** Write clear and concise documentation for all functions, classes, and modules. This is especially important given the single model file policy, as comprehensive inline documentation becomes critical. +- **Over-Complicating Model Definitions:** Keep model definitions as simple as possible while still meeting the requirements. Avoid unnecessary complexity. +- **Ignoring Warnings:** Pay attention to warnings raised by the Transformers library. They often indicate potential issues. + +### 2.4 State Management Best Practices + +- Avoid global variables. Encapsulate state within classes or functions. +- Use immutable data structures when possible to prevent accidental modifications. +- Manage model weights carefully. Load and save weights using the `state_dict` method. + + python + # Save model weights + torch.save(model.state_dict(), "model.pth") + + # Load model weights + model.load_state_dict(torch.load("model.pth")) + + +### 2.5 Error Handling Patterns + +- Use `try...except` blocks to handle potential exceptions, such as file not found errors or invalid input data. +- Provide informative error messages to help users understand the cause of the error. +- Use logging to record errors and debug information. + + python + import logging + + logging.basicConfig(level=logging.INFO) + + try: + model = AutoModelForSequenceClassification.from_pretrained("invalid-model") + except OSError as e: + logging.error(f"Failed to load model: {e}") + + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +- **Mixed Precision Training:** Use mixed precision training (e.g., using `torch.cuda.amp` in PyTorch) to reduce memory usage and improve training speed. +- **Gradient Accumulation:** Use gradient accumulation to simulate larger batch sizes when memory is limited. +- **Quantization:** Quantize models to reduce their size and improve inference speed. +- **ONNX Runtime:** Convert models to ONNX format and use the ONNX Runtime for faster inference. + +### 3.2 Memory Management + +- Delete unused variables and tensors to free up memory. +- Use `torch.no_grad()` during inference to disable gradient calculations and reduce memory usage. +- Consider using techniques like gradient checkpointing to reduce memory usage during training, especially for very large models. + +### 3.3 Rendering Optimization (If Applicable) + +- If the application involves rendering model outputs (e.g., text generation), optimize the rendering process to minimize latency and improve user experience. +- Use efficient text rendering libraries. +- Cache rendered outputs when possible. + +### 3.4 Bundle Size Optimization (If Applicable) + +- If the application is deployed as a web application or mobile app, optimize the bundle size to reduce download times. +- Use code splitting to load only the necessary code for each feature. +- Use minification and compression to reduce the size of JavaScript and CSS files. + +### 3.5 Lazy Loading Strategies + +- Load models and data lazily to reduce startup time and memory usage. +- Use generators to process large datasets in chunks. +- Only load the parts of the model that are needed for the current task. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and How to Prevent Them + +- **Prompt Injection:** Be aware of prompt injection attacks, where malicious users craft inputs that can manipulate the model's behavior. Implement input validation and sanitization techniques to mitigate this risk. +- **Data Poisoning:** Protect against data poisoning attacks by carefully vetting training data sources. +- **Model Stealing:** Implement measures to protect model weights from being stolen. Consider using encryption or access controls. + +### 4.2 Input Validation + +- Validate all inputs to ensure that they are within the expected range and format. +- Sanitize inputs to remove potentially harmful characters or code. +- Use regular expressions to enforce input constraints. + +### 4.3 Authentication and Authorization Patterns + +- Implement authentication to verify the identity of users. +- Use authorization to control access to resources based on user roles. +- Store credentials securely using encryption and hashing. + +### 4.4 Data Protection Strategies + +- Encrypt sensitive data at rest and in transit. +- Use access controls to restrict access to data. +- Anonymize or pseudonymize data to protect user privacy. + +### 4.5 Secure API Communication + +- Use HTTPS to encrypt communication between the client and the server. +- Use API keys or tokens to authenticate requests. +- Implement rate limiting to prevent abuse. + +## 5. Testing Approaches + +### 5.1 Unit Testing Strategies + +- Write unit tests for all individual functions, classes, and modules. +- Use a testing framework like `pytest` or `unittest`. +- Test edge cases and boundary conditions. + +### 5.2 Integration Testing + +- Write integration tests to verify the interaction between different components of the system. +- Test the flow of data through the system. +- Use mocks and stubs to isolate components during testing. + +### 5.3 End-to-end Testing + +- Write end-to-end tests to verify the entire system from end to end. +- Simulate user interactions to test the system's functionality. +- Use a testing framework like `Selenium` or `Cypress` for web applications. + +### 5.4 Test Organization + +- Organize tests into separate files or directories based on the component being tested. +- Use descriptive names for test functions and classes. +- Keep tests concise and focused. + +### 5.5 Mocking and Stubbing + +- Use mocks and stubs to isolate components during testing. +- Mock external dependencies, such as API calls or database connections. +- Use a mocking library like `unittest.mock` or `pytest-mock`. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes Developers Make + +- **Incorrect Tokenization:** Using the wrong tokenizer or not properly handling special tokens can lead to inaccurate results. +- **Oversized Inputs:** Forgetting to truncate or pad input sequences to the model's maximum length can cause errors or performance issues. +- **Incorrect Device Placement:** Running models on the CPU when a GPU is available can significantly slow down training and inference. +- **Ignoring Data Preprocessing:** Failing to properly preprocess data (e.g., cleaning, normalizing) can negatively impact model performance. + +### 6.2 Edge Cases to be Aware Of + +- **Rare Words:** Handle rare or out-of-vocabulary words gracefully. +- **Long Sequences:** Models may struggle with extremely long sequences. +- **Adversarial Examples:** Models can be fooled by adversarial examples. + +### 6.3 Version-Specific Issues + +- Be aware of breaking changes in new versions of the Transformers library. +- Check the release notes for details on any changes that may affect your code. +- Use version pinning in `requirements.txt` or `pyproject.toml` to ensure that your code runs with the correct version of the library. + +### 6.4 Compatibility Concerns + +- Ensure that your code is compatible with the target hardware and software environment. +- Test your code on different platforms and configurations. +- Use conditional imports to handle different dependencies. + +### 6.5 Debugging Strategies + +- Use logging to track the flow of execution and identify errors. +- Use a debugger to step through the code and inspect variables. +- Use print statements to output intermediate values. +- Use a profiler to identify performance bottlenecks. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +- **IDE:** Visual Studio Code, PyCharm +- **Version Control:** Git +- **Package Manager:** pip, conda, uv +- **Testing Framework:** pytest +- **Linting:** pylint, flake8 +- **Formatting:** black, autopep8 + +### 7.2 Build Configuration + +- Use `requirements.txt` or `pyproject.toml` to manage dependencies. +- Use a build system like `Make` or `Poetry` to automate the build process. +- Create a virtual environment for each project to isolate dependencies. + +### 7.3 Linting and Formatting + +- Use a linter to enforce code style guidelines. +- Use a formatter to automatically format code. +- Configure the linter and formatter to run automatically on save. + +### 7.4 Deployment Best Practices + +- Containerize your application using Docker. +- Use a cloud platform like AWS, Azure, or GCP to deploy your application. +- Use a deployment tool like Kubernetes to manage your application. + +### 7.5 CI/CD Integration + +- Use a CI/CD pipeline to automate the testing and deployment process. +- Use a CI/CD tool like Jenkins, GitHub Actions, or GitLab CI. +- Run tests automatically on every commit. +- Deploy the application automatically when tests pass. + +## Additional Recommendations + +- **Utilize NVIDIA RTX GPUs:** Leverage RTX GPUs for enhanced performance due to their high-speed VRAM and dedicated AI accelerators. This is especially important for larger models and higher batch sizes. +- **Token Management:** Understand how to manage tokens effectively to optimize performance, as performance is often measured in tokens per second. +- **Batch API:** Consider using the OpenAI Batch API for cost efficiency and higher rate limits, especially when immediate responses are not required. + +By following these best practices, you can write more maintainable, efficient, and secure code when working with the Hugging Face Transformers library. \ No newline at end of file diff --git a/.cursor/rules/hypothesis.mdc b/.cursor/rules/hypothesis.mdc new file mode 100644 index 000000000..6c724a73d --- /dev/null +++ b/.cursor/rules/hypothesis.mdc @@ -0,0 +1,164 @@ +--- +description: Comprehensive guide covering best practices for the Hypothesis Python library, including coding standards, testing, performance, and security. Provides actionable guidance for developers to write maintainable, robust, and efficient property-based tests. +globs: *.py +--- +# Hypothesis Library Best Practices and Coding Standards + +This document provides comprehensive guidelines for using the Hypothesis Python library effectively. Following these best practices will lead to more maintainable, robust, and efficient property-based tests. + +## I. General Python Coding Standards (Applicable to Hypothesis Projects) + +These are general Python coding standards that should be adhered to in any Python project, including those using Hypothesis. + +* **PEP 8 Compliance:** Adhere strictly to PEP 8 style guidelines for Python code. This includes: + * Indentation: Use 4 spaces per indentation level. + * Line Length: Limit lines to a maximum of 79 characters (or 72 for comments and docstrings). + * Blank Lines: Use blank lines to separate top-level function and class definitions (two lines) and method definitions inside a class (one line). + * Imports: Group imports into standard library, third-party, and local application/library-specific imports, separated by blank lines. + * Whitespace: Avoid extraneous whitespace. + * Naming Conventions: Follow appropriate naming conventions for variables, functions, classes, and constants. +* **PEP 257 Compliance:** Follow PEP 257 for docstring conventions. + * Write docstrings for all public modules, functions, classes, and methods. + * Use triple double quotes (`"""`) for docstrings. + * Include a summary line, use case (if appropriate), arguments, return type and semantics (unless None is returned), and examples in multiline docstrings. +* **Use Python 3.12+:** Ensure that your project uses a modern version of Python (3.12 or higher) for access to the latest features and security updates. +* **Virtual Environments:** Use virtual environments (e.g., `venv` or `virtualenv`) to isolate project dependencies. +* **Dependency Management:** Use a dependency management tool like `pip` with a `requirements.txt` or `pyproject.toml` file to specify project dependencies. It's recommended to use a modern package installer such as [UV](https://astral.sh/blog/uv) for faster and more reproducible builds. +* **Code Reviews:** Conduct regular code reviews to ensure code quality and adherence to coding standards. +* **Type Hinting:** Use type hints to improve code readability and enable static analysis. +* **Linters and Formatters:** Use linters (e.g., `flake8`, `pylint`) and formatters (e.g., `black`, `autopep8`) to automatically enforce coding standards. +* **Error Handling:** Implement robust error handling using `try...except` blocks. +* **Logging:** Use the `logging` module for structured logging. +* **Clear and Concise Code:** Write code that is easy to understand and maintain. +* **Modularity:** Design your code with a modular architecture. +* **Documentation:** Write clear and comprehensive documentation. +* **Follow SOLID principles** When designing code and modules. + +## II. Hypothesis-Specific Best Practices + +These guidelines are specifically tailored to the Hypothesis library for property-based testing. + +### A. Writing Properties + +* **Focus on Properties, Not Examples:** Hypothesis excels when you define *properties* that should always hold true for a wide range of inputs, rather than just providing a fixed set of examples. A property describes a general relationship that should be invariant. +* **Define Strategies Clearly:** Strategies are the core of Hypothesis. Define strategies that generate diverse and representative data for your properties. Hypothesis provides many built-in strategies, and you can combine and customize them. +* **Shrinking is Crucial:** Hypothesis shrinks failing examples to the smallest and simplest form that still causes the failure. This simplifies debugging. Ensure your strategies are designed to allow effective shrinking. +* **Avoid Direct Assertion:** Generally, avoid directly asserting specific values within your properties. Instead, focus on asserting relationships between inputs and outputs. This strengthens the test and covers more potential cases. +* **Use Assume Sparingly:** The `assume()` function tells Hypothesis to discard certain examples. Overuse of `assume()` can make your tests less effective and potentially mask bugs. Try to refine your strategies instead of filtering excessively. +* **Make Properties Declarative:** Properties should describe *what* should happen, not *how* it should happen. This makes the tests more robust and easier to understand. +* **Test Invariants, Not Implementations:** Properties should test the high-level invariants of your code, rather than implementation details. This makes the tests more resilient to code changes. +* **Minimize Side Effects:** Properties should be as pure as possible, with minimal side effects. This makes the tests more predictable and easier to debug. +* **Consider Edge Cases:** Think about edge cases and boundary conditions that might violate your properties. Design your strategies to include these cases. +* **Use Composite Strategies:** Composite strategies allow you to combine multiple strategies to generate complex data structures. Use them to create realistic and diverse test cases. +* **Label Strategies:** Label strategies with descriptive names to improve the readability of your test output. +* **Use Dataclasses for Structured Data:** Define dataclasses to represent structured data in your properties. This improves code readability and maintainability. + +### B. Strategy Design + +* **Start with Built-in Strategies:** Hypothesis provides a rich set of built-in strategies for generating common data types. Start with these strategies and customize them as needed. +* **Constrain Strategies Appropriately:** Use constraints (e.g., `min_value`, `max_value`, `length`) to narrow the range of values generated by your strategies and improve test performance. +* **Use `sampled_from` for Discrete Values:** The `sampled_from` strategy is useful for generating discrete values from a list or set. +* **Combine Strategies with `one_of`:** The `one_of` strategy allows you to combine multiple strategies to generate a variety of data types. +* **Use `lists`, `sets`, and `dictionaries` to Generate Collections:** These strategies generate collections of data. Specify the element type and size constraints as needed. +* **Consider `recursive` Strategies for Nested Structures:** The `recursive` strategy allows you to generate nested data structures, such as trees or graphs. +* **Write Custom Strategies When Necessary:** If the built-in strategies are not sufficient, you can write custom strategies to generate data specific to your application. +* **Use `register_type_strategy` for Custom Types:** Register custom strategies for your custom types to make them available to other strategies. +* **Think About Shrinking When Designing Strategies:** Design your strategies with shrinking in mind. The goal is to shrink failing examples to the smallest and simplest form possible. + +### C. Code Organization + +* **Separate Tests from Implementation:** Keep your hypothesis tests in a separate directory from your implementation code (e.g., `tests/`). +* **Organize Tests by Module:** Mirror the structure of your implementation code in your test directory. Create a test module for each implementation module. +* **Use Descriptive Test Names:** Give your tests descriptive names that indicate the property being tested. +* **Use Fixtures for Setup and Teardown:** Use pytest fixtures to handle setup and teardown tasks for your tests. +* **Create Helper Functions for Common Tasks:** Create helper functions to encapsulate common tasks in your tests. +* **Use a `conftest.py` file:** Use a `conftest.py` file to define fixtures and other configuration options that are shared across multiple test modules. +* **Keep Test Modules Small:** Break large test modules into smaller, more manageable modules. + +### D. Error Handling + +* **Test for Expected Exceptions:** Use `pytest.raises` to assert that your code raises the expected exceptions under certain conditions. +* **Use Context Managers for Exception Handling:** Use context managers to handle exceptions and ensure that resources are properly released. +* **Log Errors and Warnings:** Use the `logging` module to log errors and warnings in your tests. +* **Use `report` to Add Context to Failures:** Use `hypothesis.report` to add information about the current state of the system under test to the failure report. +* **Fail Fast:** Write tests that fail quickly when a property is violated. + +### E. Performance Considerations + +* **Minimize Computation in Properties:** Properties should be as lightweight as possible to avoid slowing down the tests. +* **Use Caching to Avoid Redundant Computations:** Use caching to avoid redundant computations in your properties. +* **Optimize Strategies for Performance:** Optimize your strategies to generate data efficiently. +* **Use `max_examples` to Limit Test Run Time:** Use the `max_examples` setting to limit the number of examples generated by Hypothesis and prevent tests from running indefinitely. +* **Profile Slow Tests:** Use profiling tools to identify slow tests and optimize their performance. +* **Consider Data Generation Overhead:** Be aware that complex strategy definitions can have a significant performance overhead. +* **Avoid Unnecessary `assume` Calls:** Excessive use of `assume` can lead to wasted test cycles. + +### F. Security Best Practices + +* **Test for Input Validation:** Use Hypothesis to test the input validation logic of your code. Generate a wide range of inputs, including invalid inputs, to ensure that your code handles them correctly. +* **Test for Common Vulnerabilities:** Use Hypothesis to test for common vulnerabilities, such as SQL injection, cross-site scripting (XSS), and buffer overflows. +* **Use Strategies to Generate Malicious Inputs:** Create strategies that generate malicious inputs to test the security of your code. +* **Sanitize Inputs:** Sanitize all inputs to prevent vulnerabilities. +* **Escape Outputs:** Escape all outputs to prevent vulnerabilities. +* **Use Secure API Communication:** Use HTTPS for secure API communication. +* **Implement Authentication and Authorization:** Implement authentication and authorization to protect your data. +* **Limit User Privileges:** Limit user privileges to the minimum necessary to perform their tasks. +* **Monitor Your Application for Security Threats:** Monitor your application for security threats and respond promptly to any incidents. + +### G. Common Pitfalls and Gotchas + +* **Overuse of `assume()`:** As mentioned before, overuse of `assume()` can hide bugs. +* **Complex Strategy Definitions:** Complex strategy definitions can be difficult to understand and maintain. +* **Unrealistic Test Data:** Generating unrealistic test data can lead to false positives or false negatives. +* **Slow Test Execution:** Slow test execution can make it difficult to iterate quickly. +* **Lack of Understanding of Shrinking:** A lack of understanding of shrinking can make it difficult to debug failing tests. +* **Not Testing Edge Cases:** Failing to test edge cases can lead to bugs in production. +* **Ignoring Hypothesis Output:** Ignoring the Hypothesis output can cause you to miss important information about failing tests. +* **Incorrect Type Annotations:** Incorrect type annotations can cause Hypothesis to generate incorrect data. +* **Misunderstanding of Strategy Composition:** Misunderstanding how strategies are composed can lead to unexpected results. + +### H. Tooling and Environment + +* **pytest:** Use pytest as your test runner. It integrates seamlessly with Hypothesis and provides a rich set of features for writing and running tests. +* **hypothesis-auto:** Use the `hypothesis-auto` extension to automatically generate strategies for your dataclasses and other data types. +* **black:** Use black to automatically format your code and ensure PEP 8 compliance. +* **flake8:** Use flake8 to lint your code and identify potential problems. +* **mypy:** Use mypy to statically type check your code and identify type errors. +* **tox:** Use tox to automate testing in multiple environments. +* **CI/CD:** Integrate your tests into your CI/CD pipeline to ensure that your code is always tested before it is deployed. +* **VS Code or PyCharm:** Use a good IDE (like VS Code with the Python extension, or PyCharm) for code editing, debugging, and testing. + +### I. Testing Approaches + +* **Unit Testing:** Focus on testing individual units of code in isolation. +* **Integration Testing:** Test how different units of code interact with each other. +* **End-to-End Testing:** Test the entire application from end to end. +* **Property-Based Testing:** Use Hypothesis to generate a wide range of inputs and test the properties of your code. +* **Mutation Testing:** Use mutation testing to assess the quality of your tests. + +### J. Recommended Workflow + +1. **Start with a Property:** Define the property that you want to test. +2. **Define a Strategy:** Create a strategy to generate data for your property. +3. **Write the Test:** Write the test using Hypothesis and pytest. +4. **Run the Test:** Run the test and see if it passes. +5. **Fix Any Bugs:** If the test fails, fix the bug in your code. +6. **Repeat:** Repeat steps 1-5 until you are confident that your code is correct. + +## III. Example Hypothesis Test + +python +import pytest +from hypothesis import given, strategies as st + +def add(x, y): + return x + y + +@given(st.integers(), st.integers()) +def test_addition_is_commutative(x, y): + assert add(x, y) == add(y, x) + + +## IV. Conclusion + +By following these best practices, you can use Hypothesis to write more effective and robust property-based tests that improve the quality and reliability of your Python code. \ No newline at end of file diff --git a/.cursor/rules/insomnia.mdc b/.cursor/rules/insomnia.mdc new file mode 100644 index 000000000..b23d65ff1 --- /dev/null +++ b/.cursor/rules/insomnia.mdc @@ -0,0 +1,91 @@ +--- +description: This rule file provides best practices for using the Insomnia API Client, including project organization, environment management, testing, and collaboration to improve API development workflows. +globs: *.json,*.yaml,*.yml,*.graphql,*.proto +--- +- **Project Organization and Structure:** + - **Workspace Structure:** Organize requests into workspaces based on projects or functional areas. Within workspaces, use folders to group related requests (e.g., authentication, user management, product catalog). + - **Environment Management:** Leverage Insomnia's environment variables for managing configurations (API keys, base URLs, authentication tokens) across different environments (development, staging, production). Use environment variables to store sensitive data and secrets instead of hardcoding them directly into requests. Create separate environments for each stage of the API lifecycle (development, testing, staging, production). This allows you to easily switch between different configurations without modifying your requests. + - **File Naming Conventions:** Use descriptive file names for Insomnia collections (e.g., `users-api.insomnia.json`, `product-management.insomnia.yaml`). Ensure that your naming convention is clear and consistent. Name files according to their function and the API they're targeting. Using date-based versioning can also be valuable. + - **Collection Splitting:** For large API definitions, consider splitting collections into smaller, more manageable files based on API modules or resources. Use Insomnia's import/export functionality to combine them when needed. + - **Git Integration:** Store your Insomnia collections and environment files in a Git repository to track changes, collaborate with team members, and implement version control. Using Git also gives you a backup of your configuration. + +- **Common Patterns and Anti-patterns:** + - **Design Patterns:** + - **Environment Overrides:** Use environment overrides to customize settings for specific requests without modifying the base environment. + - **Chained Requests:** Utilize chained requests to pass data from one request to another, enabling complex workflows (e.g., creating a user, then retrieving the user's details). + - **Dynamic Values:** Generate dynamic values (e.g., timestamps, UUIDs) using Insomnia's built-in functions to create realistic test data. + - **Recommended Approaches:** + - **API Testing:** Use Insomnia to test HTTP-based RESTful APIs as well as GraphQL APIs. Take advantage of API testing as part of your testing strategy to test your application's core business rules and help deliver better software faster. + - **GraphQL Support:** Leverage Insomnia's built-in GraphQL support for constructing and executing GraphQL queries. + - **Authentication Handling:** Configure authentication (API keys, OAuth 2.0, JWT) at the environment or request level for consistent authentication across all requests. + - **Anti-patterns:** + - **Hardcoding Sensitive Data:** Avoid storing sensitive information directly in request bodies or headers. Use environment variables instead. + - **Ignoring Response Validation:** Always validate API responses (status codes, headers, data) to ensure that the API is functioning correctly. + - **State Management:** Insomnia itself is stateless; however, you can simulate stateful interactions using chained requests and environment variables to store and reuse data across multiple requests. + - **Error Handling:** Use Insomnia's response inspection tools to analyze error responses. Define specific tests to assert that error responses are handled correctly. + +- **Performance Considerations:** + - **Request Optimization:** Minimize request payload size and optimize request headers to improve API performance. + - **Connection Pooling:** Insomnia automatically manages connection pooling for efficient API communication. + - **Response Caching (Client-Side):** Consider using a proxy or interceptor to cache API responses locally for faster testing and development. + +- **Security Best Practices:** + - **Vulnerability Prevention:** + - **Rate Limiting:** Simulate rate limiting scenarios to test your API's resilience to abuse. + - **Input Validation:** Thoroughly validate all inputs (headers, query parameters, request body) to prevent injection attacks. + - **Input Validation:** Utilize Insomnia's request body editor to ensure that the data being sent is correctly formatted and validated against the API's expected schema. + - **Authentication and Authorization:** + - **OAuth 2.0 Flows:** Implement OAuth 2.0 flows within Insomnia to test API authorization. + - **JWT Validation:** Validate JWT tokens returned by the API to ensure their integrity. + - **Data Protection:** + - **HTTPS:** Always use HTTPS for secure API communication. Insomnia automatically handles HTTPS requests. + - **End-to-End Encryption:** Ensure that sensitive data is encrypted both in transit and at rest. Consider using a VPN for added security. + - **Security Helpers**: Take advantage of security helpers, code creation, and environment variables. + +- **Testing Approaches:** + - **Unit Testing:** While Insomnia isn't a unit testing framework, you can use it to create individual requests that target specific API endpoints and validate their responses. + - **Integration Testing:** Use Insomnia to test the interaction between different API services and components. + - **End-to-End Testing:** Create comprehensive test suites in Insomnia to simulate real-world user scenarios and validate the entire API workflow. + - **Test Organization:** Organize tests into folders based on API resources or functional areas. + - **Mocking and Stubbing:** Insomnia's mocking capabilities are essential during early stages of development or when dependent services are not available. Kong Insomnia’s mocking capabilities allow teams to simulate API responses without needing access to the actual API or back-end systems, facilitating parallel development and ensuring seamless integration testing. Kong Insomnia offers the option to mock on cloud or local for data sensitive customers. + - **Test Automation:** Integrate Insomnia with CI/CD pipelines to automate API testing as part of your development workflow. Utilize the Insomnia CLI or Newman (Postman's CLI tool) to run your Insomnia collections as part of your automated test suite. + - **Assertions:** + - Validate the structure and content of API responses using Insomnia's response inspection tools and custom JavaScript assertions. + - Verify HTTP status codes, headers, and response times to ensure that the API is performing as expected. + +- **Common Pitfalls and Gotchas:** + - **Incorrect Environment Variables:** Double-check that environment variables are correctly configured and referenced in your requests. + - **Missing Authentication Headers:** Ensure that all authenticated requests include the necessary authentication headers. + - **Invalid Request Payloads:** Validate that your request payloads are correctly formatted and match the API's expected schema. + - **Rate Limit Exceeded Errors:** Be aware of API rate limits and implement appropriate retry mechanisms. + - **Data Type Mismatches:** Ensure that the data types in your request payloads match the API's expected data types. + - **Handling Large Responses:** Insomnia might struggle with extremely large API responses. Consider using a tool like `jq` to process large JSON responses. + - **Import/Export Issues:** Be aware of potential compatibility issues when importing or exporting Insomnia collections. Always test imported collections to ensure that they function correctly. + - **Version Compatibility:** Test Insomnia against the versions of APIs that you intend to target. Old API versions might have compatibility issues. + - **Debugging strategies:** + - **Response Inspection:** Use Insomnia's response inspection tools to examine the API's responses. Check the status code, headers, and body for errors or unexpected data. + - **Request Logging:** Enable request logging to capture detailed information about the API requests being sent. This can help you diagnose issues with the request payload, headers, or authentication. + - **Console Output:** Use the `console.log()` function in Insomnia's request scripts to output debug messages to the console. This can help you track the flow of data and identify potential problems. + - **Network Monitoring:** Use a network monitoring tool (e.g., Wireshark) to capture and analyze the network traffic between Insomnia and the API server. This can help you identify issues with the network connection, SSL/TLS configuration, or API server behavior. + +- **Tooling and Environment:** + - **Recommended Tools:** + - Insomnia Designer: Use Insomnia Designer to visually design and document your APIs. + - Insomnia CLI: Use the Insomnia CLI to automate API testing and integrate it with your CI/CD pipelines. + - JSON Formatter: Use a JSON formatter to validate and format your JSON request payloads. + - YAML Linter: Use a YAML linter to validate your YAML API definitions. + - **Build Configuration:** Store your Insomnia collections and environment files in a Git repository to track changes, collaborate with team members, and implement version control. + - **Linting and Formatting:** Enforce consistent code style and API design by using linting and formatting tools. + - **Deployment:** Store your API keys and other sensitive information as environment variables on your deployment server. Use a secure deployment pipeline to ensure that your Insomnia collections and environment files are deployed correctly. + - **CI/CD Integration:** Use Insomnia CLI to integrate API testing into your CI/CD pipelines. Configure automated API tests to run whenever code is committed or deployed. + +- **Additional Best Practices:** + - **Documentation:** Maintain clear and up-to-date documentation for your API collections and environments. + - **Collaboration:** Use Insomnia's team collaboration features to share API collections and environments with your team members. + - **Regular Updates:** Keep Insomnia up-to-date to take advantage of the latest features and security patches. + - **Modular Design:** Break down large API collections into smaller, more manageable modules to improve maintainability. + +By following these best practices, you can leverage Insomnia to streamline your API development workflow, improve API quality, and ensure the security of your API services. + +- **References** + - @file:./insomnia_example_rule.mdc (Example rule for chaining) \ No newline at end of file diff --git a/.cursor/rules/ionic.mdc b/.cursor/rules/ionic.mdc new file mode 100644 index 000000000..2ebe053f5 --- /dev/null +++ b/.cursor/rules/ionic.mdc @@ -0,0 +1,339 @@ +--- +description: This rule provides comprehensive best practices for Ionic Framework development, covering code organization, performance, security, testing, and more. Following these guidelines will result in more maintainable, performant, and secure Ionic applications. +globs: *.ts,*.html,*.scss +--- +--- +# Ionic Framework Best Practices + +This document outlines best practices for developing Ionic applications. Following these guidelines will help ensure your applications are maintainable, performant, secure, and testable. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure Best Practices + +A well-organized directory structure is crucial for maintainability. Consider this structure as a starting point: + + +src/ + app/ + components/ + my-component/ + my-component.component.ts + my-component.component.html + my-component.component.scss + my-component.component.spec.ts + pages/ + home/ + home.page.ts + home.page.html + home.page.scss + home.page.spec.ts + services/ + data.service.ts + app.component.ts + app.module.ts + app-routing.module.ts + assets/ + theme/ + variables.scss + environments/ + environment.ts + environment.prod.ts + + +* **`src/app`**: Contains the core application code. +* **`src/app/components`**: Reusable UI components. +* **`src/app/pages`**: Individual application pages/views. +* **`src/app/services`**: Application services for data access, business logic, etc. +* **`src/assets`**: Static assets like images, fonts, etc. +* **`src/theme`**: Global styles and variables. +* **`src/environments`**: Environment-specific configuration. + +### 1.2 File Naming Conventions + +Use consistent file naming conventions: + +* Components: `my-component.component.ts`, `my-component.component.html`, `my-component.component.scss` +* Pages: `home.page.ts`, `home.page.html`, `home.page.scss` +* Services: `data.service.ts` +* Modules: `app.module.ts`, `my-module.module.ts` +* Interfaces: `my-interface.ts` +* Enums: `my-enum.ts` + +This consistency improves readability and maintainability. + +### 1.3 Module Organization + +* **Feature Modules:** Group related components, pages, and services into feature modules. This promotes modularity and lazy loading. +* **Shared Module:** Create a shared module for commonly used components, directives, and pipes. +* **Core Module:** A core module is used for application-wide services that you only need to instantiate once. Import it only into the `AppModule`. +* **Lazy Loading:** Lazy load feature modules to improve initial load time. + +typescript +// Example of a Feature Module +import { NgModule } from '@angular/core'; +import { CommonModule } from '@angular/common'; +import { MyComponent } from './my-component/my-component.component'; +import { MyPage } from './my-page/my-page.page'; +import { MyService } from './my-service/my-service.service'; +import { MyRoutingModule } from './my-routing.module'; + +@NgModule({ + declarations: [MyComponent, MyPage], + imports: [CommonModule, MyRoutingModule], + providers: [MyService], +}) +export class MyModule {} + + +### 1.4 Component Architecture + +* **Smart vs. Dumb Components:** + * **Smart Components (Pages):** Handle data fetching, state management, and interactions with services. + * **Dumb Components (Components):** Receive data via inputs and emit events via outputs. Focus on presentation and UI logic. +* **Component Reusability:** Design components to be reusable across different parts of the application. +* **Avoid Direct DOM Manipulation:** Use Angular's data binding and directives instead of directly manipulating the DOM. +* **Follow the Single Responsibility Principle (SRP):** Ensure each component has a specific purpose. + +### 1.5 Code Splitting + +* **Lazy Loading Modules:** As mentioned, lazy load feature modules to reduce the initial bundle size. +* **Route-Based Code Splitting:** Split code based on application routes. +* **Conditional Loading:** Load components or modules only when they are needed. + +## 2. Common Patterns and Anti-Patterns + +### 2.1 Design Patterns + +* **Model-View-Controller (MVC):** Ionic, built on Angular, largely follows the MVC pattern. +* **Singleton:** Use singletons for services that need to maintain global state (e.g., authentication service). +* **Observer:** Use RxJS Observables for asynchronous operations and data streams. +* **Facade:** Provide a simplified interface to a complex subsystem. +* **Dependency Injection:** Use Angular's dependency injection system to provide components with their dependencies. + +### 2.2 Recommended Approaches for Common Tasks + +* **Data Fetching:** Use Angular's `HttpClient` with RxJS Observables for making API requests. +* **Form Handling:** Use Angular's reactive forms or template-driven forms for managing form data. +* **Navigation:** Use the Ionic `NavController` for navigating between pages. +* **State Management:** Use a state management library like NgRx or Akita for complex applications. +* **UI Components:** Leverage Ionic's built-in UI components for a consistent look and feel. + +### 2.3 Anti-Patterns and Code Smells + +* **Massive Components:** Avoid creating components that are too large or complex. Break them down into smaller, reusable components. +* **Direct DOM Manipulation:** As mentioned, avoid directly manipulating the DOM. +* **Nested Subscriptions:** Avoid deeply nested RxJS subscriptions, which can lead to memory leaks and difficult-to-debug code. Use operators like `switchMap`, `mergeMap`, or `concatMap` to flatten subscriptions. +* **Overusing `any` Type:** Avoid using the `any` type excessively. Use specific types to improve type safety and code maintainability. +* **Ignoring Errors:** Always handle errors properly to prevent unexpected behavior. +* **Global Variables:** Avoid using global variables, which can lead to naming conflicts and unpredictable behavior. + +### 2.4 State Management + +* **Component State:** For simple component-specific state, use Angular's `@Input` and `@Output` decorators. +* **Service State:** For application-wide state that needs to be shared between components, use a service with RxJS `BehaviorSubject` or `ReplaySubject`. +* **NgRx/Akita:** For complex applications with significant state management needs, consider using a state management library like NgRx or Akita. + * NgRx leverages Redux principles, offering a predictable state container with actions, reducers, and effects. This is suitable for large-scale applications needing centralized state management, change tracking, and debugging tools. However, NgRx can introduce boilerplate and complexity. + * Akita, a state management pattern on top of RxJS, simplifies the process with less boilerplate and easier learning curve compared to Redux-based solutions. It’s suitable for small to medium-sized applications requiring scalable yet simple state management. + +### 2.5 Error Handling + +* **Centralized Error Handling:** Create a centralized error handling service to handle errors consistently across the application. +* **Error Interceptors:** Use Angular's `HttpInterceptor` to intercept HTTP requests and handle errors globally. +* **User-Friendly Error Messages:** Display user-friendly error messages to the user. +* **Logging:** Log errors to a server for debugging and monitoring purposes. +* **Retry Mechanism:** Implement a retry mechanism for transient errors. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +* **Lazy Loading:** As previously emphasized, use lazy loading for modules and components. +* **Ahead-of-Time (AOT) Compilation:** Use AOT compilation to compile the application during the build process, which improves startup time. +* **Change Detection Optimization:** + * **`OnPush` Change Detection:** Use `OnPush` change detection strategy for components that only depend on input properties. This can significantly improve performance. + * **`trackBy` Function:** Use the `trackBy` function in `*ngFor` loops to improve rendering performance when the underlying data changes. +* **Virtual Scrolling:** Use virtual scrolling for long lists to render only the visible items. +* **Image Optimization:** Optimize images by compressing them and using appropriate formats. +* **Minify and Bundle:** Minify and bundle JavaScript and CSS files to reduce file sizes. +* **Caching:** Implement caching strategies for frequently accessed data. +* **Debouncing and Throttling:** Use debouncing and throttling to limit the frequency of event handlers. + +### 3.2 Memory Management + +* **Unsubscribe from Observables:** Always unsubscribe from RxJS subscriptions to prevent memory leaks. Use the `takeUntil` operator or a `Subject` to manage subscriptions. +* **Avoid Circular Dependencies:** Avoid circular dependencies between modules, which can lead to memory leaks. +* **Release Resources:** Release resources when they are no longer needed. +* **Use `async` Pipe:** Use the `async` pipe in templates to automatically unsubscribe from Observables. + +### 3.3 Rendering Optimization + +* **Reduce DOM Updates:** Minimize the number of DOM updates by using Angular's data binding and change detection mechanisms effectively. +* **Use CSS Transforms:** Use CSS transforms instead of directly manipulating the DOM for animations and transitions. +* **Avoid Complex Expressions in Templates:** Avoid complex expressions in templates, which can impact rendering performance. Move complex logic to component classes. +* **Virtual DOM:** Angular uses a virtual DOM, but understanding how change detection works is critical to performance. + +### 3.4 Bundle Size Optimization + +* **Tree Shaking:** Use tree shaking to remove unused code from the bundle. +* **Code Splitting:** As mentioned, use code splitting to reduce the initial bundle size. +* **Remove Unused Dependencies:** Remove any unused dependencies from the `package.json` file. +* **Use Production Build:** Use the production build flag (`--prod`) when building the application for deployment. +* **Analyze Bundle Size:** Use tools like `webpack-bundle-analyzer` to analyze the bundle size and identify areas for optimization. + +### 3.5 Lazy Loading + +* **Lazy Load Modules:** This is a primary optimization strategy in Ionic. +* **Lazy Load Images:** Use libraries that support lazy loading of images. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and Prevention + +* **Cross-Site Scripting (XSS):** Prevent XSS attacks by sanitizing user input and avoiding the use of `innerHTML`. Use Angular's built-in sanitization features. +* **Cross-Site Request Forgery (CSRF):** Implement CSRF protection by using tokens in HTTP requests. +* **Injection Attacks:** Prevent SQL injection and other injection attacks by using parameterized queries and input validation. +* **Insecure Data Storage:** Avoid storing sensitive data in local storage or cookies. Use secure storage mechanisms like the Ionic Native Storage plugin or a secure backend. +* **Man-in-the-Middle (MITM) Attacks:** Use HTTPS to encrypt communication between the app and the server. +* **Certificate Pinning:** Implement certificate pinning to prevent MITM attacks. + +### 4.2 Input Validation + +* **Client-Side Validation:** Implement client-side validation to provide immediate feedback to the user. +* **Server-Side Validation:** Always perform server-side validation to ensure data integrity. +* **Sanitize User Input:** Sanitize user input to prevent XSS attacks. +* **Use Regular Expressions:** Use regular expressions to validate input formats. + +### 4.3 Authentication and Authorization + +* **Use Secure Authentication:** Use a secure authentication mechanism like OAuth 2.0 or JWT (JSON Web Token). +* **Implement Role-Based Access Control (RBAC):** Implement RBAC to control access to different parts of the application based on user roles. +* **Store Tokens Securely:** Store authentication tokens securely using the Ionic Native Storage plugin or a secure backend. +* **Refresh Tokens:** Use refresh tokens to automatically renew authentication tokens. +* **Implement Multi-Factor Authentication (MFA):** Implement MFA for increased security. + +### 4.4 Data Protection + +* **Encrypt Sensitive Data:** Encrypt sensitive data at rest and in transit. +* **Use HTTPS:** Use HTTPS for all communication between the app and the server. +* **Secure API Keys:** Protect API keys by storing them securely and using them only when necessary. +* **Data Masking:** Mask sensitive data in the UI. +* **Regular Security Audits:** Conduct regular security audits to identify and address potential vulnerabilities. + +### 4.5 Secure API Communication + +* **Use HTTPS:** Use HTTPS for all API communication. +* **Validate API Responses:** Validate API responses to ensure data integrity. +* **Implement Rate Limiting:** Implement rate limiting to prevent abuse of API endpoints. +* **Use API Gateways:** Use API gateways to manage and secure API traffic. +* **Secure API Keys:** Protect API keys by storing them securely and using them only when necessary. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +* **Test Components in Isolation:** Test components in isolation using mocking and stubbing techniques. +* **Test Services in Isolation:** Test services in isolation by mocking their dependencies. +* **Use Test Doubles:** Use test doubles like stubs, mocks, and spies to isolate the code under test. +* **Test Boundary Conditions:** Test boundary conditions to ensure the code handles edge cases correctly. +* **Use Code Coverage Tools:** Use code coverage tools to measure the percentage of code that is covered by tests. + +### 5.2 Integration Testing + +* **Test Component Interactions:** Test the interactions between components. +* **Test Service Interactions:** Test the interactions between services. +* **Test Data Flow:** Test the flow of data through the application. +* **Use Mock HTTP Backend:** Use a mock HTTP backend to simulate API responses. + +### 5.3 End-to-End Testing + +* **Test User Flows:** Test the complete user flows through the application. +* **Use a Testing Framework:** Use a testing framework like Cypress or Protractor for end-to-end testing. +* **Test on Real Devices:** Test the application on real devices to ensure it works correctly in different environments. + +### 5.4 Test Organization + +* **Locate Tests with Source Files:** Keep test files in the same directory as the source files they test (e.g., `my-component.component.spec.ts` next to `my-component.component.ts`). +* **Use Descriptive Test Names:** Use descriptive test names to clearly indicate what the test is verifying. +* **Organize Tests by Feature:** Organize tests by feature to improve test maintainability. + +### 5.5 Mocking and Stubbing + +* **Use Mocking Libraries:** Use mocking libraries like Jasmine or Sinon.js to create mocks and stubs. +* **Mock Dependencies:** Mock dependencies to isolate the code under test. +* **Stub API Responses:** Stub API responses to simulate different scenarios. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +* **Not Unsubscribing from Observables:** Forgetting to unsubscribe from RxJS subscriptions. +* **Directly Manipulating the DOM:** Directly manipulating the DOM instead of using Angular's data binding. +* **Overusing `any` Type:** Overusing the `any` type. +* **Ignoring Errors:** Ignoring errors. +* **Not Validating User Input:** Not validating user input. +* **Storing Sensitive Data in Local Storage:** Storing sensitive data in local storage. +* **Not Using HTTPS:** Not using HTTPS for API communication. + +### 6.2 Edge Cases + +* **Handling Network Errors:** Handling network errors gracefully. +* **Handling Different Screen Sizes:** Handling different screen sizes and resolutions. +* **Handling Different Operating Systems:** Handling different operating systems (iOS and Android). +* **Handling Different Device Capabilities:** Handling different device capabilities (e.g., camera, GPS). + +### 6.3 Version-Specific Issues + +* **Breaking Changes:** Be aware of breaking changes in new versions of Ionic and Angular. Carefully review release notes. +* **Deprecated Features:** Avoid using deprecated features. +* **Compatibility Issues:** Test the application with different versions of Ionic and Angular to ensure compatibility. + +### 6.4 Compatibility Concerns + +* **Plugin Compatibility:** Ensure that plugins are compatible with the target platform and version of Ionic. +* **Browser Compatibility:** Ensure that the application works correctly in different browsers. +* **Cordova vs. Capacitor:** Understand the differences and implications of using Cordova versus Capacitor. Capacitor is generally recommended for new projects. + +### 6.5 Debugging Strategies + +* **Use Browser Developer Tools:** Use browser developer tools to debug JavaScript, CSS, and network requests. +* **Use Debugging Tools:** Use debugging tools like VS Code's debugger to step through code and inspect variables. +* **Use Logging:** Use logging to track the flow of execution and identify potential issues. +* **Use Remote Debugging:** Use remote debugging to debug the application on real devices. +* **Learn to read Stack Traces:** Understanding the stack trace is crucial for identifying the source of errors. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +* **VS Code:** VS Code is a popular IDE for Ionic development with excellent support for TypeScript, HTML, and CSS. +* **Ionic CLI:** The Ionic CLI is a command-line tool for creating, building, and deploying Ionic applications. +* **Chrome DevTools:** Chrome DevTools is a powerful tool for debugging and profiling Ionic applications. +* **Postman/Insomnia:** Use Postman or Insomnia to test API endpoints. + +### 7.2 Build Configuration + +* **Use Environment Variables:** Use environment variables to configure the application for different environments (e.g., development, production). +* **Configure Build Scripts:** Configure build scripts to automate build tasks. +* **Use a Build System:** Use a build system like Webpack or Parcel to manage dependencies and bundle assets. +* **Optimize Build Configuration:** Optimize the build configuration to reduce build time and bundle size. + +### 7.3 Linting and Formatting + +* **Use ESLint:** Use ESLint to enforce coding standards and identify potential issues. +* **Use Prettier:** Use Prettier to automatically format code. +* **Configure Editor Integration:** Configure editor integration to automatically lint and format code on save. + +### 7.4 Deployment + +* **Build for Production:** Build the application for production using the `--prod` flag. +* **Use a Deployment Platform:** Use a deployment platform like Firebase Hosting, Netlify, or AWS Amplify. +* **Configure HTTPS:** Configure HTTPS for the deployment environment. +* **Monitor Application Performance:** Monitor application performance after deployment. + +### 7.5 CI/CD + +* **Use a CI/CD Pipeline:** Use a CI/CD pipeline to automate the build, test, and deployment process. +* **Use a CI/CD Tool:** Use a CI/CD tool like Jenkins, CircleCI, or Travis CI. +* **Automate Testing:** Automate unit, integration, and end-to-end tests in the CI/CD pipeline. +* **Automate Deployment:** Automate deployment to different environments in the CI/CD pipeline. \ No newline at end of file diff --git a/.cursor/rules/isort.mdc b/.cursor/rules/isort.mdc new file mode 100644 index 000000000..2d2c54a30 --- /dev/null +++ b/.cursor/rules/isort.mdc @@ -0,0 +1,223 @@ +--- +description: This rule provides comprehensive guidelines for using isort in Python projects, covering code organization, common patterns, performance, security, testing, tooling, and common pitfalls. It aims to standardize import sorting and improve code quality. +globs: *.py +--- +# isort Best Practices: A Comprehensive Guide + +This document provides a comprehensive guide to using isort effectively in Python projects. It covers various aspects, including code organization, common patterns, performance considerations, security, testing, tooling, and common pitfalls. + +## Library Information: +- Name: isort +- Tags: development, python, formatter, imports + +## 1. Code Organization and Structure + +### 1.1 Directory Structure Best Practices + +While isort primarily focuses on import sorting within files, the overall directory structure impacts how imports are organized and how isort rules are applied. + +- **Flat Structure (Small Projects):** For smaller projects, a flat structure might suffice, with all modules in a single directory. +- **Modular Structure (Larger Projects):** For larger projects, adopt a modular structure using packages and subpackages. This enhances maintainability and reusability. + + + project_name/ + ├── package1/ + │ ├── __init__.py + │ ├── module1.py + │ └── module2.py + ├── package2/ + │ ├── __init__.py + │ └── module3.py + ├── main.py + └── pyproject.toml (isort configuration) + + +### 1.2 File Naming Conventions Specific to isort + +- isort doesn't enforce specific file naming conventions but follows PEP 8 recommendations. +- Use descriptive and consistent file names. For example, `utils.py`, `models.py`, `services.py`. +- Lowercase with underscores for module names: `my_module.py` + +### 1.3 Module Organization Best Practices + +- **Grouping Related Functionality:** Organize modules based on related functionality. For example, place database-related functions in a `db` module. +- **Avoiding Circular Imports:** Be cautious of circular import dependencies, where two or more modules import each other, leading to potential runtime errors. isort can help detect these through proper import ordering. +- **Using Relative Imports:** Utilize relative imports within packages for internal references. + python + # Within package1/module1.py + from .module2 import some_function # Relative import + + # From outside the package + from package1.module1 import some_function + + +### 1.4 Component Architecture Recommendations + +- **Layered Architecture:** Consider a layered architecture (e.g., presentation, business logic, data access) to separate concerns and facilitate modularity. +- **Microservices:** In larger applications, microservices can be used to break down the application into smaller, independent services. Each service can have its own isort configuration. + +### 1.5 Code Splitting Strategies + +- **Splitting Large Modules:** Decompose large modules into smaller, more manageable modules to improve readability and maintainability. Each module will have its own independent import sorting managed by isort +- **Lazy Loading:** Implement lazy loading for modules that are not immediately required to improve startup time. This may require dynamic imports, which isort can handle. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns Specific to isort + +isort itself doesn't directly relate to traditional design patterns like Singleton or Factory. However, its proper usage facilitates cleaner code, which makes it easier to apply other design patterns. + +- **Import Facade:** Use a module to consolidate and re-export imports from other modules to simplify external dependencies. + + python + # my_package/imports.py + from .module1 import ClassA + from .module2 import function_b + + __all__ = ['ClassA', 'function_b'] + + +### 2.2 Recommended Approaches for Common Tasks + +- **Sorting Imports:** Use isort to automatically sort imports alphabetically and group them by type (standard library, third-party, local). +- **Integrating with Black:** Configure isort to be compatible with Black to enforce consistent code formatting. +- **Using `pyproject.toml`:** Define isort's configuration in the `pyproject.toml` file for project-specific settings. + +### 2.3 Anti-patterns and Code Smells + +- **Ignoring isort:** Not using isort consistently across the project leads to inconsistent import styles and reduced readability. +- **Manual Sorting:** Manually sorting imports is error-prone and time-consuming. Use isort to automate the process. +- **Conflicting Configurations:** Having multiple, conflicting isort configurations in different parts of the project can cause inconsistencies. +- **Overly Complex Imports:** Avoid deep nesting or overly complex import statements, which reduce readability. + +### 2.4 State Management + +- isort is not directly involved with State Management. + +### 2.5 Error Handling + +- isort generally handles errors gracefully, such as when encountering syntax errors in files. +- Ensure that isort configuration is valid to avoid unexpected behavior. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +- **Caching:** isort utilizes caching to improve performance on subsequent runs. Ensure the cache directory is properly configured. +- **Parallel Processing:** isort supports parallel processing for faster sorting of multiple files. +- **Minimizing File Size:** Sort imports to avoid any unnecessary circular dependency issues + +### 3.2 Memory Management + +- isort is generally memory-efficient. However, processing extremely large files may require additional memory. + +### 3.3 Rendering Optimization + +- isort doesn't involve rendering. + +### 3.4 Bundle Size Optimization + +- isort doesn't directly impact bundle size, as it's a development tool. + +### 3.5 Lazy Loading + +- isort works with modules using lazy loading; it sorts the import statements as they are written. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities + +- isort itself doesn't introduce security vulnerabilities, but improper coding practices can. + +### 4.2 Input Validation + +- isort doesn't handle external input; hence, input validation is not directly relevant. + +### 4.3 Authentication and Authorization + +- Authentication/Authorization not applicable to isort + +### 4.4 Data Protection + +- isort doesn't manage data. + +### 4.5 Secure API Communication + +- isort doesn't involve API communication. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +- Although isort is primarily a formatting tool, consider writing unit tests to ensure its configuration produces the desired output. +- Test cases can include verifying that imports are sorted correctly under various scenarios. + +### 5.2 Integration Testing + +- Integrate isort with other tools like Black and Flake8 in your CI/CD pipeline. Integration tests can verify that these tools work together seamlessly. + +### 5.3 End-to-End Testing + +- isort doesn't directly require end-to-end testing. + +### 5.4 Test Organization + +- Keep test files separate from source code, usually in a dedicated `tests/` directory. + +### 5.5 Mocking and Stubbing + +- Not applicable for isort testing. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +- **Incorrect Configuration:** Misconfiguring the `pyproject.toml` file, leading to unexpected sorting behavior. +- **Ignoring Warnings:** Ignoring warnings from isort can result in subtle bugs and inconsistent code. +- **Not Integrating with Pre-commit:** Failing to integrate isort with pre-commit hooks allows unformatted code to be committed. + +### 6.2 Edge Cases + +- **Complex Import Paths:** isort might have trouble with extremely complex or dynamically generated import paths. +- **Conditional Imports:** Be careful with conditional imports (imports within `if` statements), as isort might not always handle them correctly. + +### 6.3 Version-Specific Issues + +- Check for compatibility issues between isort versions and other tools or libraries. + +### 6.4 Compatibility Concerns + +- Ensure isort is compatible with other linters (e.g., Flake8) and formatters (e.g., Black) used in the project. + +### 6.5 Debugging Strategies + +- **Verbose Mode:** Use isort's verbose mode to get more detailed output about its actions. +- **Configuration Testing:** Create small test files to experiment with isort configurations. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +- **VS Code:** Use VS Code with the Python extension for seamless integration with isort, Black, and Flake8. +- **PyCharm:** PyCharm provides excellent support for Python development, including built-in linting and formatting tools. + +### 7.2 Build Configuration + +- Use `pyproject.toml` to manage isort's configuration. Store configuration in version control +- Use pre-commit hooks to ensure isort is run before each commit. + +### 7.3 Linting and Formatting + +- Integrate isort, Black, and Flake8 for comprehensive code linting and formatting. +- Configure these tools to follow PEP 8 guidelines. + +### 7.4 Deployment + +- isort is not a deployment tool. Ensure that your deployment process includes running linters and formatters. + +### 7.5 CI/CD Integration + +- Integrate isort into your CI/CD pipeline to automatically check code quality on each commit or pull request. +- Use tools like GitHub Actions or GitLab CI to automate the process. + +By following these best practices, you can effectively use isort to maintain consistent and high-quality code in your Python projects. \ No newline at end of file diff --git a/.cursor/rules/java.mdc b/.cursor/rules/java.mdc new file mode 100644 index 000000000..3794a21a4 --- /dev/null +++ b/.cursor/rules/java.mdc @@ -0,0 +1,296 @@ +--- +description: Enforces best practices for Java development, covering code style, performance, security, and testing. Provides guidelines for writing clean, maintainable, and efficient Java code. +globs: *.java +--- +- # Java Best Practices + + This document outlines comprehensive best practices for Java development, covering code organization, common patterns, performance considerations, security, testing, common pitfalls, and tooling. Adhering to these guidelines will help you write clean, maintainable, efficient, and secure Java code. + +- ## 1. Code Organization and Structure + + - ### 1.1 Directory Structure + + - **Maven Standard Layout:** Use the standard Maven directory structure for most projects. This provides a consistent and predictable layout that's easily understood by other developers and tools. + + src/ + main/ + java/ + com/example/ <- Your package structure starts here + resources/ + test/ + java/ + com/example/ + resources/ + pom.xml <- Maven project file + + - **Gradle Layout:** Gradle supports the Maven layout and provides other ways to configure the source directories. Choose a layout that best fits your project's needs. + + - **Package by Feature:** Organize packages by feature rather than by layer (e.g., controllers, services, repositories). This improves cohesion and reduces dependencies between features. + + src/ + main/ + java/ + com/example/ + user/ + UserController.java + UserService.java + UserRepository.java + product/ + ProductController.java + ProductService.java + ProductRepository.java + + - **Modularization:** For large projects, consider using Java modules (Jigsaw, introduced in Java 9) to improve encapsulation and reduce dependencies. + + - ### 1.2 File Naming Conventions + + - **Classes and Interfaces:** Use `PascalCase` (e.g., `UserController`, `UserService`). + - **Methods and Variables:** Use `camelCase` (e.g., `getUserById`, `userName`). + - **Constants:** Use `UPPER_SNAKE_CASE` (e.g., `MAX_RETRIES`, `DEFAULT_TIMEOUT`). + - **Packages:** Use all lowercase (e.g., `com.example.user`). + - **Avoid abbreviations:** Use meaningful and descriptive names. + + - ### 1.3 Module Organization + + - **`module-info.java`:** Use `module-info.java` to define module dependencies and exported packages. This allows for strong encapsulation and controlled access to internal APIs. + - **Explicit Dependencies:** Declare all module dependencies explicitly in `module-info.java`. Avoid relying on transitive dependencies. + - **Minimize Exports:** Only export the packages that are intended for public use. Keep internal packages hidden from other modules. + + - ### 1.4 Component Architecture + + - **Dependency Injection:** Use dependency injection (DI) to manage component dependencies. Frameworks like Spring and Guice simplify DI. + - **Inversion of Control (IoC):** Apply IoC to decouple components and improve testability. + - **Layered Architecture:** Structure your application into layers (e.g., presentation, business logic, data access). This promotes separation of concerns and maintainability. + - **Microservices:** For large, complex applications, consider a microservices architecture. This allows for independent development, deployment, and scaling of individual services. + + - ### 1.5 Code Splitting + + - **Feature Toggles:** Use feature toggles to enable or disable features at runtime. This allows for incremental deployment and testing of new features. + - **Dynamic Loading:** Use dynamic class loading to load modules or components on demand. This can reduce the initial startup time and memory footprint of your application. + - **Conditional Compilation:** Use conditional compilation (e.g., with Maven profiles) to include or exclude code based on the environment. This allows for different configurations for development, testing, and production. + +- ## 2. Common Patterns and Anti-patterns + + - ### 2.1 Design Patterns + + - **Singleton:** Use the Singleton pattern sparingly and only when a single instance of a class is truly required. Consider dependency injection as an alternative. + - **Factory:** Use the Factory pattern to create objects without specifying their concrete classes. This promotes loose coupling and allows for easy substitution of different implementations. + - **Strategy:** Use the Strategy pattern to encapsulate different algorithms or behaviors. This allows you to switch between algorithms at runtime. + - **Observer:** Use the Observer pattern to define a one-to-many dependency between objects. This allows for loose coupling and easy addition of new observers. + - **Template Method:** Use the Template Method pattern to define the skeleton of an algorithm in a base class, allowing subclasses to override specific steps without changing the overall structure. + - **Builder:** Use the Builder pattern to construct complex objects with many optional parameters. This improves readability and reduces the risk of errors. + + - ### 2.2 Recommended Approaches + + - **Resource Management:** Always use try-with-resources to ensure proper resource management (e.g., closing streams, connections). This prevents resource leaks. + java + try (FileInputStream fis = new FileInputStream("file.txt")) { + // Use the file input stream + } + + - **String Concatenation:** Use `StringBuilder` or `StringBuffer` for string concatenation, especially in loops. Avoid using the `+` operator for repeated string concatenation. + java + StringBuilder sb = new StringBuilder(); + for (int i = 0; i < 100; i++) { + sb.append(i); + } + String result = sb.toString(); + + - **Collections:** Prefer Java Collections over arrays for their flexibility and utility. Use generics to ensure type safety. + + - ### 2.3 Anti-patterns and Code Smells + + - **God Class:** Avoid creating large classes that do too much. Break down large classes into smaller, more manageable components. + - **Long Method:** Avoid creating long methods. Break down long methods into smaller, more focused methods. + - **Shotgun Surgery:** Avoid making many small changes in multiple classes. This indicates a lack of cohesion and can make it difficult to maintain the code. + - **Data Clumps:** Avoid passing the same group of data items together in multiple methods. Create a class to encapsulate the data items. + - **Primitive Obsession:** Avoid using primitive types excessively. Create value objects to represent domain concepts. + - **Switch Statements:** Limit use of switch statements especially with larger number of cases. Consider using polymorphism with Strategy pattern. + - **Empty Catch Blocks:** Avoid empty catch blocks. Always handle exceptions appropriately, either by logging them, rethrowing them, or taking corrective action. + + - ### 2.4 State Management + + - **Immutability:** Prefer immutable objects whenever possible. Immutable objects are thread-safe and easier to reason about. + - **Stateless Services:** Design services to be stateless. This improves scalability and simplifies testing. + - **Session Management:** Use session management frameworks (e.g., Spring Session) to manage user sessions in web applications. + + - ### 2.5 Error Handling + + - **Exceptions for Exceptional Cases:** Use exceptions only for exceptional cases, not for normal control flow. + - **Specific Exception Types:** Catch specific exception types rather than generic `Exception`. This allows you to handle different types of errors differently. + - **Logging:** Log exceptions with sufficient context to aid debugging. Include the stack trace and any relevant data. + - **Custom Exceptions:** Create custom exception types to represent application-specific errors. + - **Don't Swallow Exceptions:** Never swallow exceptions without logging or handling them. It hides the exception making debugging much harder. + +- ## 3. Performance Considerations + + - ### 3.1 Optimization Techniques + + - **Caching:** Use caching to store frequently accessed data in memory. Frameworks like Caffeine and Guava Cache provide efficient caching implementations. + - **Connection Pooling:** Use connection pooling to reuse database connections. This reduces the overhead of creating and closing connections. + - **Efficient Algorithms:** Choose appropriate algorithms for specific tasks. Consider the time and space complexity of different algorithms. + - **Lazy Initialization:** Use lazy initialization to defer the creation of objects until they are actually needed. + - **Minimize Object Creation:** Reduce unnecessary object creation. Use object pooling or reuse existing objects whenever possible. + + - ### 3.2 Memory Management + + - **Garbage Collection:** Understand how Java's garbage collector works. Avoid creating objects that are quickly discarded, as this puts pressure on the garbage collector. + - **Memory Profiling:** Use memory profiling tools to identify memory leaks and optimize memory usage. + - **Large Objects:** Be careful when handling large objects. They can cause fragmentation and increase garbage collection times. + + - ### 3.3 Rendering Optimization (If Applicable) + + - **Buffering:** Use buffering to reduce the number of I/O operations when rendering large amounts of data. + - **Compression:** Use compression to reduce the size of rendered data. + + - ### 3.4 Bundle Size Optimization (If Applicable) + + - **Code Minification:** Use code minification to reduce the size of your codebase. + - **Dead Code Elimination:** Remove unused code to reduce the bundle size. + + - ### 3.5 Lazy Loading + + - **On-Demand Loading:** Load resources or components only when they are needed. + - **Virtual Proxy:** Use a virtual proxy to delay the loading of a heavy resource until it is accessed. + +- ## 4. Security Best Practices + + - ### 4.1 Common Vulnerabilities + + - **SQL Injection:** Prevent SQL injection by using parameterized queries or prepared statements. Never concatenate user input directly into SQL queries. + - **Cross-Site Scripting (XSS):** Prevent XSS by encoding user input before displaying it in web pages. + - **Cross-Site Request Forgery (CSRF):** Prevent CSRF by using anti-CSRF tokens. + - **Authentication and Authorization Issues:** Implement proper authentication and authorization mechanisms to protect sensitive resources. + - **Denial of Service (DoS):** Protect against DoS attacks by limiting request rates and implementing rate limiting. + - **Insecure Deserialization:** Prevent insecure deserialization by avoiding deserialization of untrusted data, or using secure deserialization methods. + - **Dependency Vulnerabilities:** Use tools like OWASP Dependency-Check to identify and mitigate vulnerabilities in third-party libraries. + + - ### 4.2 Input Validation + + - **Whitelisting:** Use whitelisting to validate input against a list of allowed values. Avoid blacklisting, as it is difficult to anticipate all possible malicious inputs. + - **Regular Expressions:** Use regular expressions to validate input patterns (e.g., email addresses, phone numbers). + - **Length Limits:** Enforce length limits on input fields to prevent buffer overflows. + - **Encoding:** Encode user input to prevent XSS attacks. + + - ### 4.3 Authentication and Authorization + + - **Strong Passwords:** Enforce strong password policies (e.g., minimum length, complexity). + - **Hashing:** Hash passwords using a strong hashing algorithm (e.g., bcrypt, Argon2) and a salt. + - **Two-Factor Authentication (2FA):** Implement 2FA to provide an extra layer of security. + - **Role-Based Access Control (RBAC):** Use RBAC to control access to resources based on user roles. + - **OAuth 2.0:** Use OAuth 2.0 for delegated authorization. + + - ### 4.4 Data Protection + + - **Encryption:** Encrypt sensitive data at rest and in transit. + - **Data Masking:** Mask sensitive data in logs and error messages. + - **Access Control:** Restrict access to sensitive data to authorized users only. + + - ### 4.5 Secure API Communication + + - **HTTPS:** Use HTTPS for all API communication. + - **TLS/SSL:** Configure TLS/SSL properly to ensure secure communication. + - **API Keys:** Use API keys to authenticate API clients. + - **Rate Limiting:** Implement rate limiting to prevent abuse of your APIs. + - **Input Validation:** Validate all input to your APIs to prevent injection attacks. + +- ## 5. Testing Approaches + + - ### 5.1 Unit Testing + + - **JUnit:** Use JUnit for unit testing. + - **Mockito:** Use Mockito for mocking dependencies. + - **Arrange-Act-Assert:** Follow the Arrange-Act-Assert pattern in your unit tests. + - **Test Coverage:** Aim for high test coverage. + - **Independent Tests:** Write independent tests such that failure of one test doesn't affect another test. + + - ### 5.2 Integration Testing + + - **Testcontainers:** Use Testcontainers to create lightweight, disposable instances of databases and other services for integration testing. + - **Spring Boot Test:** Use Spring Boot's testing support for integration testing Spring applications. + + - ### 5.3 End-to-End Testing + + - **Selenium:** Use Selenium for end-to-end testing of web applications. + - **Cypress:** Consider Cypress as an alternative to Selenium for end-to-end tests. + + - ### 5.4 Test Organization + + - **Test Directory:** Place your tests in a separate `test` directory. + - **Naming Conventions:** Use clear naming conventions for your tests (e.g., `UserServiceTest`). + - **Test Suites:** Group related tests into test suites. + + - ### 5.5 Mocking and Stubbing + + - **Mockito:** Use Mockito to create mocks and stubs for your tests. + - **Verify Interactions:** Verify that your code interacts with dependencies as expected. + - **Avoid Over-Mocking:** Avoid mocking too many dependencies. Focus on mocking the dependencies that are critical to the test. + +- ## 6. Common Pitfalls and Gotchas + + - ### 6.1 Frequent Mistakes + + - **NullPointerExceptions:** Handle null values carefully to avoid `NullPointerException`. + - **Resource Leaks:** Ensure that all resources are properly closed to avoid resource leaks. + - **Thread Safety Issues:** Be aware of thread safety issues when writing multithreaded code. + - **Ignoring Exceptions:** Never ignore exceptions without logging or handling them. It hides the exception making debugging much harder. + + - ### 6.2 Edge Cases + + - **Boundary Conditions:** Test boundary conditions to ensure that your code handles edge cases correctly. + - **Empty Collections:** Handle empty collections gracefully. + - **Invalid Input:** Validate input to ensure that it is within the expected range. + + - ### 6.3 Version-Specific Issues + + - **Deprecated APIs:** Be aware of deprecated APIs and avoid using them. + - **Compatibility Issues:** Test your code with different versions of Java to ensure compatibility. + + - ### 6.4 Compatibility Concerns + + - **JVM Compatibility:** Ensure that your code is compatible with different JVM implementations. + - **Library Compatibility:** Be aware of compatibility issues between different libraries. + + - ### 6.5 Debugging Strategies + + - **Logging:** Use logging to track the execution of your code and identify errors. + - **Debugging Tools:** Use debugging tools to step through your code and inspect variables. + - **Remote Debugging:** Use remote debugging to debug applications running on remote servers. + - **JVM Profilers:** Use profilers like JProfiler or VisualVM to identify performance bottlenecks and memory leaks. + +- ## 7. Tooling and Environment + + - ### 7.1 Recommended Tools + + - **IntelliJ IDEA:** A powerful IDE for Java development with excellent code completion, refactoring, and debugging support. + - **Eclipse:** Another popular IDE for Java development. + - **Maven:** A build automation tool for managing dependencies, building, and deploying Java projects. + - **Gradle:** A build automation tool that provides more flexibility and control than Maven. + + - ### 7.2 Build Configuration + + - **Dependency Management:** Use Maven or Gradle to manage dependencies. + - **Version Control:** Use version control (e.g., Git) to track changes to your codebase. + - **Build Profiles:** Use build profiles to configure different builds for different environments. + + - ### 7.3 Linting and Formatting + + - **Checkstyle:** Use Checkstyle to enforce coding standards. + - **PMD:** Use PMD to find potential bugs and code smells. + - **SpotBugs:** Use SpotBugs to find potential bugs. + - **Formatter:** Use automatic code formatting tools like IntelliJ's built-in formatter or plugins like `google-java-format` for consistency. + + - ### 7.4 Deployment + + - **Docker:** Use Docker to containerize your applications. + - **Kubernetes:** Use Kubernetes to orchestrate your containers. + - **Cloud Platforms:** Deploy your applications to cloud platforms like AWS, Azure, or Google Cloud. + + - ### 7.5 CI/CD + + - **Jenkins:** Use Jenkins for continuous integration and continuous delivery. + - **GitHub Actions:** Use GitHub Actions for CI/CD. + - **GitLab CI:** Use GitLab CI for CI/CD. + - **Automated Testing:** Automate your unit, integration, and end-to-end tests in your CI/CD pipeline. + +By adhering to these best practices, you can improve the quality, maintainability, and performance of your Java code. Remember to adapt these guidelines to your specific project requirements and team preferences. \ No newline at end of file diff --git a/.cursor/rules/jax.mdc b/.cursor/rules/jax.mdc new file mode 100644 index 000000000..d3ca148ab --- /dev/null +++ b/.cursor/rules/jax.mdc @@ -0,0 +1,267 @@ +--- +description: This rule provides best practices and coding standards for the JAX library, emphasizing functional programming, JIT compilation, automatic differentiation, and immutable data structures. It also covers performance considerations, common pitfalls, and tooling recommendations. +globs: *.py +--- +- **Functional Programming**: JAX emphasizes a functional programming style. Ensure functions are pure (no side effects, no reliance on global variables) for consistent JIT compilation and optimization. + +- **JIT Compilation**: Use `@jax.jit` to decorate functions for performance optimization, especially those called multiple times with the same input shapes. Understand the implications of tracing and static arguments. + +- **Automatic Differentiation**: Leverage `jax.grad()` for efficient gradient calculations and higher-order derivatives. Be mindful of how JAX handles gradients with control flow and other transformations. + +- **Vectorization with `vmap`**: Utilize `jax.vmap()` for automatic vectorization instead of explicit loops. This improves performance by mapping operations across array dimensions. + +- **Immutable Data Structures**: JAX arrays are immutable. Use `.at[].set()` for updates. Understand that these operations are out-of-place, returning a new array. In-place updates *may* occur under jit-compilation if the original value isn't reused, but rely on the functional style. + +- **Random Number Generation**: Use `jax.random` for random number generation. Employ explicit PRNG state management. Always split the key using `jax.random.split()` before generating random numbers to avoid reusing the same state. + +- **Control Flow**: Be aware of limitations when using Python control flow with `jax.jit`. Use `static_argnums` to specify arguments for tracing on concrete values if necessary. Consider structured control flow primitives like `lax.cond`, `lax.while_loop`, `lax.fori_loop`, and `lax.scan` for better traceability and avoiding large loop unrolling. Understand which control flow constructs are differentiable. + +- **Dynamic Shapes**: Avoid dynamic shapes within JAX transformations like `jax.jit`, `jax.vmap`, and `jax.grad`. The shapes of output arrays must not depend on values within other arrays. Use techniques like `jnp.where` to work around the need for dynamically-sized arrays. + +- **NaN Handling**: Use the NaN-checker during debugging by setting `JAX_DEBUG_NANS=True` or using `jax.config.update("jax_debug_nans", True)`. Be aware that this adds overhead and should be disabled in production. + +- **Double Precision**: Enable double-precision numbers by setting the `jax_enable_x64` configuration variable at startup (`JAX_ENABLE_X64=True` environment variable, `jax.config.update("jax_enable_x64", True)`, or `jax.config.parse_flags_with_absl()`). Note that not all backends support 64-bit convolutions. + +- **Non-array Inputs**: JAX generally expects NumPy arrays as inputs to its API functions. Avoid passing Python lists or tuples directly; convert them to arrays first. + +- **Out-of-Bounds Indexing**: JAX handles out-of-bounds indexing by clamping the index to the bounds of the array for retrieval operations. Array update operations at out-of-bounds indices are skipped. Use the optional parameters of `ndarray.at` for finer-grained control. + +- **Miscellaneous Divergences from NumPy**: Be aware of differences in type promotion rules, unsafe type casts, and other corner cases where JAX's behavior may differ from NumPy. + +## Code Organization and Structure: + +- **Directory Structure**: Consider a structure like this: + + project_root/ + ├── src/ + │ ├── __init__.py + │ ├── models/ + │ │ ├── __init__.py + │ │ ├── model_a.py + │ │ └── model_b.py + │ ├── layers/ + │ │ ├── __init__.py + │ │ ├── layer_a.py + │ │ └── layer_b.py + │ ├── utils/ + │ │ ├── __init__.py + │ │ ├── data_loading.py + │ │ └── evaluation.py + │ ├── train.py + │ └── predict.py + ├── tests/ + │ ├── __init__.py + │ ├── models/ + │ │ ├── test_model_a.py + │ │ └── test_model_b.py + │ ├── utils/ + │ │ ├── test_data_loading.py + │ │ └── test_evaluation.py + │ └── test_train.py + ├── notebooks/ + │ ├── exploration.ipynb + │ └── analysis.ipynb + ├── data/ + │ ├── raw/ + │ └── processed/ + ├── models/ + │ └── saved_models/ + ├── .gitignore + ├── README.md + ├── requirements.txt + └── setup.py + + +- **File Naming**: Use descriptive, lowercase names with underscores (e.g., `data_loading.py`, `model_a.py`). + +- **Module Organization**: Group related functionalities into modules. Use clear and concise module names. + +- **Component Architecture**: Favor a modular architecture. Decouple components where possible. Use interfaces or abstract base classes when appropriate. + +- **Code Splitting**: Split large files into smaller, more manageable modules. Consider splitting based on functionality or abstraction level. + +## Common Patterns and Anti-patterns: + +- **Design Patterns**: Consider patterns like: + - **Strategy**: For selecting different computation strategies at runtime. + - **Factory**: For creating JAX models or layers. + - **Observer**: For monitoring training progress. + +- **Recommended Approaches**: + - Using `jax.tree_util` for working with nested data structures (pytrees). + - Caching intermediate results with `lru_cache` (with caution, as it introduces state). + - Using `jax.experimental.optimizers` for defining optimization schedules. + +- **Anti-patterns**: + - Mutable global state within JIT-compiled functions. + - Excessive use of `static_argnums` (leads to recompilation). + - Ignoring the immutability of JAX arrays. + - Using Python loops instead of vectorized operations (`vmap`). + - Unnecessary host-device data transfers. + - Reusing PRNG keys without splitting. + +- **State Management**: + - Avoid global mutable state. Pass state explicitly as function arguments. + - Consider using immutable data structures for state. + - If mutable state is absolutely necessary (e.g., in some RL settings), use JAX's stateful computation tools carefully (e.g., `jax.lax.scan` with a carry). + +- **Error Handling**: + - Use `try...except` blocks for handling potential errors during data loading or preprocessing. + - Employ assertions to check for invalid input or unexpected conditions. + - Utilize logging to track errors and debug issues. + - The `JAX_DEBUG_NANS` and `JAX_DEBUG_NANS=True` flag is critical when debugging `NaN`. + +## Performance Considerations: + +- **Optimization Techniques**: + - JIT compilation (`jax.jit`). + - Vectorization (`jax.vmap`). + - Parallelization (`jax.pmap`). + - Fusion of operations. + - Reducing host-device data transfers. + - Choose appropriate data types (e.g., `float32` instead of `float64` if sufficient). + - Use `jax.numpy` functions instead of NumPy functions where possible. + - Avoid scalar operations inside JIT-compiled loops; use array operations. + +- **Memory Management**: + - Minimize the creation of large intermediate arrays. + - Reuse arrays where possible (using `.at[].set()` for in-place updates if safe to do so). + - Be aware of memory fragmentation. + - If you are dealing with datasets larger than memory, explore data streaming/sharding approaches. + +- **Rendering Optimization**: (Relevant if using JAX for visualization, e.g., with OpenGL or similar libraries) + - Batch rendering operations. + - Optimize data transfer to the rendering device (e.g., GPU). + - Consider using specialized rendering libraries that are compatible with JAX arrays. + +- **Bundle Size Optimization**: (For JAX-based applications deployed in web environments) + - Tree-shake unused code. + - Minify JavaScript bundles. + - Use code splitting to load only necessary modules. + - Compress assets (e.g., images, data files). + +- **Lazy Loading**: + - Load data or models only when needed. + - Use iterators or generators for large datasets. + - Implement a loading indicator to provide feedback to the user. + +## Security Best Practices: + +- **Common Vulnerabilities**: + - Input data poisoning. + - Model inversion attacks. + - Adversarial examples. + - Side-channel attacks (less relevant in typical JAX applications, but important in cryptographic applications). + +- **Input Validation**: + - Validate input data types and ranges. + - Sanitize input data to prevent injection attacks. + - Use schema validation libraries (e.g., `cerberus`, `jsonschema`). + +- **Authentication and Authorization**: (If building APIs or web services with JAX) + - Implement secure authentication mechanisms (e.g., OAuth 2.0, JWT). + - Use role-based access control (RBAC) to restrict access to sensitive resources. + - Protect API endpoints with appropriate authentication and authorization checks. + +- **Data Protection**: + - Encrypt sensitive data at rest and in transit. + - Use secure storage mechanisms for models and data. + - Implement data masking or anonymization techniques to protect personally identifiable information (PII). + +- **Secure API Communication**: + - Use HTTPS for all API communication. + - Implement proper rate limiting to prevent denial-of-service attacks. + - Use secure coding practices to prevent common web vulnerabilities (e.g., cross-site scripting, SQL injection). + +## Testing Approaches: + +- **Unit Testing**: + - Test individual functions or classes in isolation. + - Use `pytest` or `unittest` for writing unit tests. + - Mock external dependencies to isolate the code under test. + - Test different input scenarios and edge cases. + - Use `jax.random.PRNGKey` with a fixed seed for reproducible tests. + +- **Integration Testing**: + - Test the interaction between different components or modules. + - Verify that data flows correctly between components. + - Test the integration with external services or databases. + +- **End-to-End Testing**: + - Test the entire application flow from start to finish. + - Simulate real user interactions. + - Verify that the application meets the specified requirements. + - Use tools like Selenium or Cypress for end-to-end testing. + +- **Test Organization**: + - Organize tests into separate directories or modules. + - Use descriptive test names. + - Follow a consistent testing style. + +- **Mocking and Stubbing**: + - Use `unittest.mock` or `pytest-mock` for mocking external dependencies. + - Create stubs for complex or time-consuming operations. + - Mock JAX functions (e.g., `jax.random.normal`) to control the output of random number generators. + +## Common Pitfalls and Gotchas: + +- **Frequent Mistakes**: + - Mutable global state in JIT-compiled functions. + - Incorrect use of `static_argnums`. + - Ignoring JAX array immutability. + - Using Python loops instead of `vmap`. + - Reusing PRNG keys without splitting. + +- **Edge Cases**: + - Handling `NaN` and `Inf` values. + - Dealing with numerical instability. + - Working with sparse data. + - Optimizing for different hardware backends (CPU, GPU, TPU). + +- **Version-Specific Issues**: + - Be aware of breaking changes between JAX versions. + - Consult the JAX documentation for the latest updates and bug fixes. + +- **Compatibility Concerns**: + - Ensure compatibility between JAX and other libraries (e.g., TensorFlow, PyTorch). + - Be aware of potential conflicts between JAX and NumPy versions. + +- **Debugging Strategies**: + - Use the JAX debugger (`jax.debug.print`). + - Set `JAX_DEBUG_NANS=True` to detect `NaN` values. + - Use `jax.make_jaxpr` to inspect the JAX program representation. + - Use `jax.debug.visualize_jaxpr` to visualize the JAX program flow (requires graphviz). + - Profile your code to identify performance bottlenecks. + - Use a debugger (e.g., `pdb`) to step through your code and inspect variables. + +## Tooling and Environment: + +- **Recommended Development Tools**: + - VS Code with the Python extension. + - Jupyter Notebook or JupyterLab for interactive development. + - IPython for a powerful interactive shell. + - TensorBoard for visualizing training progress. + +- **Build Configuration**: + - Use `requirements.txt` or `setup.py` to manage dependencies. + - Specify JAX version requirements. + - Use a virtual environment (e.g., `venv`, `conda`) to isolate dependencies. + +- **Linting and Formatting**: + - Use `flake8` or `pylint` for linting. + - Use `black` or `autopep8` for formatting. + - Configure your editor to automatically lint and format code on save. + +- **Deployment Best Practices**: + - Package your application into a Docker container. + - Use a cloud platform (e.g., AWS, Google Cloud, Azure) for deployment. + - Use a deployment framework (e.g., Flask, FastAPI) for serving APIs. + - Monitor your application for performance and errors. + +- **CI/CD Integration**: + - Use a CI/CD platform (e.g., GitHub Actions, Travis CI, CircleCI). + - Automate testing, linting, and formatting. + - Automate deployment to staging and production environments. + + +This comprehensive guide should help developers write better JAX code, avoid common pitfalls, and build high-performance machine learning applications. \ No newline at end of file diff --git a/.cursor/rules/jenkins.mdc b/.cursor/rules/jenkins.mdc new file mode 100644 index 000000000..13fdabd83 --- /dev/null +++ b/.cursor/rules/jenkins.mdc @@ -0,0 +1,194 @@ +--- +description: Comprehensive best practices for Jenkins, covering code organization, security, performance, testing, and common pitfalls. Provides guidelines for writing robust, maintainable, and secure Jenkins pipelines and configurations. +globs: Jenkinsfile,*.jenkins,*.groovy +--- +# Jenkins Library Best Practices and Coding Standards + +This document outlines best practices for developing and maintaining Jenkins pipelines and configurations to ensure robustness, security, and efficiency. + +## 1. Code Organization and Structure + +* **Directory Structure:** + * `/vars/`: Store shared pipeline libraries (Groovy files). Each file in this directory represents a callable method. + * `/resources/`: Place non-Groovy resources like configuration files or scripts used by the pipeline. + * `Jenkinsfile`: The primary pipeline definition file, typically at the root of the repository. + * `.cursor/rules`: Where .mdc rule files should be +* **File Naming Conventions:** + * Pipeline definitions: `Jenkinsfile` (recommended), or `.jenkins` + * Shared libraries: `lowercaseCamelCase.groovy` (e.g., `buildImage.groovy`) + * Resource files: Descriptive names with appropriate extensions (e.g., `settings.xml`, `deploy.sh`) +* **Module Organization:** + * Break down complex pipelines into smaller, reusable shared libraries. + * Group related functions within shared libraries into logical modules. + * Use descriptive names for shared libraries to indicate their purpose (e.g., `awsUtils.groovy`, `dockerBuild.groovy`). +* **Component Architecture:** + * Adopt a modular design for shared libraries, separating concerns and promoting reusability. + * Create abstract functions that can be extended or customized for specific projects. + * Avoid tightly coupling shared libraries to specific projects or environments. +* **Code Splitting Strategies:** + * Split long `Jenkinsfile`s into multiple files using `load()` or shared libraries. This improves readability and maintainability. + * Delegate complex tasks to external scripts or tools invoked from the pipeline. + * Use environment variables to parameterize pipeline behavior and avoid hardcoding values. + +## 2. Common Patterns and Anti-patterns + +* **Design Patterns:** + * **Strategy Pattern:** Implement different build or deployment strategies based on input parameters. + * **Template Method Pattern:** Define a base pipeline structure with customizable steps for specific projects. + * **Facade Pattern:** Create a simplified interface to complex systems or tools. +* **Recommended Approaches:** + * Use declarative pipelines for improved readability and maintainability. + * Employ shared libraries to promote code reuse and consistency. + * Automate testing and security scanning within the pipeline. + * Monitor pipeline performance and optimize for speed and efficiency. + * Use a version control system (e.g., Git) to track changes to pipeline definitions and shared libraries. +* **Anti-patterns and Code Smells:** + * **Large, monolithic `Jenkinsfile`s:** Difficult to read, maintain, and debug. + * **Hardcoding sensitive information:** Compromises security and makes pipelines inflexible. + * **Lack of error handling:** Pipelines fail without clear error messages or recovery mechanisms. + * **Excessive Groovy scripting:** Can impact performance and increase complexity. Delegate tasks to external tools where possible. + * **Ignoring security vulnerabilities:** Exposes systems to potential attacks. + * **Duplicated code across pipelines:** Indicates a need for shared libraries. +* **State Management:** + * Use environment variables to store pipeline state. + * Persist state to external systems (e.g., databases, artifact repositories) for long-running pipelines. + * Avoid relying on global variables or mutable shared state. +* **Error Handling:** + * Use `try...catch` blocks to handle exceptions and prevent pipeline failures. + * Log error messages with sufficient detail to facilitate debugging. + * Implement retry mechanisms for transient errors. + * Use the `error` step to explicitly fail the pipeline with a custom message. + * Consider using `unstash` with error handling to clean up temporary files. + +## 3. Performance Considerations + +* **Optimization Techniques:** + * Parallelize build and test stages to reduce overall execution time. + * Use caching to avoid redundant downloads and computations. + * Optimize Groovy code for performance (e.g., avoid unnecessary iterations). + * Use lightweight agents with sufficient resources to handle the workload. + * Leverage build accelerators and distributed build systems. +* **Memory Management:** + * Avoid loading large files into memory using `JsonSlurper` or `XmlSlurper`. Use `sh` step with tools like `jq` or `xmllint` instead. + * Limit the use of global variables and shared state to reduce memory consumption. + * Monitor agent memory usage and adjust agent resources accordingly. +* **Bundle Size Optimization:** (Not directly applicable to Jenkins pipelines but relevant to applications built by Jenkins) + * Minimize dependencies and remove unused code. + * Use code splitting to load only the necessary code modules. + * Compress artifacts before archiving or deploying them. +* **Lazy Loading:** (Not directly applicable to Jenkins pipelines themselves, but is related to web applications that are built using Jenkins). + * Load resources on demand instead of upfront to reduce initial load time. + * Defer the execution of non-critical tasks until they are needed. + * Utilize asynchronous operations to avoid blocking the pipeline execution. + +## 4. Security Best Practices + +* **Common Vulnerabilities:** + * **Cross-Site Scripting (XSS):** Sanitize user input to prevent malicious code injection. + * **Cross-Site Request Forgery (CSRF):** Enable CSRF protection to prevent unauthorized requests. + * **Remote Code Execution (RCE):** Avoid executing untrusted code within the pipeline. + * **Credential Theft:** Protect sensitive credentials using Jenkins' credential management system. + * **Unauthorized Access:** Implement role-based access control to restrict access to sensitive resources. +* **Input Validation:** + * Validate all user inputs to prevent malicious code injection and data corruption. + * Use parameterized builds with predefined choices to limit user input. + * Encode or escape special characters to prevent interpretation as code. +* **Authentication and Authorization:** + * Enable security and configure authentication using Jenkins' built-in user database or an external identity provider (e.g., LDAP, Active Directory). + * Implement role-based access control (RBAC) to restrict access to sensitive resources and operations. + * Use the principle of least privilege to grant only the necessary permissions to users and groups. + * Regularly audit user permissions and remove unnecessary accounts. +* **Data Protection:** + * Use Jenkins' credential management system to securely store passwords, API keys, and other sensitive information. + * Encrypt sensitive data at rest and in transit. + * Mask sensitive information in build logs to prevent exposure. + * Rotate credentials regularly to minimize the impact of potential breaches. +* **Secure API Communication:** + * Use HTTPS to encrypt communication between Jenkins and other systems. + * Authenticate all API requests using secure tokens or credentials. + * Limit API access to authorized users or systems. + * Rate-limit API requests to prevent denial-of-service attacks. + +## 5. Testing Approaches + +* **Unit Testing:** + * Test individual functions or modules in isolation. + * Use mocking and stubbing to isolate dependencies. + * Write unit tests for shared libraries to ensure they function correctly. +* **Integration Testing:** + * Test the interaction between different components or services. + * Verify that shared libraries integrate correctly with the pipeline. + * Use containerization to create isolated test environments. +* **End-to-End Testing:** + * Test the entire pipeline workflow from start to finish. + * Simulate real-world scenarios and user interactions. + * Use automated testing tools to execute end-to-end tests. +* **Test Organization:** + * Organize tests into logical suites based on functionality or component. + * Use descriptive names for test cases to indicate their purpose. + * Run tests automatically as part of the pipeline. + * Generate test reports and track test results over time. +* **Mocking and Stubbing:** + * Use mocking frameworks to create mock objects that simulate the behavior of dependencies. + * Stub external services or APIs to isolate the system under test. + * Use environment variables to configure mocking behavior. + +## 6. Common Pitfalls and Gotchas + +* **Frequent Mistakes:** + * Failing to secure Jenkins instance. + * Overusing Groovy scripting leading to performance degradation. + * Not using shared libraries for reusable code. + * Hardcoding credentials and other secrets. + * Lack of proper error handling. + * Ignoring pipeline performance monitoring. +* **Edge Cases:** + * Handling concurrent builds and resource contention. + * Dealing with flaky tests and intermittent failures. + * Managing large files and artifacts within the pipeline. + * Supporting different operating systems and environments. +* **Version-Specific Issues:** + * Compatibility issues between Jenkins versions and plugins. + * Deprecated APIs and features. + * Security vulnerabilities in older versions. +* **Compatibility Concerns:** + * Ensure compatibility between Jenkins and the tools and technologies used in the pipeline (e.g., Docker, Kubernetes, AWS). + * Test pipelines on different platforms to ensure cross-platform compatibility. +* **Debugging Strategies:** + * Use build logs and console output to diagnose pipeline failures. + * Add debug statements to Groovy code to trace execution flow. + * Use remote debugging tools to step through pipeline execution. + * Reproduce pipeline failures locally to isolate the cause. + * Utilize the `script` step with caution for complex logic, ensuring proper error handling. + * Review plugin documentation for common issues and troubleshooting tips. + +## 7. Tooling and Environment + +* **Recommended Development Tools:** + * IDE with Groovy support (e.g., IntelliJ IDEA, Eclipse). + * Version control system (e.g., Git). + * Build automation tools (e.g., Maven, Gradle). + * Testing frameworks (e.g., JUnit, TestNG). + * Containerization tools (e.g., Docker). +* **Build Configuration:** + * Use parameterized builds to allow users to customize build behavior. + * Define build triggers to automatically start builds on code changes or schedule. + * Configure post-build actions to archive artifacts, send notifications, or deploy the application. + * Use the `buildDiscarder` directive to clean up old builds and save disk space. +* **Linting and Formatting:** + * Use a Groovy linter to enforce coding standards and identify potential errors. + * Format Groovy code consistently to improve readability. + * Use tools like `groovy-lint` or IDE plugins for linting and formatting. +* **Deployment:** + * Use deployment plugins to automate the deployment process. + * Implement blue-green deployments or rolling deployments to minimize downtime. + * Use feature flags to decouple deployment from release. + * Use a cloud-native deployment strategy (e.g., Kubernetes) for scalable and resilient deployments. +* **CI/CD Integration:** + * Integrate Jenkins with other CI/CD tools (e.g., SonarQube, Artifactory). + * Use webhooks to trigger builds from version control systems. + * Monitor pipeline performance and identify areas for improvement. + * Implement continuous feedback loops to improve the quality of the application and the pipeline. + * Leverage Infrastructure as Code (IaC) tools such as Terraform to create ephemeral testing and build environments + +By following these best practices, developers can create robust, maintainable, and secure Jenkins pipelines and configurations that streamline the software development process and improve the quality of the application. \ No newline at end of file diff --git a/.cursor/rules/jest.mdc b/.cursor/rules/jest.mdc new file mode 100644 index 000000000..3c35413a6 --- /dev/null +++ b/.cursor/rules/jest.mdc @@ -0,0 +1,139 @@ +--- +description: This rule provides guidelines for writing clean, maintainable, and effective tests using Jest. It covers code organization, performance, common pitfalls, and best practices for testing JavaScript and TypeScript projects. +globs: *.js,*.jsx,*.ts,*.tsx,*.mjs,*.cjs,*.mts,*.cts +--- +- **Organize tests by feature or module:** + - Group tests into files or directories that correspond to the features or modules they are testing. This makes it easier to locate and maintain tests. + - Create a `__tests__` directory alongside your source code files. This is a common convention that Jest recognizes. + +- **Use descriptive test names:** + - Write test names that clearly describe what the test is verifying. This makes it easier to understand the purpose of each test and to diagnose failures. + - Use the `describe` and `it` blocks to structure your tests and provide context. + - Example: `describe('User authentication', () => { it('should log in a user with valid credentials', () => { /* ... */ }); });` + +- **Keep tests isolated and independent:** + - Each test should be independent of other tests. Avoid sharing state or dependencies between tests. + - Use `beforeEach` and `afterEach` hooks to set up and tear down the environment for each test. + +- **Avoid testing implementation details:** + - Focus on testing the public API of your code, rather than the internal implementation details. + - This makes your tests more resilient to changes in the implementation. + - Test the "what", not the "how". + +- **Use setup and teardown methods:** + - Use `beforeAll`, `afterAll`, `beforeEach`, and `afterEach` hooks to set up and tear down the environment for your tests. + - `beforeAll` and `afterAll` run once before and after all tests in a `describe` block. + - `beforeEach` and `afterEach` run before and after each test in a `describe` block. + +- **Mock external dependencies:** + - Use mocking to isolate your code from external dependencies, such as network requests or database connections. + - Jest provides built-in mocking capabilities with `jest.mock` and `jest.spyOn`. + - Consider using a library like `axios-mock-adapter` for mocking HTTP requests. + +- **Write tests that are easy to read and maintain:** + - Keep your tests concise and focused. + - Use clear and consistent formatting. + - Add comments to explain complex logic. + - Refactor your tests regularly to keep them up to date. + +- **Aim for high test coverage, but prioritize meaningful tests over quantity:** + - Aim for high test coverage to ensure that your code is well-tested. + - However, prioritize writing meaningful tests that verify the core functionality of your code. + - Don't just aim for 100% coverage without considering the value of each test. + +- **Use Jest's built-in matchers effectively:** + - Jest provides a rich set of matchers for asserting different conditions. + - Use matchers like `toBe`, `toEqual`, `toBeGreaterThan`, `toContain`, `toHaveBeenCalled`, etc. + - Explore the Jest documentation for the full list of matchers. + +- **Handle asynchronous code correctly:** + - Use `async/await` or Promises to handle asynchronous code in your tests. + - Use the `resolves` and `rejects` matchers to assert that a Promise resolves or rejects. + - Example: `expect(myAsyncFunction()).resolves.toBe(expectedValue);` + +- **Test error handling:** + - Write tests to verify that your code handles errors correctly. + - Use the `toThrow` matcher to assert that a function throws an error. + - Example: `expect(() => myDangerousFunction()).toThrow(Error);` + +- **Use snapshots sparingly:** + - Snapshots can be useful for verifying the output of a component or function. + - However, they can also be brittle and difficult to maintain if used excessively. + - Use snapshots strategically and review them carefully when they change. + +- **Configure Jest correctly:** + - Configure Jest using the `jest.config.js` or `jest.config.ts` file. + - Configure settings such as test environment, module file extensions, and coverage thresholds. + - Consider using presets like `ts-jest` or `babel-jest` for TypeScript or Babel support. + +- **Leverage code coverage reports:** + - Use Jest's code coverage reports to identify areas of your code that are not well-tested. + - Aim to increase coverage in critical areas of your codebase. + - Use the `--coverage` flag to generate coverage reports. + +- **Keep test data separate from test logic:** + - Externalize test data to improve readability and maintainability. + - Use fixtures or factories to generate test data. + - Avoid hardcoding data directly in your tests. + +- **Consider using test-driven development (TDD):** + - Write tests before you write code to drive the development process. + - This can help you write more testable and well-designed code. + +- **Run tests frequently:** + - Run your tests frequently to catch errors early. + - Use Jest's watch mode to automatically run tests when files change. + - Integrate tests into your CI/CD pipeline. + +- **Document your tests:** + - Add comments to explain the purpose of each test and the expected behavior. + - This will make it easier for others to understand and maintain your tests. + +- **Code Organization and Structure:** + - **Directory structure best practices:** Organize test files alongside the component/module they test (e.g., `src/components/MyComponent/MyComponent.test.js`) + - **File naming conventions:** Use `.test.js` or `.spec.js` suffixes for test files (e.g., `MyComponent.test.js`, `MyModule.spec.ts`). + - **Module organization:** Keep test files close to the modules they are testing. Use a consistent naming convention for test files. + - **Component architecture:** Test components in isolation, mocking dependencies where necessary. + - **Code splitting strategies:** Ensure tests cover all code paths in dynamically imported modules. + +- **Common Patterns and Anti-patterns:** + - **Design patterns specific to Jest:** Use Page Object Model (POM) for UI tests, Factory pattern for test data generation. + - **Recommended approaches for common tasks:** Use `jest.mock` for mocking modules, `jest.spyOn` for spying on methods, and `fakeTimers` for controlling time-dependent behavior. + - **Anti-patterns and code smells to avoid:** Avoid testing implementation details, relying on global state, and creating brittle snapshots. + - **State management best practices:** Mock external state dependencies and verify state changes using `expect` assertions. + - **Error handling patterns:** Test for specific error messages and ensure error boundaries are properly tested. + +- **Performance Considerations:** + - **Optimization techniques:** Use `jest.clearAllMocks` and `jest.resetAllMocks` in `beforeEach` blocks to prevent state leakage and improve test performance. + - **Memory management:** Avoid creating large data structures in tests that are not necessary. Manually trigger garbage collection in tests where large objects are created and released. + - **Rendering optimization:** Mock expensive rendering logic to speed up tests that involve React components. + - **Bundle size optimization:** Not applicable, as Jest primarily tests individual modules or components. + - **Lazy loading strategies:** Ensure tests cover all code paths in dynamically imported modules. + +- **Security Best Practices:** + - **Common vulnerabilities and how to prevent them:** Avoid using sensitive data in test snapshots. Sanitize test data to prevent potential injection attacks. + - **Input validation:** Test input validation logic and ensure that invalid inputs are handled correctly. + - **Authentication and authorization patterns:** Mock authentication and authorization logic to test different user roles and permissions. + - **Data protection strategies:** Not directly applicable, as Jest primarily focuses on functional testing. + - **Secure API communication:** Mock API calls and verify that data is transmitted securely. + +- **Testing Approaches:** + - **Unit testing strategies:** Test individual functions, classes, or components in isolation. + - **Integration testing:** Test interactions between different modules or components. + - **End-to-end testing:** Use tools like Cypress or Playwright for end-to-end tests. + - **Test organization:** Group tests into logical suites based on functionality or module. + - **Mocking and stubbing:** Use `jest.mock` and `jest.spyOn` to create mocks and stubs for dependencies. + +- **Common Pitfalls and Gotchas:** + - **Frequent mistakes developers make:** Forgetting to mock dependencies, testing implementation details, and creating brittle snapshots. + - **Edge cases to be aware of:** Testing asynchronous code correctly, handling errors, and testing different input types. + - **Version-specific issues:** Be aware of breaking changes in Jest updates and update your tests accordingly. + - **Compatibility concerns:** Ensure tests are compatible with different browsers or environments. + - **Debugging strategies:** Use `console.log` statements, debuggers, or Jest's interactive mode to debug tests. + +- **Tooling and Environment:** + - **Recommended development tools:** VS Code, WebStorm, Jest Runner extension. + - **Build configuration:** Configure Jest using `jest.config.js` or `package.json`. + - **Linting and formatting:** Use ESLint and Prettier to enforce code style and prevent errors. + - **Deployment best practices:** Integrate tests into your CI/CD pipeline to ensure code quality. + - **CI/CD integration:** Use tools like GitHub Actions, Jenkins, or CircleCI to automate testing and deployment. \ No newline at end of file diff --git a/.cursor/rules/jetpack-compose.mdc b/.cursor/rules/jetpack-compose.mdc new file mode 100644 index 000000000..05e9e7e6f --- /dev/null +++ b/.cursor/rules/jetpack-compose.mdc @@ -0,0 +1,168 @@ +--- +description: Enforces Jetpack Compose best practices for code organization, performance, and maintainability. This rule provides guidelines for writing efficient and idiomatic Compose code. +globs: *.kt +--- +- **Code Organization and Structure** + - **Directory Structure:** + - Organize composables by feature or screen. Each feature should have its own directory. + - Example: + + app/ + src/ + main/ + java/ + com/example/app/ + feature_a/ + FeatureAScreen.kt + FeatureAViewModel.kt + components/ + FeatureAButton.kt + FeatureATextField.kt + feature_b/ + ... + + - **File Naming Conventions:** + - Use PascalCase for composable function names (e.g., `MyComposable`). + - Use descriptive names that clearly indicate the composable's purpose. + - Name files after the primary composable they contain (e.g., `MyComposable.kt`). + - **Module Organization:** + - Separate UI code (composables) from business logic (ViewModels, repositories). Use modules to enforce this separation. + - Consider using feature modules for large applications to improve build times and maintainability. + - **Component Architecture:** + - Design composables as small, reusable components. + - Follow the single responsibility principle: each composable should have a single, well-defined purpose. + - Use a hierarchical component structure to build complex UIs from smaller, simpler components. + - **Code Splitting Strategies:** + - Break down large composables into smaller, more manageable parts. + - Use inline functions judiciously. Overuse can hurt performance. + - Consider using `derivedStateOf` to only recompose when necessary based on derived state. + +- **Common Patterns and Anti-patterns** + - **Design Patterns:** + - **State Hoisting:** Move state as high as possible in the composable tree to maximize reusability and testability. Pass state down as parameters. + - **Unidirectional Data Flow:** Data flows down the composable tree, and events flow up. This makes it easier to reason about the state of the UI. + - **Composition Local:** Use sparingly to provide implicit dependencies to composables. Avoid overusing as it can make dependencies less explicit. + - **ViewModel:** Use a ViewModel to hold UI state and handle business logic. This separates the UI from the data layer and makes the UI easier to test. + - **Recommended Approaches:** + - Use `remember` to cache expensive calculations and avoid unnecessary recompositions. + - Use `mutableStateOf` or `rememberSaveable` to store UI state. + - Use `LaunchedEffect` or `rememberCoroutineScope` for side effects (e.g., network requests, database access). + - Use `animate*AsState` functions for smooth animations. + - **Anti-patterns and Code Smells:** + - **Reading State from Too High a Scope:** Avoid reading state too high in the composition tree, as this can cause unnecessary recompositions. + - **Mutable State in Composables without remember:** If you're not using remember, every recomposition will create a new state, leading to unexpected behavior. + - **Long Composable Functions:** Break large composables into smaller, more manageable parts. + - **Hardcoded Values:** Avoid hardcoding values directly in composables. Use resources or constants instead. + - **Unnecessary Recomposition:** Profile your code to identify and eliminate unnecessary recompositions. Tools like the Layout Inspector can help. + - **State Management Best Practices:** + - Choose the right state management solution for your needs (e.g., `remember`, `mutableStateOf`, `StateFlow`, `LiveData`). + - Keep state as immutable as possible. Use `copy()` to create new state objects instead of modifying existing ones. + - Use `derivedStateOf` to derive state from other state objects. + - **Error Handling Patterns:** + - Use `try-catch` blocks to handle exceptions in composables. + - Display error messages to the user in a clear and informative way. + - Use a central error handling mechanism to log errors and prevent crashes. Consider using a `Snackbar` or a dedicated error display composable. + +- **Performance Considerations** + - **Optimization Techniques:** + - **`remember`:** Cache expensive calculations and resources. + - **`derivedStateOf`:** Only recompose when derived state changes. + - **`SnapshotFlow`:** Collect data from mutable state without recomposing. + - **`CompositionLocalProvider`:** Provides an alternative way to pass data if recomposition is a bottleneck. Use with caution as it can make dependencies implicit. + - **`Skippable Composable Functions`:** Compose Compiler performs the best optimization for functions that are skippable. To allow skipping, a composable must meet the following criteria: + - All parameters passed to the composable are stable. + - The composable's result is the same given the same parameters. + - **Inline Functions (with care):** Useful for small functions but can increase bytecode size if overused. + - **Memory Management:** + - Avoid creating large objects in composables. + - Release resources when they are no longer needed. + - Use `WeakReference` to avoid memory leaks. + - **Rendering Optimization:** + - Use `Modifier.drawBehind` and `Modifier.drawWithContent` for custom drawing. + - Avoid overdraw by using `Modifier.clip` and `Modifier.background`. + - Use `Spacer` to control layout instead of adding padding to multiple elements. + - **Bundle Size Optimization:** + - Use R8 to shrink and obfuscate your code. + - Remove unused resources. + - Use dynamic feature modules to deliver features on demand. + - **Lazy Loading Strategies:** + - Use `LazyColumn` and `LazyRow` to display large lists of data. + - Use `rememberLazyListState` to persist the scroll position of lazy lists. + +- **Security Best Practices** + - **Common Vulnerabilities:** + - **Input Validation:** Sanitize user input to prevent injection attacks. + - **Data Protection:** Encrypt sensitive data at rest and in transit. + - **Secure API Communication:** Use HTTPS to encrypt communication between the app and the server. + - **Input Validation:** + - Validate user input to prevent injection attacks and other security vulnerabilities. + - Use regular expressions or other validation techniques to ensure that input is in the expected format. + - **Authentication and Authorization Patterns:** + - Use a secure authentication and authorization mechanism to protect sensitive data and functionality. + - Use OAuth 2.0 or OpenID Connect for authentication. + - Use role-based access control (RBAC) for authorization. + - **Data Protection Strategies:** + - Encrypt sensitive data at rest and in transit. + - Use a strong encryption algorithm. + - Store encryption keys securely. + - **Secure API Communication:** + - Use HTTPS to encrypt communication between the app and the server. + - Use certificate pinning to prevent man-in-the-middle attacks. + +- **Testing Approaches** + - **Unit Testing Strategies:** + - Test composables in isolation using `ComposeTestRule`. + - Verify that composables render correctly and handle user input as expected. + - Use `composeTestRule.setContent { ... }` to set the content of the test. + - Use `composeTestRule.onNodeWithText("...").performClick()` to simulate user interactions. + - **Integration Testing:** + - Test the interaction between different composables and ViewModels. + - Use `Hilt` or `Koin` to inject dependencies into tests. + - **End-to-end Testing:** + - Test the entire application flow from the user's perspective. + - Use `UIAutomator` or `Espresso` for end-to-end testing. + - **Test Organization:** + - Organize tests by feature or screen. + - Create a separate test module for each feature module. + - **Mocking and Stubbing:** + - Use `Mockito` or `Mockk` to mock dependencies in unit tests. + - Use `Fake` implementations for integration tests. + +- **Common Pitfalls and Gotchas** + - **Frequent Mistakes:** + - Forgetting to use `remember` for state. + - Reading state from too high a scope. + - Not handling errors properly. + - Not optimizing performance. + - **Edge Cases:** + - Handling different screen sizes and orientations. + - Handling different locales and languages. + - Handling different accessibility requirements. + - **Version-Specific Issues:** + - Be aware of breaking changes in new versions of Jetpack Compose. + - Use the latest version of Jetpack Compose to take advantage of new features and bug fixes. + - **Compatibility Concerns:** + - Ensure that your app is compatible with different Android versions. + - Use Jetpack Compose libraries that are compatible with your target Android version. + - **Debugging Strategies:** + - Use the Android Studio debugger to step through your code and inspect variables. + - Use the Layout Inspector to inspect the composable tree and identify performance bottlenecks. + - Use the Profiler to measure the performance of your app. + +- **Tooling and Environment** + - **Recommended Development Tools:** + - Android Studio + - Kotlin Compiler + - Jetpack Compose Libraries + - **Build Configuration:** + - Use Gradle to manage dependencies and build your app. + - Configure the Kotlin compiler to use the latest version of Jetpack Compose. + - **Linting and Formatting:** + - Use `ktlint` or `detekt` to enforce code style guidelines. + - Use `Prettier` to format your code automatically. + - **Deployment Best Practices:** + - Use the Google Play Store to distribute your app. + - Use a staged rollout to gradually release your app to users. + - **CI/CD Integration:** + - Use a CI/CD system to automate the build, test, and deployment process. + - Use `GitHub Actions` or `CircleCI` for CI/CD. \ No newline at end of file diff --git a/.cursor/rules/jquery.mdc b/.cursor/rules/jquery.mdc new file mode 100644 index 000000000..e3b1a2640 --- /dev/null +++ b/.cursor/rules/jquery.mdc @@ -0,0 +1,410 @@ +--- +description: This rule file provides guidelines for jQuery development, covering code organization, performance, security, and testing. It helps developers write maintainable, efficient, and secure jQuery code. +globs: *.js +--- +# jQuery Best Practices: A Comprehensive Guide + +This document outlines best practices for jQuery development to ensure code quality, performance, security, and maintainability. + +## Library Information: + +- Name: jQuery +- Tags: javascript, dom, library, frontend + +## 1. Code Organization and Structure + +A well-structured project improves maintainability, collaboration, and scalability. + +### 1.1. Directory Structure Best Practices + +Adopt a clear and consistent directory structure: + + +project-root/ +├── css/ +│ ├── style.css +│ └── components/ +│ └── component.css +├── js/ +│ ├── jquery.min.js # jQuery library (ideally from a CDN) +│ ├── app.js # Main application file +│ ├── modules/ +│ │ ├── module1.js +│ │ └── module2.js +│ ├── components/ +│ │ ├── component1.js +│ │ └── component2.js +│ └── utils/ +│ ├── helperFunctions.js +│ └── ajaxUtils.js +├── img/ +│ └── ... +├── index.html +├── .eslintrc.js # ESLint configuration (see linting section) +└── package.json # Project dependencies and scripts + + +**Explanation:** + +- `css/`: Contains all CSS files, potentially organized into components. +- `js/`: Contains all JavaScript files, including the main application script, modules, components, and utility functions. +- `img/`: Contains images. +- `index.html`: The main HTML file. +- `.eslintrc.js`: Configuration file for ESLint (JavaScript linter). +- `package.json`: Defines project dependencies and scripts. + +### 1.2. File Naming Conventions + +Use descriptive and consistent file names: + +- JavaScript files: `[componentName].js`, `[moduleName].js`, `[utilityName].js` (e.g., `navigation.js`, `userAuthentication.js`, `dateFormatter.js`). +- CSS files: `[componentName].css`, `style.css`. +- jQuery plugins: `jquery.[pluginName].js`. + +### 1.3. Module Organization Best Practices + +Organize code into logical modules: + +- **Encapsulation:** Each module should encapsulate a specific functionality or feature. +- **Loose Coupling:** Modules should be loosely coupled to minimize dependencies and promote reusability. +- **Revealing Module Pattern:** Use the revealing module pattern to expose only the necessary functions and variables. + +javascript +// js/modules/userAuthentication.js +const userAuthentication = (function() { + let isAuthenticated = false; + + function login(username, password) { + // Authentication logic (e.g., AJAX call) + // ... + isAuthenticated = true; + } + + function logout() { + // Logout logic + //... + isAuthenticated = false; + } + + function isLoggedIn() { + return isAuthenticated; + } + + return { + login: login, + logout: logout, + isLoggedIn: isLoggedIn + }; +})(); + +// In app.js +$(document).ready(function() { + if (userAuthentication.isLoggedIn()) { + // Update UI for logged-in user + } +}); + + +### 1.4. Component Architecture Recommendations + +Break down the UI into reusable components: + +- **Modularity:** Components should be self-contained and independent. +- **Reusability:** Components should be designed for reuse across the application. +- **Maintainability:** Component-based architecture simplifies code updates and maintenance. + +javascript +// js/components/navigation.js +function Navigation(elementId, options) { + this.element = $('#' + elementId); + this.options = $.extend({}, { /* Default options */ }, options); + + this.init = function() { + // Component initialization + this.element.on('click', '.nav-item', this.handleNavigation.bind(this)); + }; + + this.handleNavigation = function(event) { + event.preventDefault(); + // Navigation logic + console.log('Navigating to:', $(event.target).attr('href')); + }; + + this.init(); +} + +// In app.js +$(document).ready(function() { + const nav = new Navigation('main-nav', { + // Custom options + }); +}); + + +### 1.5. Code Splitting Strategies + +Improve initial load time by splitting code into smaller chunks: + +- **On-Demand Loading:** Load modules or components only when they are needed. +- **Route-Based Splitting:** Load code specific to a particular route or page. +- **Conditional Loading:** Load code based on user interactions or device capabilities. + +javascript +// Example: Loading a module on button click +$('#load-module-button').on('click', function() { + $.getScript('js/modules/heavyModule.js', function() { + // Module loaded and executed + heavyModule.init(); + }); +}); + + +## 2. Common Patterns and Anti-patterns + +Employ established design patterns to improve code quality and avoid common mistakes. + +### 2.1. Design Patterns Specific to jQuery + +- **Module Pattern:** (See section 1.3). Encapsulates code and prevents global scope pollution. +- **Observer Pattern:** Facilitates communication between components without tight coupling. jQuery's event system is essentially an implementation of the observer pattern. +- **Facade Pattern:** Provides a simplified interface to a complex system (e.g., jQuery's `$`). + +### 2.2. Recommended Approaches for Common Tasks + +- **DOM Manipulation:** Minimize direct DOM manipulation. Use document fragments for batch updates. +- **Event Handling:** Implement event delegation for dynamically added elements. +- **AJAX:** Use `$.ajax()` for flexible AJAX requests. Use promises for asynchronous operations. + +### 2.3. Anti-patterns and Code Smells + +- **Global Variables:** Avoid using global variables to prevent naming conflicts and unexpected behavior. +- **Chaining Overuse:** Long chains can be difficult to read. Break them into smaller, more manageable chunks. +- **Excessive DOM Traversal:** Minimize DOM traversal by caching selectors. +- **Ignoring Performance:** Avoid complex selectors and inefficient DOM manipulation. +- **Inconsistent Coding Style:** Adhere to a consistent coding style to improve readability. +- **Mixing Concerns:** Separate HTML structure, CSS styling, and JavaScript behavior. +- **Deprecated Methods**: Avoid using deprecated methods. + +### 2.4. State Management Best Practices + +- **Keep it Simple:** For small applications, simple variables or data attributes may be sufficient. +- **Centralized Store:** For larger applications, consider a centralized state management solution (though jQuery is rarely used on its own for complex single page apps that require advanced state management). +- **Data Attributes:** Use data attributes (`data-*`) to store component-specific data. + +### 2.5. Error Handling Patterns + +- **Try-Catch Blocks:** Use `try-catch` blocks to handle exceptions gracefully. +- **AJAX Error Handling:** Implement error handling for AJAX requests using the `error` callback or `$.Deferred`'s `fail` method. +- **Global Error Handler:** Set up a global error handler to catch unhandled exceptions. + +javascript +$.ajax({ + url: 'api/data', + dataType: 'json', + success: function(data) { + // Process data + }, + error: function(jqXHR, textStatus, errorThrown) { + console.error('AJAX error:', textStatus, errorThrown); + // Display error message to the user + } +}); + + +## 3. Performance Considerations + +Optimize code for speed and efficiency. + +### 3.1. Optimization Techniques + +- **Selector Optimization:** Use ID selectors whenever possible. Be specific on the right-hand side of your selector and less specific on the left. Give your Selectors a Context. +- **Caching Selectors:** Cache jQuery selector returned objects in variables for reuse. +- **Minimize DOM Manipulations:** Batch updates, use document fragments, and detach elements before manipulation. +- **Event Delegation:** Use event delegation to reduce the number of event listeners. +- **Debouncing and Throttling:** Limit the rate at which a function is executed. + +### 3.2. Memory Management Considerations + +- **Avoid Memory Leaks:** Remove event listeners and data when elements are removed from the DOM. +- **Release References:** Set variables to `null` to release memory when they are no longer needed. +- **Garbage Collection:** Understand how JavaScript garbage collection works and avoid creating unnecessary objects. + +### 3.3. Rendering Optimization + +- **Minimize Reflows and Repaints:** Reduce the number of DOM manipulations that cause reflows and repaints. +- **Use CSS Transitions and Animations:** Use CSS transitions and animations instead of jQuery animations for better performance. +- **Hardware Acceleration:** Leverage hardware acceleration for smooth animations. + +### 3.4. Bundle Size Optimization + +- **Minification:** Use minification to reduce file sizes. +- **Gzip Compression:** Enable Gzip compression on the server to reduce the size of transferred files. +- **Code Splitting:** Split code into smaller chunks to improve initial load time (see section 1.5). + +### 3.5. Lazy Loading Strategies + +- **Lazy Load Images:** Load images only when they are visible in the viewport. +- **Lazy Load Modules:** Load modules or components only when they are needed (see section 1.5). +- **Use a Lazy Loading Library:** Consider using a library like `lozad.js` for easy lazy loading. + +## 4. Security Best Practices + +Protect your application against common vulnerabilities. + +### 4.1. Common Vulnerabilities and Prevention + +- **Cross-Site Scripting (XSS):** Sanitize user input to prevent XSS attacks. Use jQuery's `text()` method to set text content instead of `html()` to prevent injecting HTML code. +- **Cross-Site Request Forgery (CSRF):** Implement CSRF protection tokens. +- **SQL Injection:** Never directly use client-side data in database queries. Use parameterized queries or ORMs. +- **Open Redirects:** Validate and sanitize redirect URLs to prevent open redirects. +- **Dependency Vulnerabilities:** Keep jQuery and all dependencies up to date to patch security vulnerabilities. + +### 4.2. Input Validation Best Practices + +- **Client-Side Validation:** Implement client-side validation to provide immediate feedback to the user. +- **Server-Side Validation:** Always perform server-side validation to ensure data integrity. +- **Sanitize Input:** Sanitize user input to remove potentially harmful characters. + +### 4.3. Authentication and Authorization Patterns + +- **Use HTTPS:** Always use HTTPS to encrypt data in transit. +- **Secure Cookies:** Set the `secure` and `httpOnly` flags for cookies. +- **Authentication Tokens:** Use authentication tokens (e.g., JWT) for secure authentication. +- **Role-Based Access Control (RBAC):** Implement RBAC to restrict access to sensitive resources. + +### 4.4. Data Protection Strategies + +- **Encrypt Sensitive Data:** Encrypt sensitive data at rest and in transit. +- **Data Masking:** Mask sensitive data in the UI to prevent unauthorized access. +- **Regular Backups:** Perform regular backups of your data. + +### 4.5. Secure API Communication + +- **Use HTTPS:** Always use HTTPS for API communication. +- **API Keys:** Use API keys to authenticate requests. +- **Rate Limiting:** Implement rate limiting to prevent abuse. +- **Input Validation:** Validate all input data on the server-side. + +## 5. Testing Approaches + +Write tests to ensure code quality and prevent regressions. + +### 5.1. Unit Testing Strategies + +- **Test Individual Components:** Write unit tests for individual components and modules. +- **Use a Testing Framework:** Use a JavaScript testing framework like QUnit or Jasmine. +- **Mock Dependencies:** Mock dependencies to isolate the component being tested. + +### 5.2. Integration Testing Approaches + +- **Test Interactions:** Write integration tests to test the interactions between components. +- **Use a Testing Framework:** Use a testing framework like Mocha or Jest. +- **Test API Integrations:** Test the integration with external APIs. + +### 5.3. End-to-End Testing Recommendations + +- **Simulate User Interactions:** Write end-to-end tests to simulate user interactions. +- **Use a Testing Framework:** Use a testing framework like Cypress or Puppeteer. +- **Test Critical Paths:** Test the critical paths through the application. + +### 5.4. Test Organization + +- **Separate Test Files:** Create separate test files for each component or module. +- **Use Descriptive Names:** Use descriptive names for test cases. +- **Organize Tests:** Organize tests into logical groups. + +### 5.5. Mocking and Stubbing Techniques + +- **Mock AJAX Requests:** Mock AJAX requests to isolate the component being tested. +- **Stub Functions:** Stub functions to control their behavior. +- **Use a Mocking Library:** Use a mocking library like Sinon.js. + +javascript +// Example using QUnit and Sinon.js +QUnit.module('User Authentication Module', function(hooks) { + hooks.beforeEach(function() { + this.ajax = sinon.stub($, 'ajax'); + }); + + hooks.afterEach(function() { + this.ajax.restore(); + }); + + QUnit.test('login() should make an AJAX request', function(assert) { + this.ajax.resolves({ success: true }); + userAuthentication.login('testuser', 'password'); + assert.ok(this.ajax.calledOnce, 'AJAX request was made'); + }); +}); + + +## 6. Common Pitfalls and Gotchas + +Be aware of common mistakes and edge cases. + +### 6.1. Frequent Mistakes + +- **Not Caching Selectors:** Re-querying the DOM for the same element repeatedly. +- **Using Incorrect Scope (`this`):** Understanding the scope of `this` in event handlers and callbacks. +- **Ignoring Errors:** Not handling errors properly in AJAX requests and other asynchronous operations. +- **Overusing jQuery:** Using jQuery for tasks that can be done more efficiently with native JavaScript. + +### 6.2. Edge Cases + +- **Browser Compatibility:** Testing your code in different browsers and versions. +- **Mobile Devices:** Optimizing your code for mobile devices. +- **Accessibility:** Ensuring your code is accessible to users with disabilities. + +### 6.3. Version-Specific Issues + +- **Deprecated Methods:** Being aware of deprecated methods in newer versions of jQuery. +- **API Changes:** Understanding API changes between different versions of jQuery. + +### 6.4. Compatibility Concerns + +- **Conflicting Libraries:** Avoiding conflicts with other JavaScript libraries (e.g., Prototype, MooTools) using `$.noConflict()`. +- **Plugin Compatibility:** Ensuring that jQuery plugins are compatible with the version of jQuery being used. + +### 6.5. Debugging Strategies + +- **Use Browser Developer Tools:** Use browser developer tools to inspect elements, debug JavaScript code, and profile performance. +- **Console Logging:** Use `console.log()` to debug your code. +- **Breakpoints:** Set breakpoints in your code to pause execution and inspect variables. + +## 7. Tooling and Environment + +Use the right tools to improve productivity and code quality. + +### 7.1. Recommended Development Tools + +- **Text Editor/IDE:** Visual Studio Code, Sublime Text, Atom. +- **Browser:** Chrome, Firefox, Safari. +- **Debugging Tools:** Chrome DevTools, Firefox Developer Tools. +- **Build Tools:** Webpack, Parcel, Gulp. +- **Testing Frameworks:** QUnit, Jasmine, Mocha, Jest. + +### 7.2. Build Configuration + +- **Use a Build Tool:** Use a build tool like Webpack or Parcel to bundle and optimize your code. +- **Configure Loaders:** Configure loaders to handle different file types (e.g., JavaScript, CSS, images). +- **Optimize Output:** Optimize the output of the build process for production. + +### 7.3. Linting and Formatting + +- **Use a Linter:** Use a linter like ESLint to enforce coding standards and catch errors. +- **Configure Rules:** Configure linting rules to match your project's coding style. +- **Use a Formatter:** Use a code formatter like Prettier to automatically format your code. + +### 7.4. Deployment + +- **Minify and Gzip:** Minify and Gzip your code before deploying it. +- **Use a CDN:** Use a CDN to host static assets. +- **Cache Control Headers:** Set appropriate cache control headers. + +### 7.5. CI/CD Integration + +- **Automated Builds:** Automate the build process using a CI/CD tool like Jenkins, Travis CI, or GitHub Actions. +- **Automated Testing:** Automate the testing process using a CI/CD tool. +- **Automated Deployment:** Automate the deployment process using a CI/CD tool. + +By following these best practices, developers can write high-quality, performant, and secure jQuery code that is easy to maintain and scale. \ No newline at end of file diff --git a/.cursor/rules/junit.mdc b/.cursor/rules/junit.mdc new file mode 100644 index 000000000..b6e13474d --- /dev/null +++ b/.cursor/rules/junit.mdc @@ -0,0 +1,256 @@ +--- +description: Comprehensive guidelines and best practices for writing effective, maintainable, and performant JUnit tests in Java projects. This rule file covers code organization, patterns, performance, security, testing strategies, common pitfalls, and tooling. +globs: *Test.java +--- +# JUnit Best Practices: A Comprehensive Guide + +This document provides a comprehensive guide to JUnit best practices, covering code organization, patterns, performance considerations, security, testing approaches, common pitfalls, and tooling. Following these guidelines will help you write effective, maintainable, and performant unit tests. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure + +- **Standard Maven/Gradle Project Structure:** Maintain the standard Maven/Gradle project structure to separate test code from production code. + - Production code: `src/main/java/com/example/app/...` + - Test code: `src/test/java/com/example/app/...` +- **Mirror Production Structure:** Mirror the production code structure in the test directory to improve discoverability and maintainability. + - Example: + - Production: `src/main/java/com/example/app/service/UserService.java` + - Test: `src/test/java/com/example/app/service/UserServiceTest.java` +- **Test Resources:** Place test resources (e.g., configuration files, data files, mock response files) in the `src/test/resources` directory. + +### 1.2. File Naming Conventions + +- **Test Class Names:** Use the `*Test.java` or `*Tests.java` suffix for test class names to clearly identify them. + - Example: `UserServiceTest.java`, `StringUtilsTests.java` +- **Test Method Names:** Use descriptive test method names that clearly indicate the scenario and expected outcome. + - Follow a naming convention like `given_condition_when_action_then_expectedResult`. + - Example: `givenValidInput_whenCalculateSum_thenReturnsCorrectSum` + +### 1.3. Module Organization + +- **Modular Testing:** Organize tests by module or component to improve maintainability and reduce dependencies. +- **Test Suites:** Use JUnit test suites to group related tests and run them together. This is especially useful for integration tests. +- **Separate Test Configuration:** Keep test-specific configurations separate from production configurations. + +### 1.4. Component Architecture + +- **Testable Components:** Design components to be easily testable by following the SOLID principles (especially Dependency Inversion). +- **Loose Coupling:** Minimize dependencies between components to facilitate isolated unit testing. +- **Dependency Injection:** Use dependency injection to provide mock implementations of dependencies during testing. + +### 1.5. Code Splitting Strategies + +- **Small Test Methods:** Keep test methods small and focused on testing a single behavior. +- **Helper Methods:** Use helper methods to avoid code duplication in test setup and assertions. +- **Parameterized Tests:** Utilize JUnit's parameterized tests to test the same logic with different input values. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +- **Arrange-Act-Assert (AAA):** Structure tests using the AAA pattern to clearly separate setup (arrange), execution (act), and verification (assert) phases. +- **Test Fixture:** Use `@BeforeEach` and `@AfterEach` annotations to set up and tear down test data and resources. +- **Mock Object:** Utilize mocking frameworks (Mockito, EasyMock, PowerMock) to isolate the unit under test by simulating dependencies. +- **Page Object Model (POM):** Employ POM for UI testing to create reusable representations of web pages, making tests more readable and robust. + +### 2.2. Recommended Approaches + +- **Test-Driven Development (TDD):** Write tests before implementing the code to drive development and ensure testability. +- **Behavior-Driven Development (BDD):** Use BDD frameworks (e.g., Cucumber) to write tests in a natural language format, improving collaboration and understanding. +- **Code Coverage:** Aim for high code coverage (around 80%) to ensure most of the code is tested, but remember that coverage alone doesn't guarantee quality. +- **Continuous Integration:** Integrate unit tests into the CI/CD pipeline to automatically run tests on every code commit and provide immediate feedback. + +### 2.3. Anti-patterns and Code Smells + +- **Testing Implementation Details:** Avoid testing implementation details that might change, leading to brittle tests. Focus on testing behavior and outcomes. +- **Hard-coded Values:** Avoid hard-coding values in tests. Use constants or test data to make tests more maintainable. +- **Complex Test Logic:** Keep test logic simple and avoid complex calculations or conditional statements within tests. +- **Ignoring Edge Cases:** Don't ignore edge cases or boundary conditions. Ensure tests cover a wide range of inputs, including invalid or unexpected values. +- **Slow Tests:** Avoid slow tests that discourage developers from running them frequently. +- **Over-reliance on Mocks:** Mock judiciously; too many mocks can obscure the actual behavior and make tests less reliable. +- **Ignoring Test Failures:** Never ignore failing tests. Investigate and fix them promptly. + +### 2.4. State Management + +- **Isolated State:** Ensure each test has its own isolated state to avoid interference between tests. Use `@BeforeEach` to reset the state before each test. +- **Immutable Objects:** Prefer immutable objects to simplify state management and avoid unexpected side effects. +- **Stateless Components:** Design stateless components whenever possible to reduce the need for state management in tests. + +### 2.5. Error Handling + +- **Expected Exceptions:** Use `assertThrows` to verify that a method throws the expected exception under specific conditions. +- **Exception Messages:** Assert the exception message to ensure the correct error is being thrown with helpful context. +- **Graceful Degradation:** Test how the application handles errors and gracefully degrades when dependencies are unavailable. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +- **Fast Execution:** Keep tests fast to encourage frequent execution. Aim for millisecond execution times. +- **Parallel Execution:** Utilize JUnit's parallel execution feature to run tests concurrently and reduce overall test execution time. +- **Data Setup Optimization:** Optimize data setup by using in-memory databases or test data builders to avoid costly database interactions. +- **Mocking Optimization:** Optimize mocking by using lightweight mocks and avoiding unnecessary stubbing. + +### 3.2. Memory Management + +- **Resource Management:** Ensure tests properly release resources (e.g., database connections, file streams) to avoid memory leaks. +- **Large Data Sets:** Avoid loading large data sets into memory during testing. Use smaller, representative data sets. +- **Garbage Collection:** Be mindful of garbage collection. Create and destroy objects efficiently within your tests. + +### 3.3 Rendering Optimization (if applicable) + +- **Headless Testing:** If UI tests are involved, consider using headless browsers to minimize resource consumption. +- **Optimized Locators:** Use efficient locators (e.g., IDs, CSS selectors) to quickly find elements during UI testing. + +### 3.4 Bundle Size Optimization (if applicable) + +- **Code Splitting:** Utilize code splitting techniques to minimize the bundle size of the test code. Ensure only the necessary code is included in each test bundle. +- **Tree Shaking:** Enable tree shaking to remove unused code from test dependencies. + +### 3.5 Lazy Loading + +- **On-Demand Initialization:** Initialize test data and mocks only when they are needed to reduce startup time and resource consumption. +- **Lazy Evaluation:** Use lazy evaluation techniques to defer the execution of expensive operations until they are actually required. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities and Prevention + +- **Injection Attacks:** Prevent injection attacks (SQL injection, command injection) by using parameterized queries and input validation. +- **Cross-Site Scripting (XSS):** Protect against XSS attacks by encoding user inputs before rendering them in the UI. +- **Authentication and Authorization Bypass:** Ensure proper authentication and authorization mechanisms are in place to prevent unauthorized access. +- **Data Exposure:** Avoid storing sensitive data in plain text. Use encryption and secure storage mechanisms. + +### 4.2. Input Validation + +- **Validate All Inputs:** Validate all inputs to ensure they conform to the expected format and range. Use regular expressions, data type validation, and range checks. +- **Sanitize User Inputs:** Sanitize user inputs to remove potentially harmful characters or code. Use appropriate encoding and escaping techniques. + +### 4.3. Authentication and Authorization + +- **Secure Authentication:** Implement secure authentication using strong passwords, multi-factor authentication, and token-based authentication. +- **Role-Based Access Control:** Use role-based access control to restrict access to sensitive resources based on user roles. +- **Least Privilege Principle:** Grant users only the minimum privileges required to perform their tasks. + +### 4.4. Data Protection + +- **Encryption:** Use encryption to protect sensitive data at rest and in transit. Use strong encryption algorithms and secure key management practices. +- **Data Masking:** Mask sensitive data to prevent unauthorized access. Use appropriate masking techniques to protect personally identifiable information (PII). +- **Secure Storage:** Store sensitive data in secure storage locations with proper access controls. + +### 4.5. Secure API Communication + +- **HTTPS:** Use HTTPS to encrypt communication between the client and server. Obtain and install a valid SSL certificate. +- **API Authentication:** Implement API authentication using API keys, tokens, or OAuth 2.0 to verify the identity of the client. +- **Rate Limiting:** Implement rate limiting to prevent denial-of-service (DoS) attacks. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +- **Isolated Testing:** Test individual units of code in isolation from dependencies. +- **Mocking:** Use mocking frameworks (Mockito, EasyMock) to simulate dependencies. +- **Test Coverage:** Aim for high code coverage to ensure most of the code is tested. +- **Edge Cases:** Test edge cases and boundary conditions to ensure code handles unexpected inputs correctly. +- **Assertion Libraries:** Leverage assertion libraries (AssertJ, Hamcrest) for more expressive and readable assertions. + +### 5.2. Integration Testing + +- **Component Interactions:** Test interactions between different components or modules. +- **External Systems:** Test integration with external systems (databases, APIs) using integration tests. +- **Real Dependencies:** Use real dependencies or in-memory substitutes for integration tests. +- **Data Consistency:** Verify data consistency and integrity across different components. + +### 5.3. End-to-End Testing + +- **Full System Testing:** Test the entire system from end to end to ensure it functions correctly. +- **UI Testing:** Use UI testing frameworks (Selenium, Cypress) to test the user interface. +- **User Scenarios:** Simulate real user scenarios to verify the system meets user requirements. +- **Environment Parity:** Test in an environment that closely resembles the production environment. + +### 5.4. Test Organization + +- **Test Suites:** Organize tests into suites based on functionality or component. +- **Test Categories:** Use JUnit categories to group tests and run them selectively. +- **Test Runners:** Use test runners to execute tests and generate reports. + +### 5.5. Mocking and Stubbing + +- **Mocking Frameworks:** Use mocking frameworks (Mockito, EasyMock) to create mock objects that simulate dependencies. +- **Stubbing:** Stub methods to return specific values or throw exceptions during testing. +- **Verification:** Verify that methods are called with the expected arguments during testing. +- **Avoid Over-Mocking:** Mock only the necessary dependencies to avoid over-complicating tests. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +- **Ignoring Test Failures:** Ignoring failing tests and not fixing them promptly. +- **Writing Brittle Tests:** Writing tests that are tightly coupled to implementation details. +- **Not Testing Edge Cases:** Neglecting to test edge cases and boundary conditions. +- **Writing Slow Tests:** Writing tests that take a long time to run, discouraging frequent execution. +- **Not Using Mocking Frameworks:** Failing to use mocking frameworks to isolate units under test. + +### 6.2. Edge Cases + +- **Null Values:** Handle null values gracefully in tests and production code. +- **Empty Collections:** Test how code handles empty collections. +- **Invalid Inputs:** Test how code handles invalid inputs (e.g., negative numbers, invalid dates). +- **Boundary Conditions:** Test boundary conditions to ensure code behaves correctly at the limits of its input range. + +### 6.3. Version-Specific Issues + +- **API Changes:** Be aware of API changes in different JUnit versions and update tests accordingly. +- **Dependency Conflicts:** Resolve dependency conflicts between JUnit and other libraries. +- **Compatibility Issues:** Ensure compatibility between JUnit and the Java version being used. + +### 6.4. Compatibility Concerns + +- **Third-Party Libraries:** Ensure compatibility between JUnit and third-party libraries being used. +- **IDE Support:** Choose an IDE that provides good support for JUnit testing. +- **Build Tools:** Integrate JUnit with build tools (Maven, Gradle) for automated testing. + +### 6.5. Debugging Strategies + +- **Debugging Mode:** Run tests in debugging mode to step through the code and inspect variables. +- **Logging:** Add logging statements to tests to track execution flow and identify issues. +- **Remote Debugging:** Use remote debugging to debug tests running on remote servers. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +- **Integrated Development Environment (IDE):** IntelliJ IDEA, Eclipse, NetBeans +- **Build Tools:** Maven, Gradle +- **Mocking Frameworks:** Mockito, EasyMock +- **Assertion Libraries:** AssertJ, Hamcrest +- **Code Coverage Tools:** JaCoCo, Cobertura +- **Static Analysis Tools:** SonarQube, FindBugs + +### 7.2. Build Configuration + +- **Dependency Management:** Use Maven or Gradle to manage JUnit dependencies. +- **Test Configuration:** Configure test execution options (e.g., parallel execution, code coverage) in the build file. +- **Plugin Integration:** Integrate JUnit plugins with build tools for automated testing and reporting. + +### 7.3. Linting and Formatting + +- **Code Style:** Follow a consistent code style (e.g., Google Java Style, Checkstyle) to improve code readability and maintainability. +- **Linting:** Use linting tools (e.g., Checkstyle, PMD) to enforce code style rules and identify potential issues. +- **Formatting:** Use code formatting tools (e.g., IntelliJ IDEA formatter, Eclipse formatter) to automatically format code according to the configured style. + +### 7.4. Deployment + +- **Test Environment:** Deploy applications to a test environment that closely resembles the production environment. +- **Automated Testing:** Run automated tests in the deployment pipeline to verify the application functions correctly. +- **Rollback Strategy:** Implement a rollback strategy to quickly revert to a previous version if issues are detected after deployment. + +### 7.5. CI/CD Integration + +- **Continuous Integration:** Integrate JUnit tests into the CI/CD pipeline to automatically run tests on every code commit. +- **Automated Testing:** Automate the execution of JUnit tests as part of the build process. +- **Reporting:** Generate test reports and publish them to the CI/CD system. +- **Failure Notifications:** Configure failure notifications to alert developers when tests fail. + +By following these best practices, you can write more effective, maintainable, and performant JUnit tests, leading to higher-quality software and faster development cycles. \ No newline at end of file diff --git a/.cursor/rules/keras.mdc b/.cursor/rules/keras.mdc new file mode 100644 index 000000000..191913fd1 --- /dev/null +++ b/.cursor/rules/keras.mdc @@ -0,0 +1,323 @@ +--- +description: This rule enforces Keras library best practices, focusing on code clarity, modularity, performance optimization, and security considerations. It provides actionable guidance for developers to improve the quality and maintainability of Keras-based machine learning projects. +globs: *.py +--- +# Keras Development Best Practices + +This document outlines best practices for developing Keras applications. It covers various aspects of software engineering, including code organization, common patterns, performance, security, testing, and tooling. + +Library Information: +- Name: keras +- Tags: ai, ml, machine-learning, python, deep-learning + +## 1. Code Organization and Structure + +### 1.1. Directory Structure + +Adopt a well-defined directory structure to enhance maintainability and collaboration. + + +project_root/ +├── data/ # Contains datasets (raw, processed) +├── models/ # Saved models (weights, architectures) +├── src/ # Source code +│ ├── layers/ # Custom Keras layers +│ ├── models/ # Model definitions +│ ├── utils/ # Utility functions +│ ├── callbacks/ # Custom Keras Callbacks +│ ├── preprocessing/ # Data preprocessing scripts +│ └── __init__.py # Makes 'src' a Python package +├── notebooks/ # Jupyter notebooks for experimentation +├── tests/ # Unit and integration tests +├── requirements.txt # Project dependencies +├── README.md # Project overview +└── .gitignore # Specifies intentionally untracked files that Git should ignore + + +### 1.2. File Naming Conventions + +Use descriptive and consistent file names. + +- `model_name.py`: For defining Keras models. +- `layer_name.py`: For custom Keras layers. +- `utils.py`: For utility functions. +- `data_preprocessing.py`: For data preprocessing scripts. +- `training_script.py`: Main training script. + +### 1.3. Module Organization + +- **Single Responsibility Principle:** Each module should have a clear and specific purpose. +- **Loose Coupling:** Minimize dependencies between modules. +- **High Cohesion:** Keep related functions and classes within the same module. + +python +# src/models/my_model.py + +import keras +from keras import layers + +def create_model(input_shape, num_classes): + inputs = keras.Input(shape=input_shape) + x = layers.Conv2D(32, (3, 3), activation='relu')(inputs) + x = layers.MaxPooling2D((2, 2))(x) + x = layers.Flatten()(x) + outputs = layers.Dense(num_classes, activation='softmax')(x) + model = keras.Model(inputs, outputs) + return model + + +### 1.4. Component Architecture + +- **Layers:** Encapsulate reusable blocks of computation (e.g., custom convolutional layers, attention mechanisms). +- **Models:** Define the overall architecture by combining layers. +- **Callbacks:** Implement custom training behaviors (e.g., early stopping, learning rate scheduling). +- **Preprocessing:** Separate data loading, cleaning, and transformation logic. + +### 1.5. Code Splitting + +- **Functions:** Break down complex logic into smaller, well-named functions. +- **Classes:** Use classes to represent stateful components (e.g., custom layers with trainable parameters). +- **Packages:** Organize modules into packages for larger projects. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +- **Functional API:** Use the Keras Functional API for building complex, multi-input/output models. + + python + input_tensor = keras.Input(shape=(784,)) + hidden_layer = layers.Dense(units=64, activation='relu')(input_tensor) + output_tensor = layers.Dense(units=10, activation='softmax')(hidden_layer) + model = keras.Model(inputs=input_tensor, outputs=output_tensor) + + +- **Subclassing:** Subclass `keras.Model` or `keras.layers.Layer` for maximum customization. + + python + class MyLayer(layers.Layer): + def __init__(self, units=32, **kwargs): + super(MyLayer, self).__init__(**kwargs) + self.units = units + + def build(self, input_shape): + self.w = self.add_weight(shape=(input_shape[-1], self.units), + initializer='random_normal', + trainable=True) + self.b = self.add_weight(shape=(self.units,), + initializer='zeros', + trainable=True) + + def call(self, inputs): + return keras.activations.relu(tf.matmul(inputs, self.w) + self.b) + + +- **Callbacks:** Implement custom training behaviors (e.g., custom logging, model checkpointing). + + python + class CustomCallback(keras.callbacks.Callback): + def on_epoch_end(self, epoch, logs=None): + print(f'Epoch {epoch}: Loss = {logs['loss']}') + + +### 2.2. Recommended Approaches + +- **Data Input Pipelines:** Use `tf.data.Dataset` for efficient data loading and preprocessing. +- **Model Checkpointing:** Save model weights during training to prevent data loss and allow for resuming training. +- **Early Stopping:** Monitor validation loss and stop training when it plateaus to prevent overfitting. +- **Learning Rate Scheduling:** Adjust the learning rate during training to improve convergence. + +### 2.3. Anti-Patterns + +- **Hardcoding:** Avoid hardcoding values directly into your code. Use variables and configuration files instead. +- **Global Variables:** Minimize the use of global variables to prevent namespace pollution and unexpected side effects. +- **Over-Engineering:** Don't overcomplicate your code with unnecessary abstractions or complex patterns. +- **Ignoring Warnings:** Pay attention to warnings and deprecation messages, as they often indicate potential problems. +- **Training on the entire dataset without validation:** Always split your data into training, validation and testing sets to avoid overfitting. + +### 2.4. State Management + +- **Stateless Operations:** Prefer stateless operations whenever possible to simplify testing and debugging. +- **Model Weights:** Store model weights separately from the model architecture. +- **Configuration Files:** Use configuration files (e.g., JSON, YAML) to manage hyperparameters and other settings. + +### 2.5. Error Handling + +- **Exception Handling:** Use `try...except` blocks to handle potential exceptions gracefully. +- **Logging:** Log errors and warnings to help diagnose problems. +- **Validation:** Validate input data to prevent unexpected errors. +- **Assertions:** Use `assert` statements to check for conditions that should always be true. + +python +try: + model = keras.models.load_model('my_model.h5') +except FileNotFoundError: + logging.error('Model file not found.') + raise + + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +- **GPU Acceleration:** Utilize GPUs for faster training and inference. +- **Data Preprocessing:** Optimize data preprocessing pipelines to reduce overhead. +- **Batch Size:** Adjust the batch size to maximize GPU utilization. +- **Model Pruning:** Remove unnecessary weights from the model to reduce its size and improve its speed. +- **Quantization:** Reduce the precision of model weights to reduce memory consumption and improve inference speed. +- **Mixed Precision Training:** Use `tf.keras.mixed_precision.Policy` to enable mixed precision training for faster training on modern GPUs. + +### 3.2. Memory Management + +- **Garbage Collection:** Be mindful of memory leaks and use garbage collection to reclaim unused memory. +- **Data Types:** Use appropriate data types to minimize memory consumption (e.g., `tf.float16` instead of `tf.float32`). +- **Generators:** Use generators to load data in batches, reducing memory usage. + +### 3.3. Rendering Optimization (If applicable) + +Not directly applicable to Keras itself, but relevant when visualizing model outputs or training progress. Use libraries like `matplotlib` or `seaborn` efficiently and consider downsampling large datasets before plotting. + +### 3.4. Bundle Size Optimization + +- **Model Pruning and Quantization:** as above. +- **Selectively Import Keras Modules**: Only import the specific Keras modules needed to reduce the overall bundle size, e.g., `from keras.layers import Dense, Conv2D` instead of `import keras.layers`. + +### 3.5. Lazy Loading + +- **Lazy Initialization:** Defer the initialization of resources until they are actually needed. +- **Data Loading:** Load data on demand rather than loading the entire dataset into memory. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities + +- **Adversarial Attacks:** Protect against adversarial attacks that can fool models into making incorrect predictions. +- **Data Poisoning:** Ensure the integrity of training data to prevent data poisoning attacks. +- **Model Extraction:** Protect against model extraction attacks that can steal intellectual property. + +### 4.2. Input Validation + +- **Sanitize Input:** Sanitize input data to prevent injection attacks. +- **Validate Input:** Validate input data to ensure that it conforms to the expected format and range. + +python +def predict(model, input_data): + if not isinstance(input_data, np.ndarray): + raise TypeError('Input data must be a NumPy array.') + if input_data.shape != (1, 784): + raise ValueError('Input data must have shape (1, 784).') + return model.predict(input_data) + + +### 4.3. Authentication and Authorization + +- **Secure API:** Implement secure API communication using HTTPS. +- **Authentication:** Require authentication for access to sensitive data and functionality. +- **Authorization:** Enforce authorization policies to control access to resources. + +### 4.4. Data Protection + +- **Encryption:** Encrypt sensitive data at rest and in transit. +- **Anonymization:** Anonymize data to protect privacy. +- **Data Governance:** Implement data governance policies to ensure data quality and security. + +### 4.5. Secure API Communication + +- **HTTPS:** Use HTTPS for all API communication to encrypt data in transit. +- **API Keys:** Use API keys to authenticate requests. +- **Rate Limiting:** Implement rate limiting to prevent denial-of-service attacks. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +- **Test-Driven Development:** Write unit tests before writing code to ensure that the code meets the requirements. +- **Test Cases:** Create test cases for different scenarios, including edge cases and error conditions. +- **Assertions:** Use assertions to verify that the code behaves as expected. + +### 5.2. Integration Testing + +- **Component Interaction:** Test the interaction between different components of the application. +- **Data Flow:** Test the flow of data through the application. +- **System Integration:** Test the integration of the application with other systems. + +### 5.3. End-to-End Testing + +- **User Interface:** Test the user interface to ensure that it is functional and user-friendly. +- **Workflow:** Test the entire workflow from start to finish. +- **Real-World Scenarios:** Test the application in real-world scenarios to ensure that it meets the needs of the users. + +### 5.4. Test Organization + +- **Test Directory:** Create a dedicated `tests` directory to store test files. +- **Test Modules:** Organize tests into modules based on the components they test. +- **Test Naming Conventions:** Use clear and consistent naming conventions for test files and functions. + +### 5.5. Mocking and Stubbing + +- **Mock Objects:** Use mock objects to simulate the behavior of external dependencies. +- **Stub Functions:** Use stub functions to replace complex or time-consuming operations with simple, predictable results. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +- **Incorrect Input Shapes:** Ensure that the input shapes match the expected dimensions. +- **Data Type Mismatches:** Use consistent data types throughout the application. +- **Gradient Vanishing/Exploding:** Use appropriate activation functions and weight initialization techniques to prevent gradient problems. +- **Overfitting:** Use regularization techniques (e.g., dropout, L1/L2 regularization) to prevent overfitting. + +### 6.2. Edge Cases + +- **Empty Datasets:** Handle empty datasets gracefully. +- **Missing Values:** Handle missing values appropriately (e.g., imputation, deletion). +- **Outliers:** Identify and handle outliers in the data. + +### 6.3. Version-Specific Issues + +- **API Changes:** Be aware of API changes between different versions of Keras and TensorFlow. +- **Compatibility:** Ensure that your code is compatible with the versions of Keras and TensorFlow that you are using. + +### 6.4. Compatibility Concerns + +- **TensorFlow Compatibility:** Verify the compatibility between Keras and TensorFlow versions. Keras 3 can run on TensorFlow 2.16 onwards but there can be backwards compatibility issues. +- **Hardware Compatibility:** Ensure compatibility with different hardware platforms (e.g., CPU, GPU, TPU). + +### 6.5. Debugging Strategies + +- **Logging:** Use logging to track the execution of the code and identify potential problems. +- **Debugging Tools:** Use debugging tools (e.g., `pdb`, `TensorBoard`) to inspect the state of the application. +- **Print Statements:** Use print statements to display intermediate values and debug the code. +- **TensorBoard:** Use TensorBoard to visualize the model architecture, training progress, and performance metrics. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +- **IDE:** Use an IDE (e.g., VS Code, PyCharm) with Keras and TensorFlow support. +- **Virtual Environment:** Use a virtual environment (e.g., `venv`, `conda`) to isolate project dependencies. +- **Jupyter Notebook:** Use Jupyter notebooks for experimentation and prototyping. + +### 7.2. Build Configuration + +- **Requirements File:** Use a `requirements.txt` file to specify project dependencies. +- **Setup Script:** Use a `setup.py` script to define the project metadata and installation instructions. + +### 7.3. Linting and Formatting + +- **PEP 8:** Adhere to PEP 8 style guidelines for Python code. +- **Linters:** Use linters (e.g., `flake8`, `pylint`) to enforce code style and identify potential problems. +- **Formatters:** Use formatters (e.g., `black`, `autopep8`) to automatically format code. + +### 7.4. Deployment + +- **Containerization:** Use containerization (e.g., Docker) to package the application and its dependencies. +- **Cloud Platforms:** Deploy the application to a cloud platform (e.g., AWS, Google Cloud, Azure). +- **Serving Frameworks:** Use serving frameworks (e.g., TensorFlow Serving, KServe) to deploy models for inference. + +### 7.5. CI/CD Integration + +- **Continuous Integration:** Automate the build, test, and integration process using CI tools (e.g., Jenkins, Travis CI, GitHub Actions). +- **Continuous Deployment:** Automate the deployment process using CD tools (e.g., AWS CodePipeline, Google Cloud Build, Azure DevOps). + +This comprehensive guide provides a strong foundation for developing high-quality, maintainable, and secure Keras applications. By following these best practices, developers can improve their productivity, reduce the risk of errors, and build robust machine learning systems. \ No newline at end of file diff --git a/.cursor/rules/kivy.mdc b/.cursor/rules/kivy.mdc new file mode 100644 index 000000000..90065c326 --- /dev/null +++ b/.cursor/rules/kivy.mdc @@ -0,0 +1,347 @@ +--- +description: This rule file outlines best practices for Kivy UI development, including code organization, performance, security, and testing. Adhering to these guidelines ensures maintainable, efficient, and secure Kivy applications. +globs: *.py,*.kv +--- +# Kivy Development Best Practices + +This document provides a comprehensive guide to Kivy development best practices, covering various aspects from code organization to security and testing. + +## 1. Code Organization and Structure + +Proper code organization is crucial for maintainability, scalability, and collaboration in Kivy projects. + +### 1.1 Directory Structure + +A well-defined directory structure helps organize your Kivy project. Here's a suggested structure: + + +my_kivy_app/ +├── app.py # Main application file +├── main.kv # Root KV file +├── screens/ # Directory for screen definitions +│ ├── __init__.py # Initialize screens package +│ ├── main_screen.py +│ ├── main_screen.kv +│ ├── settings_screen.py +│ └── settings_screen.kv +├── components/ # Directory for reusable UI components +│ ├── __init__.py # Initialize components package +│ ├── custom_button.py +│ └── custom_button.kv +├── utils/ # Utility modules +│ ├── __init__.py +│ └── helpers.py +├── data/ # Data files (JSON, images, fonts, etc.) +│ ├── images/ +│ ├── fonts/ +│ └── settings.json +├── tests/ # Unit and integration tests +│ ├── __init__.py +│ ├── test_main_screen.py +│ └── test_helpers.py +├── .gitignore # Git ignore file +├── README.md # Project README +└── requirements.txt # Project dependencies + + +* `app.py`: The main application file where you define your `App` subclass and run the application. +* `main.kv`: The root KV file that typically handles overall layout structure and screen management. It is loaded automatically if named appropriately (e.g., `myapp.kv` for a `MyApp` class). +* `screens/`: Contains individual screen definitions. Each screen can have a Python file for logic and a KV file for layout. +* `components/`: Holds reusable UI components, like custom buttons, labels, or layouts. This promotes code reuse and consistency. +* `utils/`: Includes helper functions and utility modules that provide common functionalities across the application. +* `data/`: Stores application data like images, fonts, and configuration files. +* `tests/`: Contains unit and integration tests to ensure code quality and prevent regressions. +* `.gitignore`: Specifies intentionally untracked files that Git should ignore. +* `README.md`: Provides a project overview, setup instructions, and other relevant information. +* `requirements.txt`: Lists the Python packages required to run the application. Use `pip freeze > requirements.txt` to generate this. + +### 1.2 File Naming Conventions + +* **Python files:** Use lowercase with underscores (snake_case), e.g., `main_screen.py`, `custom_button.py`. +* **KV files:** Use the same name as the corresponding Python file, but with the `.kv` extension, e.g., `main_screen.kv`, `custom_button.kv`. +* **Class names:** Use CamelCase, e.g., `MainScreen`, `CustomButton`. +* **Variables and functions:** Use snake_case, e.g., `screen_width`, `calculate_layout()`. + +Consistent naming conventions improve code readability and maintainability. + +### 1.3 Module Organization + +* **Separate concerns:** Divide your application into logical modules based on functionality. For example, a module for data access, a module for UI components, and a module for business logic. +* **Use packages:** Group related modules into packages using `__init__.py` files. This helps organize your project and prevent naming conflicts. +* **Keep modules small:** Avoid creating overly large modules. Smaller modules are easier to understand, test, and reuse. +* **Use relative imports:** Within a package, use relative imports (e.g., `from . import helpers`) to refer to other modules in the same package. This makes your code more portable and less prone to errors. + +### 1.4 Component Architecture + +* **Create reusable components:** Design your UI with reusable components. This promotes code reuse, reduces redundancy, and makes it easier to maintain your application. Kivy's KV language and custom widget creation are ideal for this. +* **Use custom widgets:** Create custom widgets by subclassing Kivy's existing widgets or combining multiple widgets. This allows you to encapsulate complex UI elements and behavior into reusable components. +* **Follow the DRY principle (Don't Repeat Yourself):** Abstract common functionalities into reusable components or utility functions. This reduces code duplication and makes your code more maintainable. + +### 1.5 Code Splitting + +* **Break down large screens:** If a screen becomes too complex, break it down into smaller, manageable sub-components. +* **Lazy loading:** Load parts of the UI or data only when they are needed. This can improve startup time and reduce memory usage. +* **Dynamic loading of KV files:** Load KV files dynamically using `Builder.load_file()` when a screen or component is needed. This allows you to split your UI definition into multiple files and load them on demand. + +## 2. Common Patterns and Anti-patterns + +Understanding common design patterns and anti-patterns helps you write better Kivy code. + +### 2.1 Design Patterns + +* **Model-View-Controller (MVC) / Model-View-Presenter (MVP) / Model-View-ViewModel (MVVM):** Consider using these patterns to separate data (model), UI (view), and logic (controller/presenter/viewmodel). Kivy does not enforce a specific architecture, but these patterns can help organize your code and improve testability. MVVM is often considered more suitable due to Kivy's declarative nature. +* **Observer:** Kivy's properties and event dispatching mechanism are based on the Observer pattern. Use properties to observe changes in data and trigger UI updates accordingly. +* **Factory:** Use the Factory pattern to create different types of widgets or objects based on certain conditions or data. This can be useful for dynamically generating UI elements. +* **Singleton:** While often debated, Singletons can be useful for global configuration or access to shared resources. Use with caution and consider alternatives like dependency injection. + +### 2.2 Recommended Approaches + +* **Use KV language for UI definition:** Separate your UI definition from your Python code using the KV language. This improves code readability and maintainability. +* **Bind properties to UI elements:** Use Kivy's property binding mechanism to automatically update UI elements when data changes. This simplifies UI updates and reduces boilerplate code. +* **Use events for user interaction:** Use Kivy's event system to handle user interactions, such as button clicks and text input. This allows you to react to user actions and update the UI accordingly. +* **Asynchronous operations:** Use asynchronous operations (e.g., `Clock.schedule_once()`, `Clock.schedule_interval()`, or `async/await`) for long-running tasks to avoid blocking the UI thread. This ensures that your application remains responsive. +* **Data binding with properties:** Leverage Kivy properties (StringProperty, NumericProperty, ObjectProperty, etc.) for data binding between UI elements and your application's data model. This enables automatic updates of UI elements when the underlying data changes, and vice versa. + +### 2.3 Anti-patterns and Code Smells + +* **Tight coupling:** Avoid tight coupling between UI elements and business logic. Separate your UI definition from your logic using the KV language and appropriate design patterns. +* **God object:** Avoid creating overly large and complex classes that handle too many responsibilities. Break down large classes into smaller, more manageable classes. +* **Code duplication:** Avoid duplicating code. Abstract common functionalities into reusable components or utility functions. +* **Ignoring errors:** Don't ignore errors. Handle errors gracefully and provide meaningful feedback to the user. +* **Blocking the UI thread:** Avoid performing long-running tasks on the UI thread. Use asynchronous operations to prevent the UI from freezing. +* **Excessive use of `eval()` or `exec()`:** Avoid using `eval()` or `exec()` to execute arbitrary code, as this can introduce security vulnerabilities. Prefer safer alternatives like data binding or function calls. +* **Over-reliance on global variables:** Minimize the use of global variables, as they can make your code harder to understand and maintain. Prefer passing data explicitly between components. + +### 2.4 State Management + +* **Kivy Properties:** For simple state management within a widget or screen, Kivy Properties are often sufficient. They offer automatic notification when their value changes, making them easy to bind to UI elements. +* **ScreenManager:** The `ScreenManager` widget can store state associated with each screen. This is useful for managing the overall application state and passing data between screens. +* **External state management libraries:** For more complex state management requirements, consider using external libraries like: `RxPY`, `KivyMD`, or even integrating with more comprehensive frameworks like Flask for the backend. +* **Minimize mutable state:** Prefer immutable data structures to reduce the risk of unexpected side effects and improve code predictability. +* **Centralized State:** Implement a centralized state management system to ensure data consistency across different parts of the application. + +### 2.5 Error Handling + +* **Use `try...except` blocks:** Use `try...except` blocks to handle potential exceptions. Provide specific exception handling for different types of errors. +* **Log errors:** Log errors to a file or console for debugging purposes. Include relevant information, such as the error message, stack trace, and timestamp. +* **Provide user-friendly error messages:** Display user-friendly error messages to the user. Avoid displaying technical details that the user may not understand. +* **Handle unhandled exceptions:** Implement a global exception handler to catch unhandled exceptions and prevent the application from crashing. Log the error and display a generic error message to the user. +* **Retry mechanism:** Implement a retry mechanism for operations that may fail due to transient errors, such as network connectivity issues. + +## 3. Performance Considerations + +Optimizing performance is crucial for creating smooth and responsive Kivy applications. + +### 3.1 Optimization Techniques + +* **Use the right layout:** Choose the most efficient layout for your UI. `RelativeLayout` and `FloatLayout` can be more expensive than `BoxLayout` or `GridLayout`. +* **Minimize widget count:** Reduce the number of widgets in your UI. Consider using custom drawing or more efficient layouts to achieve the same visual effect with fewer widgets. +* **Use textures efficiently:** Load images and textures at the correct size. Avoid scaling large images down in the UI, as this can be inefficient. Use texture atlases to combine multiple small images into a single texture, reducing the number of texture swaps. +* **Use `AsyncImage` for loading images:** Load images asynchronously using `AsyncImage` to avoid blocking the UI thread. +* **Optimize drawing:** Use Kivy's drawing instructions efficiently. Batch drawing operations to reduce the number of OpenGL calls. +* **Freeze layouts:** Use `Widget.size_hint` with fixed values to help Kivy optimize layout calculations. +* **Use appropriate property types:** Use specific Kivy property types (StringProperty, NumericProperty, etc.) instead of generic ObjectProperty to improve performance. +* **Avoid complex calculations in property callbacks:** Keep property callback functions short and simple. Move complex calculations to separate functions and call them from the callbacks. + +### 3.2 Memory Management + +* **Release unused resources:** Release unused resources, such as images and textures, when they are no longer needed. Use `Widget.clear()` to remove all children from a widget and release their resources. +* **Use weak references:** Use weak references to avoid creating circular dependencies that can prevent objects from being garbage collected. +* **Avoid creating large lists or dictionaries:** If you need to store large amounts of data, consider using more efficient data structures, such as NumPy arrays or SQLite databases. +* **Use generators:** Use generators to process large data sets lazily, reducing memory consumption. +* **Profile your application:** Use a memory profiler to identify memory leaks and optimize memory usage. + +### 3.3 Rendering Optimization + +* **Use the correct OpenGL context:** Ensure that you are using the correct OpenGL context for your platform. Kivy supports both OpenGL and OpenGL ES. +* **Reduce overdraw:** Reduce overdraw by minimizing the number of overlapping UI elements. +* **Use shaders:** Use shaders to perform complex rendering effects efficiently. +* **Optimize texture loading:** Optimize texture loading by using compressed textures and mipmaps. +* **Use the `kivy.graphics` module efficiently:** Minimize the number of drawing calls and state changes. Batch drawing operations together. + +### 3.4 Bundle Size Optimization + +* **Remove unused code:** Remove unused code from your application to reduce the bundle size. +* **Compress images:** Compress images to reduce their file size. Use lossless compression for images that require high quality and lossy compression for images where some quality loss is acceptable. +* **Use efficient audio formats:** Use efficient audio formats, such as MP3 or Opus, to reduce the size of your audio files. +* **Obfuscate your code:** Obfuscate your code to make it harder to reverse engineer and reduce the bundle size. +* **Use code minification:** Minify your code to remove unnecessary whitespace and comments, reducing the bundle size. +* **Use code splitting:** Split your code into multiple modules and load them on demand to reduce the initial bundle size. + +### 3.5 Lazy Loading + +* **Load screens on demand:** Load screens only when they are needed. This can improve startup time and reduce memory usage. Use `ScreenManager.add_widget()` and `ScreenManager.remove_widget()` to manage screens dynamically. +* **Load data on demand:** Load data only when it is needed. Use asynchronous operations to load data in the background without blocking the UI thread. +* **Load images on demand:** Load images only when they are needed. Use `AsyncImage` to load images asynchronously. +* **Virtualization for lists:** When displaying long lists, use virtualization techniques to only render the visible items. This can significantly improve performance and reduce memory usage. + +## 4. Security Best Practices + +Security should be a primary concern when developing Kivy applications, especially those that handle sensitive data or interact with external services. + +### 4.1 Common Vulnerabilities + +* **Code injection:** Avoid using `eval()` or `exec()` to execute arbitrary code, as this can allow attackers to inject malicious code into your application. +* **Cross-site scripting (XSS):** Be careful when displaying user-generated content in your UI. Sanitize the content to prevent attackers from injecting malicious scripts. +* **SQL injection:** When interacting with databases, use parameterized queries or object-relational mappers (ORMs) to prevent SQL injection attacks. +* **Man-in-the-middle (MITM) attacks:** Use HTTPS to encrypt communication between your application and external services, preventing attackers from intercepting sensitive data. +* **Data leakage:** Be careful not to leak sensitive data in your application's logs or error messages. Disable debug mode in production builds. +* **Insecure storage:** Avoid storing sensitive data in plain text. Encrypt sensitive data before storing it locally or in the cloud. + +### 4.2 Input Validation + +* **Validate all user input:** Validate all user input to ensure that it conforms to the expected format and range. Use regular expressions or custom validation functions to validate input. +* **Sanitize user input:** Sanitize user input to remove potentially harmful characters or code. Use Kivy's built-in functions or external libraries to sanitize input. +* **Limit input length:** Limit the length of input fields to prevent buffer overflows and other security vulnerabilities. +* **Escape output:** Escape output to prevent cross-site scripting (XSS) attacks. Use Kivy's built-in functions or external libraries to escape output. +* **Use whitelisting:** Use whitelisting to allow only specific characters or patterns in input fields. This can be more secure than blacklisting, which attempts to block specific harmful characters or patterns. + +### 4.3 Authentication and Authorization + +* **Use secure authentication protocols:** Use secure authentication protocols, such as OAuth 2.0 or JWT, to authenticate users. Avoid storing passwords in plain text. +* **Implement multi-factor authentication (MFA):** Implement multi-factor authentication (MFA) to add an extra layer of security to user accounts. +* **Use role-based access control (RBAC):** Use role-based access control (RBAC) to restrict access to sensitive data and functionalities based on user roles. +* **Regularly audit your authentication and authorization system:** Regularly audit your authentication and authorization system to identify and fix any vulnerabilities. +* **Use password hashing:** Always hash passwords using a strong hashing algorithm (e.g., bcrypt or Argon2) before storing them. Never store passwords in plain text. + +### 4.4 Data Protection + +* **Encrypt sensitive data:** Encrypt sensitive data before storing it locally or in the cloud. Use strong encryption algorithms, such as AES. +* **Use secure storage:** Store sensitive data in secure storage locations, such as the Android Keystore or iOS Keychain. +* **Protect data in transit:** Use HTTPS to encrypt communication between your application and external services. +* **Comply with data privacy regulations:** Comply with data privacy regulations, such as GDPR and CCPA. +* **Implement data masking:** Mask sensitive data when displaying it in the UI or logs. This can prevent unauthorized access to sensitive information. + +### 4.5 Secure API Communication + +* **Use HTTPS:** Always use HTTPS to communicate with APIs. This encrypts the data transmitted between your application and the API server, preventing eavesdropping. +* **Validate API responses:** Validate API responses to ensure that they are in the expected format and do not contain any malicious data. +* **Use API keys or tokens:** Use API keys or tokens to authenticate your application with the API server. Store API keys and tokens securely. +* **Rate limiting:** Implement rate limiting to prevent denial-of-service attacks. This limits the number of requests that a client can make to the API server within a given time period. +* **Input validation on the server-side:** Validate all input data on the server-side, even if you have already validated it on the client-side. This prevents attackers from bypassing client-side validation and injecting malicious data. + +## 5. Testing Approaches + +Testing is essential for ensuring the quality and reliability of Kivy applications. + +### 5.1 Unit Testing + +* **Test individual components:** Unit tests should focus on testing individual components, such as widgets, functions, and classes, in isolation. +* **Use a testing framework:** Use a testing framework, such as `unittest` or `pytest`, to organize and run your unit tests. +* **Write clear and concise tests:** Write clear and concise tests that are easy to understand and maintain. +* **Test edge cases:** Test edge cases to ensure that your code handles unexpected input or conditions correctly. +* **Use mock objects:** Use mock objects to isolate components from external dependencies, such as databases or network services. +* **Aim for high test coverage:** Aim for high test coverage to ensure that most of your code is tested. +* **Test Kivy properties:** Ensure your tests check for proper updates to Kivy Properties, including bindings and event triggering. + +### 5.2 Integration Testing + +* **Test interactions between components:** Integration tests should focus on testing the interactions between different components, such as screens, widgets, and data models. +* **Test the application as a whole:** Integration tests should also test the application as a whole to ensure that all components work together correctly. +* **Use a testing framework:** Use a testing framework, such as `unittest` or `pytest`, to organize and run your integration tests. +* **Use a GUI testing framework:** If you need to test the GUI directly, consider using a GUI testing framework, such as `kivy.tests` or `pytest-kivy`. + +### 5.3 End-to-End Testing + +* **Simulate user interactions:** End-to-end tests should simulate user interactions to test the application from the user's perspective. +* **Test the entire application flow:** End-to-end tests should test the entire application flow to ensure that all steps work correctly. +* **Use a testing framework:** Use a testing framework, such as `Selenium` or `Appium`, to automate your end-to-end tests. +* **Run tests on different platforms:** Run your end-to-end tests on different platforms to ensure that your application works correctly on all supported platforms. + +### 5.4 Test Organization + +* **Create a dedicated `tests` directory:** Create a dedicated `tests` directory to store your tests. +* **Organize tests by module:** Organize your tests by module to make it easier to find and run tests. +* **Use descriptive test names:** Use descriptive test names to make it clear what each test is testing. +* **Write test documentation:** Write test documentation to explain the purpose of each test and how it works. +* **Run tests automatically:** Run your tests automatically whenever you make changes to your code. Use a continuous integration system to automate this process. + +### 5.5 Mocking and Stubbing + +* **Use mock objects to isolate components:** Use mock objects to isolate components from external dependencies, such as databases or network services. +* **Use stub objects to simulate external dependencies:** Use stub objects to simulate external dependencies, such as APIs or hardware devices. +* **Use a mocking framework:** Use a mocking framework, such as `unittest.mock` or `mockito`, to create mock and stub objects easily. +* **Avoid over-mocking:** Avoid over-mocking, as this can make your tests less effective. Only mock dependencies that are truly external to the component being tested. + +## 6. Common Pitfalls and Gotchas + +Be aware of common pitfalls and gotchas when developing Kivy applications to avoid wasting time and effort. + +### 6.1 Frequent Mistakes + +* **Forgetting to call `super()`:** When subclassing Kivy widgets or classes, remember to call the `super()` method to initialize the parent class properly. +* **Not understanding Kivy's property system:** Kivy's property system is powerful but can be confusing at first. Make sure you understand how properties work and how to use them effectively. +* **Blocking the UI thread:** Avoid performing long-running tasks on the UI thread, as this can cause the application to freeze. +* **Not handling events correctly:** Handle events correctly to ensure that your application responds to user interactions as expected. +* **Not using the KV language effectively:** The KV language is a powerful tool for defining UI layouts and styles. Use it effectively to separate your UI definition from your Python code. +* **Incorrect file paths in deployment:** When packaging your app, ensure that file paths to assets and other dependencies are correct relative to the packaged application. + +### 6.2 Edge Cases + +* **Different screen resolutions:** Test your application on different screen resolutions to ensure that the UI scales correctly. +* **Different operating systems:** Test your application on different operating systems to ensure that it works correctly on all supported platforms. +* **Different hardware devices:** Test your application on different hardware devices to ensure that it works correctly on all target devices. +* **Handling orientation changes:** Implement proper handling of orientation changes (portrait/landscape) in your layouts. +* **Input from different devices:** Be aware of differences in input from touchscreens vs. mouse/keyboard, especially when designing interactions. + +### 6.3 Version-Specific Issues + +* **Kivy API changes:** Be aware of API changes between different versions of Kivy. Consult the Kivy documentation for migration guides. +* **Dependency compatibility:** Ensure that your dependencies are compatible with the version of Kivy you are using. +* **Deprecated features:** Be aware of deprecated features and avoid using them in new code. Migrate existing code to use the recommended alternatives. + +### 6.4 Compatibility Concerns + +* **Operating system dependencies:** Be aware of operating system dependencies and ensure that your application works correctly on all supported platforms. +* **Hardware dependencies:** Be aware of hardware dependencies and ensure that your application works correctly on all target devices. +* **Python version compatibility:** Ensure your code is compatible with supported Python versions. Kivy officially supports certain Python versions, so check the Kivy documentation for details. + +### 6.5 Debugging Strategies + +* **Use the Kivy console:** Use the Kivy console to inspect the state of your application and execute commands dynamically. +* **Use logging:** Use logging to record information about your application's behavior. This can be helpful for debugging issues that occur in production. +* **Use a debugger:** Use a debugger, such as `pdb` or `pycharm`, to step through your code and inspect variables. +* **Use Kivy's interactive mode:** Use Kivy's interactive mode to test UI layouts and styles without running the entire application. +* **Read Kivy's error messages carefully:** Kivy's error messages often provide valuable information about the cause of the error. Read them carefully and try to understand what they mean. +* **Search the Kivy documentation and online forums:** Search the Kivy documentation and online forums for solutions to common problems. The Kivy community is very active and helpful. + +## 7. Tooling and Environment + +Using the right tools and environment can significantly improve your Kivy development experience. + +### 7.1 Recommended Development Tools + +* **Text editor/IDE:** Use a good text editor or IDE, such as VS Code, PyCharm, or Sublime Text. These tools provide features such as syntax highlighting, code completion, and debugging. +* **Kivy Designer:** Kivy Designer is a visual UI designer that allows you to create UI layouts visually. It can be helpful for prototyping and experimenting with different UI designs. +* **Buildozer:** Buildozer is a tool for packaging Kivy applications for Android, iOS, and other platforms. It simplifies the process of creating deployment packages. +* **Python virtual environment manager:** Use virtualenv, venv (Python 3.3+), or conda to create isolated Python environments for your Kivy projects. This helps manage dependencies and avoid conflicts. + +### 7.2 Build Configuration + +* **Use a build system:** Use a build system, such as Make or CMake, to automate the build process. +* **Create a build script:** Create a build script to automate the build process. The build script should handle tasks such as installing dependencies, compiling code, and packaging the application. +* **Use environment variables:** Use environment variables to configure the build process. This allows you to customize the build process without modifying the code. +* **Use a configuration file:** Use a configuration file to store application settings. This allows you to change application settings without modifying the code. + +### 7.3 Linting and Formatting + +* **Use a linter:** Use a linter, such as `flake8` or `pylint`, to check your code for style errors and potential problems. +* **Use a code formatter:** Use a code formatter, such as `black` or `autopep8`, to automatically format your code according to PEP 8 guidelines. +* **Configure your editor/IDE:** Configure your editor/IDE to run the linter and code formatter automatically whenever you save a file. This helps ensure that your code is always clean and consistent. + +### 7.4 Deployment + +* **Choose the right deployment method:** Choose the right deployment method for your target platform. For Android, you can use Buildozer to create an APK. For desktop platforms, you can use PyInstaller or cx_Freeze to create executable files. +* **Create a deployment script:** Create a deployment script to automate the deployment process. The deployment script should handle tasks such as building the application, packaging it, and uploading it to the target platform. +* **Test your application thoroughly:** Test your application thoroughly on the target platform before deploying it to users. +* **Follow platform-specific guidelines:** Adhere to platform-specific guidelines for packaging and distributing applications (e.g., Google Play Store guidelines, Apple App Store guidelines). + +### 7.5 CI/CD Integration + +* **Use a CI/CD system:** Use a continuous integration/continuous delivery (CI/CD) system, such as Jenkins, Travis CI, GitHub Actions, or GitLab CI, to automate the build, test, and deployment process. +* **Create a CI/CD pipeline:** Create a CI/CD pipeline to automate the build, test, and deployment process. The pipeline should handle tasks such as checking out the code, installing dependencies, running tests, building the application, and deploying it to the target platform. +* **Use automated testing:** Use automated testing to ensure that your code is always working correctly. The CI/CD pipeline should run your tests automatically whenever you make changes to your code. +* **Automate deployment:** Automate deployment to the target platform. The CI/CD pipeline should deploy your application automatically whenever it passes all tests. +* **Integration with code review:** Integrate your CI/CD pipeline with your code review process. The CI/CD pipeline should run tests and linters automatically whenever a new pull request is created. + +By following these best practices, you can create maintainable, efficient, secure, and testable Kivy applications. \ No newline at end of file diff --git a/.cursor/rules/kubernetes.mdc b/.cursor/rules/kubernetes.mdc new file mode 100644 index 000000000..921fb530e --- /dev/null +++ b/.cursor/rules/kubernetes.mdc @@ -0,0 +1,353 @@ +--- +description: This rule provides comprehensive best practices for developing and maintaining Kubernetes applications and infrastructure, covering coding standards, security, performance, testing, and deployment. +globs: *.go,*.yaml,*.yml,*.sh,*.tf,*.tfvars,*.json +--- +# Kubernetes Development and Operations Best Practices + +This document outlines a collection of guidelines, style suggestions, and tips for writing code and managing infrastructure within the Kubernetes ecosystem. It emphasizes clarity, maintainability, security, and performance. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure + +- **Root Level:** + - `cmd/`: Main application entry points. Each subdirectory represents a separate command-line tool or service. + - `pkg/`: Reusable libraries and components that can be imported by other projects. + - `internal/`: Private code that should not be imported by external projects. Enforces encapsulation. + - `api/`: API definitions, including protobuf files and OpenAPI specifications. + - `config/`: Configuration files, such as YAML manifests, Kustomize configurations, and Helm charts. + - `scripts/`: Utility scripts for building, testing, and deploying the application. + - `docs/`: Documentation for the project. + - `examples/`: Example usage of the library or application. + - `vendor/`: (If using `go modules` without external dependency management) Contains vendored dependencies. Generally discouraged in modern Go with `go modules`. +- **Component-Specific Directories:** Inside `pkg/` or `internal/`, organize code by component or module. Each component should have its own directory with clear separation of concerns. + +Example: + + +my-kubernetes-project/ +├── cmd/ +│ └── controller/ +│ └── main.go +├── pkg/ +│ └── api/ +│ ├── types.go +│ └── controller/ +│ ├── controller.go +│ ├── reconciler.go +│ └── util/ +│ └── util.go +├── internal/ +│ └── admission/ +│ └── webhook.go +├── config/ +│ ├── deploy/ +│ │ └── deployment.yaml +│ └── kustomize/ +│ ├── base/ +│ │ ├── kustomization.yaml +│ │ └── ... +│ └── overlays/ +│ ├── dev/ +│ │ ├── kustomization.yaml +│ │ └── ... +│ └── prod/ +│ ├── kustomization.yaml +│ └── ... +├── scripts/ +│ └── build.sh +├── docs/ +│ └── architecture.md +└── go.mod + + +### 1.2 File Naming Conventions + +- **Go Files:** Use lowercase with underscores (e.g., `my_controller.go`). +- **YAML Files:** Use lowercase with dashes (e.g., `deployment.yaml`). +- **Configuration Files:** Be descriptive and consistent (e.g., `config.yaml`, `kustomization.yaml`). +- **Test Files:** Follow the standard Go convention: `*_test.go` (e.g., `my_controller_test.go`). + +### 1.3 Module Organization (Go) + +- **Packages:** Organize code into meaningful packages that represent logical units of functionality. +- **Internal Packages:** Use `internal/` directories to create packages that are only visible within the project. +- **Interfaces:** Define interfaces to abstract dependencies and promote testability. + +### 1.4 Component Architecture + +- **Microservices:** Design applications as a collection of loosely coupled microservices. +- **Separation of Concerns:** Each component should have a single responsibility and well-defined interfaces. +- **API Gateway:** Use an API gateway to handle routing, authentication, and rate limiting for external requests. +- **Service Mesh:** Consider using a service mesh (e.g., Istio, Linkerd) to manage inter-service communication, observability, and security. + +### 1.5 Code Splitting Strategies + +- **Feature-Based Splitting:** Group code by feature or functionality. +- **Layer-Based Splitting:** Separate code into layers, such as data access, business logic, and presentation. +- **Component-Based Splitting:** Divide code into reusable components that can be shared across multiple projects. + +## 2. Common Patterns and Anti-Patterns + +### 2.1 Design Patterns + +- **Controller Pattern:** Implement controllers to reconcile the desired state of Kubernetes resources with the actual state. +- **Operator Pattern:** Extend the Kubernetes API with custom resources and controllers to automate complex application management tasks. +- **Sidecar Pattern:** Deploy a sidecar container alongside the main application container to provide supporting functionality, such as logging, monitoring, or security. +- **Ambassador Pattern:** Use an ambassador container to proxy network traffic to the main application container, providing features such as load balancing, routing, and authentication. +- **Adapter Pattern:** Translate requests from one interface to another, allowing different components to work together. +- **Singleton Pattern:** Implement a singleton pattern for managing global resources, such as database connections or configuration settings. Be extremely cautious, as this can hurt testability and introduce implicit dependencies. + +### 2.2 Recommended Approaches for Common Tasks + +- **Resource Management:** Use Kubernetes resource requests and limits to ensure that applications have sufficient resources and prevent resource contention. +- **Configuration Management:** Use ConfigMaps and Secrets to manage configuration data and sensitive information separately from the application code. +- **Service Discovery:** Use Kubernetes services to provide a stable endpoint for accessing applications, even when pods are scaled up or down. +- **Health Checks:** Implement liveness and readiness probes to monitor the health of applications and automatically restart unhealthy pods. +- **Logging and Monitoring:** Use a centralized logging and monitoring system to collect and analyze application logs and metrics. +- **Rolling Updates:** Use Kubernetes deployments to perform rolling updates of applications with zero downtime. + +### 2.3 Anti-Patterns and Code Smells + +- **Naked Pods:** Avoid creating pods directly without a deployment or replica set, as they will not be automatically rescheduled if a node fails. +- **Hardcoded Configuration:** Avoid hardcoding configuration data in the application code. Use ConfigMaps and Secrets instead. +- **Ignoring Resource Limits:** Failing to set resource requests and limits can lead to resource contention and performance issues. +- **Oversized Containers:** Keep container images small and focused to improve startup time and reduce security risks. +- **Privileged Containers:** Avoid running containers in privileged mode, as it can create security vulnerabilities. +- **Long-Lived Branches:** Avoid creating long-lived branches, and prefer small, frequent merges to the main branch. +- **God Classes:** Avoid creating classes that are too large and complex. Break them down into smaller, more manageable classes. +- **Shotgun Surgery:** Avoid making changes to multiple classes when a single feature is modified. This suggests poor class design and coupling. +- **Feature Envy:** Avoid methods that access the data of another object more than their own. This suggests that the method might be in the wrong class. + +### 2.4 State Management Best Practices + +- **Stateless Applications:** Prefer stateless applications whenever possible, as they are easier to scale and manage. +- **Persistent Volumes:** Use Persistent Volumes to store persistent data for stateful applications. +- **External Databases:** Consider using external databases for managing application state, such as databases hosted on cloud providers. +- **Kubernetes Operators:** Implement Kubernetes operators to automate the management of stateful applications. +- **Etcd:** Understand the importance of etcd as Kubernetes' data store and protect it accordingly. + +### 2.5 Error Handling Patterns + +- **Centralized Error Handling:** Implement a centralized error handling mechanism to handle exceptions and log errors consistently. +- **Retry Mechanism:** Implement a retry mechanism to automatically retry failed operations. +- **Circuit Breaker Pattern:** Use a circuit breaker pattern to prevent cascading failures in distributed systems. +- **Logging Error Details:** Log detailed error messages, including stack traces and relevant context, to help with debugging. +- **Graceful Degradation:** Design applications to gracefully degrade functionality when errors occur. +- **Alerting on Critical Errors:** Set up alerts to notify administrators when critical errors occur. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +- **Caching:** Implement caching to reduce latency and improve performance. +- **Load Balancing:** Use load balancing to distribute traffic across multiple instances of an application. +- **Connection Pooling:** Use connection pooling to reuse database connections and reduce overhead. +- **Compression:** Use compression to reduce the size of data transmitted over the network. +- **Gzip:** Enable Gzip compression in web servers to reduce the size of HTTP responses. + +### 3.2 Memory Management + +- **Memory Profiling:** Use memory profiling tools to identify memory leaks and optimize memory usage. +- **Garbage Collection:** Understand how garbage collection works in the programming language used for the application. +- **Resource Limits:** Set memory resource limits for containers to prevent them from consuming excessive memory. +- **Monitor Memory Usage:** Monitor memory usage regularly to identify potential issues. + +### 3.3 Rendering Optimization + +- **Minimize DOM Manipulation:** Reduce the number of DOM manipulations to improve rendering performance in web applications. +- **Virtual DOM:** Use a virtual DOM to optimize rendering updates in web applications. +- **Lazy Loading:** Use lazy loading to load images and other resources only when they are needed. + +### 3.4 Bundle Size Optimization + +- **Code Minification:** Use code minification to reduce the size of JavaScript and CSS files. +- **Tree Shaking:** Use tree shaking to remove unused code from JavaScript bundles. +- **Image Optimization:** Optimize images to reduce their file size without sacrificing quality. +- **Code Splitting:** Split the application code into smaller bundles that can be loaded on demand. + +### 3.5 Lazy Loading Strategies + +- **On-Demand Loading:** Load resources only when they are needed by the application. +- **Intersection Observer:** Use the Intersection Observer API to detect when elements are visible in the viewport and load them accordingly. +- **Placeholder Images:** Use placeholder images while loading the actual images to improve the user experience. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and How to Prevent Them + +- **Injection Attacks:** Prevent injection attacks by validating and sanitizing all user input. +- **Cross-Site Scripting (XSS):** Prevent XSS attacks by escaping all user-generated content before rendering it in the browser. +- **Cross-Site Request Forgery (CSRF):** Prevent CSRF attacks by using anti-CSRF tokens. +- **Authentication and Authorization Flaws:** Implement robust authentication and authorization mechanisms to protect sensitive data and resources. +- **Security Misconfiguration:** Avoid using default configurations and ensure that all components are properly configured with security in mind. +- **Using Components with Known Vulnerabilities:** Keep all dependencies up to date to patch known vulnerabilities. +- **Insufficient Logging and Monitoring:** Implement comprehensive logging and monitoring to detect and respond to security incidents. +- **Container Security:** Follow best practices for securing containers, such as using minimal images, running as non-root, and limiting capabilities. +- **Network Policies:** Use network policies to restrict network traffic between pods. +- **RBAC (Role-Based Access Control):** Implement RBAC to control access to Kubernetes resources. +- **Secrets Management:** Use Kubernetes Secrets to store sensitive information, and encrypt secrets at rest. +- **Pod Security Policies/Pod Security Standards:** Enforce Pod Security Standards to restrict the capabilities of pods. + +### 4.2 Input Validation + +- **Validate All Input:** Validate all input, including user input, API requests, and configuration data. +- **Use Strong Data Types:** Use strong data types to enforce data integrity. +- **Sanitize Input:** Sanitize input to remove potentially harmful characters and prevent injection attacks. +- **Whitelist Input:** Use a whitelist approach to only allow known good input. +- **Blacklist Input:** Avoid using a blacklist approach, as it can be easily bypassed. + +### 4.3 Authentication and Authorization Patterns + +- **Multi-Factor Authentication (MFA):** Use MFA to enhance authentication security. +- **OAuth 2.0:** Use OAuth 2.0 for authorization and delegation of access. +- **JSON Web Tokens (JWT):** Use JWTs for securely transmitting claims between parties. +- **Role-Based Access Control (RBAC):** Implement RBAC to control access to resources based on roles. +- **Least Privilege Principle:** Grant users and applications only the minimum necessary permissions. + +### 4.4 Data Protection Strategies + +- **Encryption at Rest:** Encrypt sensitive data at rest to protect it from unauthorized access. +- **Encryption in Transit:** Encrypt sensitive data in transit using HTTPS or other secure protocols. +- **Data Masking:** Mask sensitive data to prevent it from being exposed to unauthorized users. +- **Data Anonymization:** Anonymize data to remove personally identifiable information (PII). +- **Data Loss Prevention (DLP):** Implement DLP measures to prevent sensitive data from leaving the organization. + +### 4.5 Secure API Communication + +- **HTTPS:** Use HTTPS for all API communication to encrypt data in transit. +- **API Authentication:** Implement API authentication to verify the identity of clients. +- **API Authorization:** Implement API authorization to control access to API endpoints. +- **Rate Limiting:** Implement rate limiting to prevent abuse and denial-of-service attacks. +- **Input Validation:** Validate all API requests to prevent injection attacks and other vulnerabilities. + +## 5. Testing Approaches + +### 5.1 Unit Testing Strategies + +- **Test-Driven Development (TDD):** Write unit tests before writing the application code. +- **Mock Dependencies:** Use mocks to isolate the unit being tested from its dependencies. +- **Test Boundary Conditions:** Test boundary conditions and edge cases to ensure that the code handles them correctly. +- **Test Error Conditions:** Test error conditions to ensure that the code handles errors gracefully. +- **Code Coverage:** Aim for high code coverage to ensure that all parts of the code are tested. +- **Table-Driven Tests:** Use table-driven tests to easily test multiple inputs and outputs. + +### 5.2 Integration Testing + +- **Test Interactions Between Components:** Test the interactions between different components of the application. +- **Test with Real Dependencies:** Use real dependencies or integration mocks for integration tests. +- **Test Data Flows:** Test the data flows through the application to ensure that data is processed correctly. +- **Contract Tests:** Use contract tests to ensure that services adhere to a defined contract. + +### 5.3 End-to-End Testing + +- **Test the Entire System:** Test the entire system from end to end to ensure that all components work together correctly. +- **Automate End-to-End Tests:** Automate end-to-end tests to ensure that they are run regularly. +- **Use Realistic Test Data:** Use realistic test data to simulate real-world scenarios. +- **CI/CD Integration:** Integrate end-to-end tests into the CI/CD pipeline. + +### 5.4 Test Organization + +- **Keep Tests Separate from Code:** Keep tests separate from the application code in a dedicated `test/` directory. +- **Organize Tests by Component:** Organize tests by component or module to make them easier to find and maintain. +- **Use Clear Naming Conventions:** Use clear naming conventions for test files and test functions. + +### 5.5 Mocking and Stubbing + +- **Use Mocking Frameworks:** Use mocking frameworks to simplify the creation of mocks and stubs. +- **Mock External Dependencies:** Mock external dependencies, such as databases and APIs, to isolate the unit being tested. +- **Stub Responses:** Use stubs to provide predefined responses for external dependencies. +- **Verify Interactions:** Verify that the code under test interacts with dependencies as expected. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes Developers Make + +- **Ignoring Error Handling:** Failing to handle errors properly can lead to unexpected behavior and crashes. +- **Not Using Version Control:** Not using version control can lead to lost code and conflicts. +- **Hardcoding Configuration:** Hardcoding configuration data can make it difficult to deploy the application in different environments. +- **Not Securing Sensitive Data:** Not securing sensitive data can lead to security breaches. +- **Not Testing Thoroughly:** Not testing thoroughly can lead to bugs and performance issues. +- **Over-Engineering:** Adding unnecessary complexity to the code can make it difficult to understand and maintain. +- **Premature Optimization:** Optimizing code before it is necessary can waste time and make the code harder to read. +- **Not Documenting Code:** Not documenting code can make it difficult for others to understand and maintain it. + +### 6.2 Edge Cases to Be Aware Of + +- **Network Connectivity Issues:** Handle network connectivity issues gracefully. +- **Resource Exhaustion:** Handle resource exhaustion gracefully. +- **Concurrency Issues:** Avoid concurrency issues by using proper synchronization mechanisms. +- **Data Corruption:** Protect against data corruption by using checksums and other data integrity techniques. +- **Time Zone Issues:** Be aware of time zone issues when working with dates and times. + +### 6.3 Version-Specific Issues + +- **API Version Compatibility:** Be aware of API version compatibility issues when upgrading Kubernetes or other dependencies. +- **Feature Deprecation:** Be aware of feature deprecation when upgrading Kubernetes or other dependencies. +- **Configuration Changes:** Be aware of configuration changes when upgrading Kubernetes or other dependencies. + +### 6.4 Compatibility Concerns + +- **Operating System Compatibility:** Ensure that the application is compatible with the target operating systems. +- **Architecture Compatibility:** Ensure that the application is compatible with the target architectures (e.g., x86, ARM). +- **Browser Compatibility:** Ensure that web applications are compatible with the target browsers. + +### 6.5 Debugging Strategies + +- **Logging:** Use detailed logging to help identify the root cause of issues. +- **Debugging Tools:** Use debugging tools, such as debuggers and profilers, to analyze the code and identify performance bottlenecks. +- **Remote Debugging:** Use remote debugging to debug applications running in Kubernetes. +- **Log Aggregation:** Use log aggregation tools (e.g., Elasticsearch, Loki) to centralize and analyze logs. +- **Metrics Monitoring:** Use metrics monitoring tools (e.g., Prometheus, Grafana) to track application performance. +- **Tracing:** Implement distributed tracing (e.g., Jaeger, Zipkin) to track requests across multiple services. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +- **IDE:** Use a modern IDE with support for the programming language used for the application (e.g., VS Code, IntelliJ IDEA, GoLand). +- **Kubectl:** Use `kubectl` for interacting with Kubernetes clusters. +- **Minikube/Kind:** Use Minikube or Kind for local Kubernetes development. +- **Helm:** Use Helm for managing Kubernetes packages. +- **Kustomize:** Use Kustomize for customizing Kubernetes configurations. +- **Docker:** Use Docker for building and managing container images. +- **Tilt:** Use Tilt for fast, local Kubernetes development. +- **Skaffold:** Use Skaffold for automated build, push, and deploy workflows. +- **Telepresence:** Use Telepresence to debug applications running in Kubernetes from your local machine. + +### 7.2 Build Configuration + +- **Makefile:** Use a Makefile to automate common build tasks. +- **CI/CD Pipeline:** Integrate the build process into a CI/CD pipeline. +- **Dependency Management:** Use a dependency management tool, such as `go modules`, to manage dependencies. +- **Version Control:** Use version control to track changes to the build configuration. + +### 7.3 Linting and Formatting + +- **Linters:** Use linters to enforce code style and best practices (e.g., `golangci-lint`, `eslint`, `stylelint`). +- **Formatters:** Use formatters to automatically format code according to a predefined style (e.g., `go fmt`, `prettier`). +- **Pre-Commit Hooks:** Use pre-commit hooks to run linters and formatters before committing code. + +### 7.4 Deployment Best Practices + +- **Infrastructure as Code (IaC):** Use IaC tools, such as Terraform or CloudFormation, to manage infrastructure. +- **Immutable Infrastructure:** Deploy immutable infrastructure to ensure consistency and repeatability. +- **Blue-Green Deployments:** Use blue-green deployments to minimize downtime during deployments. +- **Canary Deployments:** Use canary deployments to test new versions of the application with a small subset of users. +- **Rolling Updates:** Use rolling updates to gradually update the application with zero downtime. +- **Automated Rollbacks:** Implement automated rollbacks to quickly revert to a previous version of the application if something goes wrong. + +### 7.5 CI/CD Integration + +- **Automated Testing:** Automate all tests in the CI/CD pipeline. +- **Automated Deployment:** Automate the deployment process in the CI/CD pipeline. +- **Continuous Integration:** Use continuous integration to automatically build and test the application whenever code is committed. +- **Continuous Delivery:** Use continuous delivery to automatically deploy the application to production whenever a new version is released. +- **Pipeline Security:** Secure the CI/CD pipeline to prevent unauthorized access and code injection. + +## Bibliography + +- Kubernetes documentation: [https://kubernetes.io/docs/](https://kubernetes.io/docs/) +- Kubernetes Best Practices: [https://kubernetes.io/docs/concepts/configuration/overview/](https://kubernetes.io/docs/concepts/configuration/overview/) +- Application Security Checklist: [https://kubernetes.io/docs/concepts/security/application-security-checklist/](https://kubernetes.io/docs/concepts/security/application-security-checklist/) +- Kubernetes coding conventions: [https://www.kubernetes.dev/docs/guide/coding-convention/](https://www.kubernetes.dev/docs/guide/coding-convention/) \ No newline at end of file diff --git a/.cursor/rules/langchain-js.mdc b/.cursor/rules/langchain-js.mdc new file mode 100644 index 000000000..dbea59a36 --- /dev/null +++ b/.cursor/rules/langchain-js.mdc @@ -0,0 +1,179 @@ +--- +description: Comprehensive best practices and coding standards for developing applications using LangChain.js. Focuses on code organization, performance, security, testing, and common pitfalls to ensure robust and maintainable AI-driven solutions. +globs: *.js,*.ts,*.jsx,*.tsx +--- +- **Monitor and Evaluate LLM applications**: Utilize tools like LangSmith for monitoring application performance, including logging traces, analyzing latency, and evaluating LLM outputs against predefined metrics. This helps identify bottlenecks and ensures the application meets quality standards. Enable tracing by setting environment variables: + `export LANGCHAIN_TRACING_V2="true"` + `export LANGCHAIN_API_KEY="..."` + +- **Implement Stateful Agents**: Use LangGraph to build stateful agents, crucial for applications like chatbots where remembering past interactions enhances user experience. Model interactions as a graph with nodes (states) and edges (transitions). + +- **Maintain High Code Quality**: Enforce strict code quality through regular testing, ESLint, and Prettier for consistent code formatting and linting. + +- **Comprehensive Documentation**: Ensure all components and their interactions are well-documented for maintainability and scalability. + +- **Use LangChain Expression Language (LCEL)**: Employ LCEL for composing chains in a declarative way, supporting production deployment without code changes. + +- **Explore Trade-offs in Deployment**: Choose between using external LLM providers or self-hosting open-source models based on cost, latency, and privacy considerations. + +- **Secure Coding Practices**: Read up on our Security best practices to make sure you're developing safely with LangChain. + +## 1. Code Organization and Structure + +- **Directory Structure**: Organize code into logical modules based on functionality (e.g., `chains`, `agents`, `tools`, `memory`). + + src/ + ├── chains/ + │ ├── conversationalChain.ts + │ └── summarizationChain.ts + ├── agents/ + │ ├── agent.ts + │ └── agentExecutor.ts + ├── tools/ + │ ├── searchTool.ts + │ └── calculatorTool.ts + ├── memory/ + │ ├── bufferMemory.ts + │ └── conversationBufferMemory.ts + ├── utils/ + │ ├── api.ts + │ └── helpers.ts + ├── index.ts + └── types.ts + + +- **File Naming Conventions**: Use descriptive names, typically in `camelCase` for variables and functions, and `PascalCase` for classes and interfaces. Consider prefixes or suffixes to denote the type of module e.g., `*_chain.ts` or `*_agent.ts` + +- **Module Organization**: Group related functionalities into modules with clear interfaces. Use `index.ts` files to export module members for cleaner imports. + typescript + // chains/index.ts + export * from './conversationalChain'; + export * from './summarizationChain'; + + // Usage: + import { ConversationalChain, SummarizationChain } from '@/chains'; + + +- **Component Architecture**: Design modular components for reusability. Implement interfaces to define contracts between components. + typescript + // Define an interface for a Tool + interface ToolInterface { + name: string; + description: string; + execute(input: string): Promise; + } + + // Implement the interface in a specific Tool + class SearchTool implements ToolInterface { + name = 'search'; + description = 'Useful for searching the internet.'; + async execute(input: string): Promise { + // Implementation + } + } + + +- **Code Splitting**: Implement code splitting using dynamic imports to reduce initial load time, especially for large applications. + typescript + async function loadLargeModule() { + const largeModule = await import('./largeModule'); + largeModule.initialize(); + } + + +## 2. Common Patterns and Anti-patterns + +- **Design Patterns**: Use the Factory pattern for creating different types of chains or agents, and the Strategy pattern for choosing different LLMs. + typescript + // Factory Pattern for creating chains + class ChainFactory { + static createChain(type: 'conversational' | 'summarization', llm: LLM) { + if (type === 'conversational') { + return new ConversationalChain(llm); + } else if (type === 'summarization') { + return new SummarizationChain(llm); + } else { + throw new Error('Invalid chain type'); + } + } + } + + const chain = ChainFactory.createChain('conversational', new OpenAIChat()); + + +- **Recommended Approaches**: Leverage LangChain's built-in modules and chains when possible, and customize them as needed. Prioritize asynchronous operations to prevent blocking the main thread. + +- **Anti-patterns**: Avoid deeply nested callbacks, which can lead to callback hell. Use `async/await` and promises for cleaner asynchronous code. + +- **State Management**: For simple applications, manage state with React's `useState` or similar. For complex applications, consider state management libraries like Zustand or Redux. + +- **Error Handling**: Implement robust error handling with `try/catch` blocks and global error handlers. Log errors with context for debugging. + typescript + async function processData() { + try { + const result = await fetchData(); + // Process result + } catch (error) { + console.error('Error processing data:', error); + // Handle error (e.g., display error message to user) + } + } + + +## 3. Performance Considerations + +- **Optimization Techniques**: Use caching to store and reuse LLM responses. Optimize prompts to reduce token usage. Debounce computationally expensive operations. + +- **Memory Management**: Be mindful of memory leaks, especially when using streams or subscriptions. Properly clean up resources when components unmount. + +- **Bundle Size Optimization**: Use tools like Webpack Bundle Analyzer to identify large dependencies. Use tree shaking to remove unused code. + +- **Lazy Loading**: Implement lazy loading for components and modules that are not immediately needed to improve initial load time. + +## 4. Security Best Practices + +- **Vulnerabilities**: Prevent prompt injection attacks by carefully validating user inputs and using sandboxed execution environments. + +- **Input Validation**: Sanitize user inputs to prevent malicious code execution. Limit the length of inputs to prevent denial-of-service attacks. + +- **Authentication/Authorization**: Use secure authentication and authorization mechanisms to protect sensitive data and prevent unauthorized access. + +- **Data Protection**: Encrypt sensitive data at rest and in transit. Comply with data privacy regulations like GDPR and CCPA. + +- **Secure API Communication**: Use HTTPS for all API communication. Validate API responses to prevent data corruption. + +## 5. Testing Approaches + +- **Unit Testing**: Test individual components in isolation using Jest or Mocha. Mock dependencies to control test environments. + +- **Integration Testing**: Test interactions between components to ensure they work together correctly. Use in-memory databases or mock APIs for integration tests. + +- **End-to-end Testing**: Test the entire application flow using tools like Cypress or Puppeteer. Simulate user interactions to verify functionality. + +- **Test Organization**: Organize tests into separate directories based on component or module. Use clear and descriptive test names. + +- **Mocking/Stubbing**: Use mocking libraries like Jest's `jest.mock()` or Sinon.js to replace dependencies with controlled test doubles. + +## 6. Common Pitfalls and Gotchas + +- **Frequent Mistakes**: Incorrectly configuring API keys, not handling errors properly, and overlooking prompt injection vulnerabilities. + +- **Edge Cases**: Handling unexpected input formats, dealing with rate limits from LLM providers, and managing long-running operations. + +- **Version-Specific Issues**: Check release notes for breaking changes when upgrading LangChain.js versions. + +- **Compatibility Concerns**: Ensure compatibility between LangChain.js and other libraries, especially those related to data processing or UI frameworks. + +- **Debugging Strategies**: Use console logging, debuggers, and monitoring tools like LangSmith to diagnose issues. + +## 7. Tooling and Environment + +- **Recommended Tools**: VS Code with TypeScript support, ESLint, Prettier, and Jest. + +- **Build Configuration**: Use Webpack, Parcel, or Rollup for bundling and optimization. Configure TypeScript compiler options for strict type checking. + +- **Linting/Formatting**: Enforce consistent code style with ESLint and Prettier. Use linting rules to catch potential errors and enforce best practices. + +- **Deployment Best Practices**: Use serverless functions or containerization for deployment. Implement monitoring and alerting to detect issues in production. + +- **CI/CD Integration**: Automate testing, linting, and deployment with CI/CD pipelines using tools like GitHub Actions or Jenkins. \ No newline at end of file diff --git a/.cursor/rules/langchain.mdc b/.cursor/rules/langchain.mdc new file mode 100644 index 000000000..f3293ca31 --- /dev/null +++ b/.cursor/rules/langchain.mdc @@ -0,0 +1,310 @@ +--- +description: This rule provides best practices for developing LangChain applications, covering code organization, performance, security, testing, and common pitfalls. It aims to improve code quality, maintainability, and overall project success. +globs: *.py +--- +# LangChain Development Best Practices + +This document outlines the best practices for developing LangChain applications to ensure code quality, maintainability, performance, security, and overall project success. These guidelines cover various aspects of development, from code organization to testing and deployment. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure Best Practices + +Adopt a clear and consistent directory structure to improve code discoverability and maintainability. A recommended structure is: + + +project_root/ +├── data/ # Raw data, processed data, and datasets +├── src/ # Source code directory +│ ├── components/ # Reusable LangChain components (e.g., custom chains, tools) +│ ├── chains/ # Definitions of LangChain chains +│ ├── agents/ # Agent implementations +│ ├── memory/ # Memory implementations +│ ├── utils/ # Utility functions and modules +│ ├── models/ # Custom model definitions or wrappers +│ ├── callbacks/ # Custom callback handlers +│ ├── vectorstores/ # Vectorstore configurations and connections +│ ├── document_loaders/ # Custom document loaders +│ ├── prompts/ # Prompt templates and management +│ ├── config/ # Configuration files +│ └── main.py # Entry point of the application +├── tests/ # Unit and integration tests +├── notebooks/ # Jupyter notebooks for experimentation and documentation +├── docs/ # Project documentation +├── requirements.txt # Project dependencies +├── pyproject.toml # Project metadata and build configuration +└── README.md # Project README file + + +### 1.2 File Naming Conventions + +Use descriptive and consistent file names: + +- `module_name.py`: For general modules. +- `component_name.py`: For LangChain components (e.g., `custom_chain.py`). +- `test_module_name.py`: For test files. +- Use lowercase and underscores for file names (snake_case). + +### 1.3 Module Organization + +Organize code into logical modules based on functionality. Each module should have a clear purpose and minimal dependencies. + +- **Cohesion**: Modules should have high cohesion, meaning their elements are closely related. +- **Coupling**: Modules should have low coupling, meaning they are independent of each other as much as possible. + +### 1.4 Component Architecture + +Design LangChain applications using a component-based architecture. Components should be reusable, testable, and well-defined. + +- **Chains**: Define chains as reusable components that encapsulate specific workflows. +- **Agents**: Implement agents as modular entities that interact with the environment using tools. +- **Memory**: Manage conversation history and state using memory components. +- **Tools**: Create tools as independent units that perform specific actions. +- **Callbacks**: Utilize callbacks for logging, monitoring, and custom event handling. + +### 1.5 Code Splitting Strategies + +Split large files into smaller, manageable chunks to improve readability and maintainability. + +- **Function-level splitting**: Break down large functions into smaller, single-purpose functions. +- **Class-level splitting**: Divide large classes into smaller, more focused classes. +- **Module-level splitting**: Separate modules based on functionality to reduce complexity. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns Specific to LangChain + +- **Chain of Responsibility**: Implement chains of operations where each component handles a specific task, passing the result to the next component. +- **Strategy Pattern**: Use strategy patterns to encapsulate different algorithms or behaviors within interchangeable strategy objects. +- **Template Method**: Define the skeleton of an algorithm in a base class, allowing subclasses to override specific steps without changing the algorithm's structure. +- **Factory Pattern**: Use factory patterns to create instances of LangChain components dynamically, based on configuration or runtime conditions. + +### 2.2 Recommended Approaches for Common Tasks + +- **Prompt Engineering**: Use prompt templates to manage and reuse prompts. Optimize prompts for clarity, context, and desired output. +- **Data Loading**: Implement custom data loaders to handle various data sources and formats. Use text splitters to chunk large documents into smaller pieces for retrieval. +- **Vector Storage**: Use vector stores to store and retrieve embeddings efficiently. Choose the appropriate vector store based on performance, scalability, and cost. +- **Agent Design**: Design agents with clear objectives, tools, and decision-making logic. Use observation logs to track agent actions and outcomes. +- **Memory Management**: Implement memory components to maintain conversation history and context. Use sliding window or summarization techniques to manage long conversations. + +### 2.3 Anti-patterns and Code Smells to Avoid + +- **God Classes**: Avoid creating large classes that handle too many responsibilities. +- **Long Methods**: Avoid creating long methods that are difficult to understand and maintain. +- **Duplicated Code**: Avoid duplicating code across multiple modules. Extract common code into reusable functions or components. +- **Magic Numbers**: Avoid using magic numbers or hardcoded values. Define constants or configuration variables instead. +- **Tight Coupling**: Avoid creating tight coupling between modules. Use interfaces and dependency injection to promote loose coupling. + +### 2.4 State Management Best Practices + +- **Stateless Components**: Design components to be stateless whenever possible. This improves testability and scalability. +- **Centralized State**: Manage application state in a centralized location (e.g., a state management class or library). +- **Immutable State**: Use immutable data structures to prevent unintended side effects and improve predictability. +- **Explicit State Transitions**: Define explicit state transitions to make state changes clear and traceable. + +### 2.5 Error Handling Patterns + +- **Try-Except Blocks**: Use try-except blocks to handle exceptions and prevent application crashes. +- **Logging**: Log errors and exceptions to facilitate debugging and monitoring. +- **Custom Exceptions**: Define custom exceptions to represent specific error conditions. +- **Retry Logic**: Implement retry logic for transient errors (e.g., network timeouts). +- **Fallback Strategies**: Implement fallback strategies for critical operations to ensure application resilience. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +- **Caching**: Implement caching mechanisms to store frequently accessed data and results. +- **Batch Processing**: Process data in batches to reduce overhead and improve throughput. +- **Asynchronous Operations**: Use asynchronous operations to perform non-blocking I/O and improve responsiveness. +- **Connection Pooling**: Use connection pooling to reuse database connections and reduce latency. +- **Data Compression**: Compress data to reduce storage space and network bandwidth. +- **Vectorstore Optimization**: Use efficient vectorstore implementations (e.g., FAISS, Annoy) and optimize indexing parameters for fast retrieval. + +### 3.2 Memory Management + +- **Object Pooling**: Use object pooling to reuse objects and reduce memory allocation overhead. +- **Garbage Collection**: Monitor garbage collection performance and tune parameters to minimize pauses. +- **Memory Profiling**: Use memory profiling tools to identify memory leaks and optimize memory usage. +- **Lazy Loading**: Load data on demand to reduce initial memory footprint. +- **Chunking Large Documents**: Process large documents in smaller chunks to avoid memory overflow. + +### 3.3 Rendering Optimization (if applicable for UI components) + +- **Virtualization**: Use virtualization techniques to render large lists efficiently. +- **Debouncing and Throttling**: Use debouncing and throttling to reduce the frequency of UI updates. +- **Memoization**: Use memoization to cache expensive rendering calculations. + +### 3.4 Bundle Size Optimization (if applicable for web apps) + +- **Code Splitting**: Split code into smaller chunks to reduce initial load time. +- **Tree Shaking**: Use tree shaking to remove unused code from bundles. +- **Minification and Compression**: Minify and compress code to reduce bundle size. +- **Lazy Loading**: Load components and modules on demand. + +### 3.5 Lazy Loading Strategies + +- **On-Demand Loading**: Load data or components only when they are needed. +- **Intersection Observer**: Use the Intersection Observer API to load components when they become visible in the viewport. +- **Dynamic Imports**: Use dynamic imports to load modules asynchronously. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and How to Prevent Them + +- **Prompt Injection**: Prevent prompt injection by validating and sanitizing user inputs. Use prompt templates and parameterized queries to avoid direct injection of malicious code. +- **Data Exfiltration**: Prevent data exfiltration by restricting access to sensitive data and implementing data masking techniques. +- **Code Execution**: Prevent arbitrary code execution by avoiding the use of `eval()` or similar functions. Use safe alternatives for dynamic code generation. +- **Denial of Service (DoS)**: Prevent DoS attacks by implementing rate limiting, input validation, and resource quotas. + +### 4.2 Input Validation + +- **Whitelisting**: Validate inputs against a whitelist of allowed values or patterns. +- **Sanitization**: Sanitize inputs to remove or escape potentially harmful characters or code. +- **Type Checking**: Enforce type checking to ensure that inputs conform to expected data types. +- **Length Limits**: Enforce length limits to prevent buffer overflows or excessive memory usage. + +### 4.3 Authentication and Authorization Patterns + +- **Authentication**: Use strong authentication mechanisms (e.g., multi-factor authentication) to verify user identities. +- **Authorization**: Implement role-based access control (RBAC) to restrict access to resources based on user roles. +- **Least Privilege**: Grant users the minimum necessary privileges to perform their tasks. +- **Secure Storage**: Store sensitive credentials (e.g., API keys) securely using encryption or secret management tools. + +### 4.4 Data Protection Strategies + +- **Encryption**: Encrypt sensitive data at rest and in transit. +- **Data Masking**: Mask sensitive data to protect it from unauthorized access. +- **Data Anonymization**: Anonymize data to remove personally identifiable information (PII). +- **Access Logging**: Log all data access events to track and monitor usage. +- **Data Retention**: Define and enforce data retention policies to minimize the risk of data breaches. + +### 4.5 Secure API Communication + +- **HTTPS**: Use HTTPS to encrypt communication between clients and servers. +- **API Keys**: Protect API keys and other sensitive credentials. +- **Rate Limiting**: Implement rate limiting to prevent abuse and DoS attacks. +- **Input Validation**: Validate all API inputs to prevent injection attacks. +- **Output Encoding**: Encode API outputs to prevent cross-site scripting (XSS) attacks. + +## 5. Testing Approaches + +### 5.1 Unit Testing Strategies + +- **Test-Driven Development (TDD)**: Write unit tests before writing the code to be tested. +- **Mocking**: Use mocking to isolate components and test them independently. +- **Assertion**: Use assertions to verify that the code behaves as expected. +- **Coverage**: Aim for high code coverage to ensure that all code paths are tested. +- **Parameterized Tests**: Use parameterized tests to test multiple scenarios with different inputs. + +### 5.2 Integration Testing + +- **Component Integration**: Test the integration between components to ensure that they work together correctly. +- **API Integration**: Test the integration with external APIs to ensure that data is exchanged correctly. +- **Database Integration**: Test the integration with databases to ensure that data is stored and retrieved correctly. +- **End-to-End Flows**: Test end-to-end flows to ensure that the application works as a whole. + +### 5.3 End-to-End Testing + +- **UI Testing**: Test the user interface to ensure that it is functional and user-friendly. +- **Functional Testing**: Test the functional requirements of the application to ensure that it meets the specifications. +- **Performance Testing**: Test the performance of the application to ensure that it is responsive and scalable. +- **Security Testing**: Test the security of the application to identify and mitigate vulnerabilities. +- **Accessibility Testing**: Test the accessibility of the application to ensure that it is usable by people with disabilities. + +### 5.4 Test Organization + +- **Test Suites**: Organize tests into test suites based on functionality or component. +- **Test Naming**: Use descriptive test names to make it clear what each test is testing. +- **Test Data**: Use realistic test data to simulate real-world scenarios. +- **Test Environment**: Set up a dedicated test environment to isolate tests from production data. + +### 5.5 Mocking and Stubbing + +- **Mocking**: Use mocking to replace external dependencies with controlled substitutes. +- **Stubbing**: Use stubbing to provide predefined responses to external dependencies. +- **Dependency Injection**: Use dependency injection to make it easier to mock and stub dependencies. +- **Mocking Frameworks**: Use mocking frameworks (e.g., `unittest.mock`) to simplify the mocking process. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes Developers Make + +- **Hardcoding API Keys**: Storing API keys directly in the code instead of using environment variables. +- **Ignoring Rate Limits**: Failing to handle API rate limits, leading to errors and service disruptions. +- **Lack of Input Validation**: Not validating user inputs, making the application vulnerable to prompt injection attacks. +- **Insufficient Error Handling**: Not handling errors properly, leading to application crashes and data loss. +- **Over-Reliance on Default Settings**: Using default settings without considering their impact on performance and security. + +### 6.2 Edge Cases to Be Aware Of + +- **Empty Inputs**: Handling empty inputs gracefully to prevent errors. +- **Long Inputs**: Handling long inputs efficiently to avoid performance issues. +- **Special Characters**: Handling special characters correctly to prevent injection attacks. +- **Unicode Support**: Ensuring proper Unicode support to handle different languages and character sets. +- **Network Errors**: Handling network errors gracefully to ensure application resilience. + +### 6.3 Version-Specific Issues + +- **API Changes**: Being aware of API changes in different LangChain versions and updating code accordingly. +- **Compatibility**: Ensuring compatibility between different LangChain components and versions. +- **Deprecated Features**: Avoiding the use of deprecated features and migrating to their replacements. + +### 6.4 Compatibility Concerns + +- **Python Versions**: Ensuring compatibility with different Python versions. +- **Operating Systems**: Ensuring compatibility with different operating systems (e.g., Windows, macOS, Linux). +- **Dependency Conflicts**: Resolving dependency conflicts between different libraries. + +### 6.5 Debugging Strategies + +- **Logging**: Use logging to track the execution flow and identify errors. +- **Debugging Tools**: Use debugging tools (e.g., `pdb`) to step through code and inspect variables. +- **Print Statements**: Use print statements strategically to output debugging information. +- **Error Messages**: Pay attention to error messages and stack traces to understand the root cause of errors. +- **Remote Debugging**: Use remote debugging to debug applications running on remote servers. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +- **IDE**: Use a powerful IDE (e.g., VS Code, PyCharm) with support for Python and LangChain. +- **Linters**: Use linters (e.g., `flake8`, `pylint`) to enforce code style and identify potential errors. +- **Formatters**: Use formatters (e.g., `black`, `autopep8`) to automatically format code according to PEP 8 standards. +- **Debuggers**: Use debuggers (e.g., `pdb`, `ipdb`) to step through code and inspect variables. +- **Version Control**: Use Git for version control and collaboration. + +### 7.2 Build Configuration + +- **`pyproject.toml`**: Use `pyproject.toml` file to manage project metadata, dependencies, and build configuration. +- **`requirements.txt`**: Generate and update the `requirements.txt` file to specify project dependencies. +- **Virtual Environments**: Use virtual environments (`venv`, `conda`) to isolate project dependencies. + +### 7.3 Linting and Formatting + +- **Linting**: Configure linters to enforce code style and identify potential errors automatically. +- **Formatting**: Configure formatters to automatically format code according to PEP 8 standards. +- **Pre-commit Hooks**: Use pre-commit hooks to run linters and formatters before committing code. + +### 7.4 Deployment Best Practices + +- **Containerization**: Use containerization (e.g., Docker) to package the application and its dependencies. +- **Orchestration**: Use orchestration tools (e.g., Kubernetes) to manage and scale the application. +- **Infrastructure as Code (IaC)**: Use IaC tools (e.g., Terraform, CloudFormation) to provision and manage infrastructure. +- **Monitoring**: Implement monitoring and logging to track application performance and identify issues. +- **Continuous Deployment**: Implement continuous deployment to automate the deployment process. + +### 7.5 CI/CD Integration + +- **Continuous Integration (CI)**: Use CI tools (e.g., GitHub Actions, GitLab CI, Jenkins) to automatically build, test, and analyze code. +- **Continuous Delivery (CD)**: Use CD tools to automatically deploy code to staging or production environments. +- **Automated Testing**: Integrate automated testing into the CI/CD pipeline to ensure code quality. +- **Rollback Strategies**: Implement rollback strategies to quickly revert to previous versions in case of deployment failures. + +By following these best practices, developers can build robust, scalable, and maintainable LangChain applications that meet the needs of their users and stakeholders. + +This comprehensive guide is designed to help developers create high-quality LangChain applications by adhering to industry-standard coding practices and principles. + + +@file best_practices_python.mdc +@file best_practices_langchain_specific.mdc \ No newline at end of file diff --git a/.cursor/rules/langgraph.mdc b/.cursor/rules/langgraph.mdc new file mode 100644 index 000000000..160b6e9a6 --- /dev/null +++ b/.cursor/rules/langgraph.mdc @@ -0,0 +1,312 @@ +--- +description: This rule file provides comprehensive best practices for developing with LangGraph, covering code organization, performance, security, testing, and common pitfalls. It offers actionable guidance for developers to build robust and maintainable LangGraph applications. +globs: *.py +--- +# LangGraph Best Practices and Coding Standards + +This document outlines best practices and coding standards for developing with LangGraph. It aims to provide clear, actionable guidance for developers to build robust, maintainable, and scalable LangGraph applications. It covers various aspects, including code organization, common patterns, performance considerations, security, testing, and common pitfalls. + +## Library Information: + +- Name: langgraph +- Tags: ai, ml, llm, python, agent-framework, workflow + +## 1. Code Organization and Structure + +### 1.1. Directory Structure Best Practices + + +my_langgraph_project/ +├── data/ # Datasets, knowledge bases, or other data files. +├── src/ # Source code. +│ ├── components/ # Reusable components (e.g., custom nodes, tools). +│ │ ├── __init__.py +│ │ ├── retrieval.py # Retrieval-related nodes +│ │ ├── tool_selector.py # Logic for selecting which tool to use +│ │ └── ... +│ ├── graphs/ # Graph definitions. +│ │ ├── __init__.py +│ │ ├── customer_support.py # Example: Customer support graph. +│ │ ├── rag_pipeline.py # Example: RAG pipeline graph. +│ │ └── ... +│ ├── utils/ # Utility functions and helpers. +│ │ ├── __init__.py +│ │ ├── config.py # Configuration loading +│ │ ├── logging.py # Logging setup +│ │ └── ... +│ ├── schemas/ # Data schemas and type definitions. +│ │ ├── __init__.py +│ │ ├── agent_state.py # Definition of agent state +│ │ └── ... +│ ├── main.py # Entry point of the application. +│ └── ... +├── tests/ # Unit and integration tests. +│ ├── __init__.py +│ ├── components/ # Tests for custom components. +│ ├── graphs/ # Tests for graph definitions. +│ ├── utils/ # Tests for utility functions. +│ └── ... +├── .env # Environment variables. +├── requirements.txt # Project dependencies. +├── pyproject.toml # Project metadata and build settings +└── README.md # Project documentation. + + +### 1.2. File Naming Conventions + +- Python files: `snake_case.py` (e.g., `customer_support.py`, `retrieval_node.py`). +- Class names: `PascalCase` (e.g., `CustomerSupportGraph`, `RetrievalNode`). +- Variables and functions: `snake_case` (e.g., `user_query`, `process_message`). +- Configuration files: `config.yaml` or `config.json` + +### 1.3. Module Organization Best Practices + +- Group related functionalities into modules (e.g., `components`, `graphs`, `utils`). +- Use `__init__.py` files to make directories packages. +- Keep modules focused and avoid overly large files. +- Use relative imports within modules to avoid naming conflicts. + +### 1.4. Component Architecture Recommendations + +- Design reusable components for common tasks (e.g., data retrieval, text summarization, tool selection). +- Create abstract base classes or interfaces for components to promote code reuse and modularity. +- Use dependency injection to configure components and their dependencies. +- Adhere to the Single Responsibility Principle (SRP) when designing components. + +### 1.5. Code Splitting Strategies + +- Split large graph definitions into smaller, more manageable files. +- Use lazy loading for components that are not immediately needed. +- Consider using a module bundler (e.g., esbuild via a plugin) to optimize bundle size for deployment. +- Break the system down into microservices if warranted by scale and complexity of the overall system. Communicate between microservices using REST or message queues. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +- **State Management Pattern**: Encapsulate the agent state in a dedicated class or data structure to ensure consistency and maintainability. Use LangGraph's `StateGraph` to clearly define the state transitions. +- **Node Pattern**: Define reusable nodes for common tasks such as information retrieval, tool selection, and response generation. +- **Conditional Edge Pattern**: Use conditional edges to implement branching logic based on the agent state or external factors. This makes the graph more dynamic and responsive. +- **Retry Pattern:** Implement retry logic within nodes or edges to handle transient errors or API rate limits. Use exponential backoff to avoid overwhelming failing services. +- **Orchestration Pattern**: Use LangGraph as the orchestrator for complex agentic workflows, delegating specific tasks to specialized components or services. + +### 2.2. Recommended Approaches for Common Tasks + +- **Information Retrieval**: Use LangChain's retrieval chain or custom nodes to fetch relevant information from external sources. +- **Tool Selection**: Implement a tool selection node that dynamically chooses the appropriate tool based on the user query and agent state. +- **Response Generation**: Use LangChain's LLMChain or custom nodes to generate responses based on the retrieved information and agent state. +- **Error Handling:** Implement robust error handling within nodes and edges to gracefully handle exceptions and prevent application crashes. Log all errors and implement monitoring to quickly detect and resolve issues. + +### 2.3. Anti-patterns and Code Smells + +- **Monolithic Graphs**: Avoid creating overly complex graphs with too many nodes and edges. Break them down into smaller, more manageable subgraphs. +- **Hardcoded Values**: Avoid hardcoding values directly into the graph definition. Use configuration files or environment variables to manage configurable parameters. +- **Ignoring Errors**: Always handle exceptions and log errors appropriately. Ignoring errors can lead to unexpected behavior and difficult-to-debug issues. +- **Over-Reliance on Global State**: Minimize the use of global state to avoid unintended side effects and make the application more testable. +- **Lack of Testing**: Thoroughly test all components and graph definitions to ensure they function correctly and handle edge cases. +- **Infinite Loops:** Ensure the conditional edges within the graph are well-defined to avoid infinite loops. + +### 2.4. State Management Best Practices + +- Define a clear and concise agent state schema. +- Use immutable data structures for the agent state to avoid accidental modifications. +- Persist the agent state to a database or other storage medium to support long-running conversations or task executions. Consider using vector databases for efficient retrieval. +- Implement versioning for the agent state schema to support schema migrations. +- Use LangGraph's checkpointing feature to save and restore the agent state. + +### 2.5. Error Handling Patterns + +- Use try-except blocks to catch exceptions within nodes and edges. +- Log all errors and warnings with relevant context information. +- Implement retry logic for transient errors or API rate limits. +- Use fallback mechanisms to gracefully handle unrecoverable errors. +- Centralize error handling logic in a dedicated module or class. +- Implement circuit breaker pattern to prevent cascading failures. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +- **Caching**: Implement caching for frequently accessed data or LLM responses. +- **Batching**: Batch multiple requests to external APIs to reduce latency. +- **Asynchronous Operations**: Use asynchronous operations to perform non-blocking I/O and improve responsiveness. +- **Parallel Processing**: Use multi-threading or multi-processing to parallelize computationally intensive tasks. +- **Graph Optimization**: Optimize the graph structure to minimize the number of nodes and edges. +- **Prompt Optimization**: Carefully design prompts to reduce the number of tokens and improve LLM performance. +- **Reduce LLM calls**: Cache LLM responses when possible. Fine-tune smaller models for specific tasks to reduce latency and cost. + +### 3.2. Memory Management Considerations + +- Monitor memory usage to detect memory leaks or excessive memory consumption. +- Use garbage collection to reclaim unused memory. +- Avoid storing large objects in the agent state. +- Use streaming or lazy loading for large data sets. + +### 3.3. (Not applicable, as LangGraph doesn't directly handle rendering) + +### 3.4. Bundle Size Optimization + +- Use a module bundler (e.g., esbuild) to optimize bundle size. +- Remove unused code and dependencies. +- Use code splitting to load only the necessary code for each route or component. +- Compress the bundle using gzip or Brotli. + +### 3.5. Lazy Loading Strategies + +- Use lazy loading for components that are not immediately needed. +- Load large data sets or models on demand. +- Implement code splitting to load only the necessary code for each graph or component. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities and Prevention + +- **Prompt Injection**: Prevent prompt injection by carefully validating user inputs and sanitizing prompts. +- **Data Leaks**: Protect sensitive data by encrypting it at rest and in transit. +- **Unauthorized Access**: Implement strong authentication and authorization mechanisms to control access to the application and its data. +- **Denial of Service (DoS)**: Implement rate limiting and request filtering to prevent DoS attacks. +- **Code Injection**: Avoid executing arbitrary code based on user inputs to prevent code injection vulnerabilities. +- **API Key Exposure**: Store API keys securely using environment variables or a secrets management system and avoid committing them to version control. + +### 4.2. Input Validation + +- Validate all user inputs to prevent prompt injection and other vulnerabilities. +- Use regular expressions or other validation techniques to ensure that inputs conform to the expected format. +- Sanitize inputs to remove potentially harmful characters or code. +- Enforce input length limits to prevent buffer overflows. + +### 4.3. Authentication and Authorization + +- Use strong authentication mechanisms (e.g., multi-factor authentication) to verify user identities. +- Implement role-based access control (RBAC) to restrict access to sensitive data and functionality. +- Use secure session management to protect user sessions from hijacking. +- Store passwords securely using hashing and salting. + +### 4.4. Data Protection + +- Encrypt sensitive data at rest and in transit. +- Use secure protocols (e.g., HTTPS) for all API communication. +- Implement data masking to protect sensitive data from unauthorized access. +- Regularly back up data to prevent data loss. +- Comply with relevant data privacy regulations (e.g., GDPR, CCPA). + +### 4.5. Secure API Communication + +- Use HTTPS for all API communication. +- Implement API authentication and authorization. +- Validate API requests and responses. +- Use rate limiting to prevent API abuse. +- Monitor API traffic for suspicious activity. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +- Write unit tests for all components and utility functions. +- Use mocking and stubbing to isolate components during testing. +- Test edge cases and error conditions. +- Aim for high test coverage. + +### 5.2. Integration Testing + +- Write integration tests to verify the interactions between different components. +- Test the integration with external APIs and services. +- Use a test environment that closely resembles the production environment. + +### 5.3. End-to-End Testing + +- Write end-to-end tests to verify the entire application flow. +- Use a testing framework such as Playwright or Selenium to automate end-to-end tests. +- Test the application from the user's perspective. + +### 5.4. Test Organization + +- Organize tests into separate directories for unit tests, integration tests, and end-to-end tests. +- Use descriptive names for test files and test functions. +- Follow a consistent naming convention for test files and test functions. +- Use test suites to group related tests. + +### 5.5. Mocking and Stubbing + +- Use mocking to replace external dependencies with mock objects. +- Use stubbing to replace complex components with simplified versions. +- Use a mocking framework such as pytest-mock to simplify mocking and stubbing. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +- **Incorrect State Management**: Failing to properly manage the agent state can lead to inconsistent behavior and incorrect results. +- **Ignoring Edge Cases**: Neglecting to handle edge cases can cause unexpected errors and application crashes. +- **Over-Engineering**: Over-complicating the graph definition can make it difficult to understand and maintain. +- **Insufficient Testing**: Lack of thorough testing can lead to undetected bugs and application failures. +- **Not Handling Asynchronous Operations Correctly:** LangGraph, and LLMs generally, use async operations, and failing to await these operations will cause unpredictable results. + +### 6.2. Edge Cases + +- **Empty User Inputs**: Handle cases where the user provides empty or invalid inputs. +- **API Rate Limits**: Implement retry logic and rate limiting to handle API rate limits. +- **Unexpected API Responses**: Handle cases where external APIs return unexpected responses. +- **Large Data Sets**: Use streaming or lazy loading to handle large data sets. + +### 6.3. Version-Specific Issues + +- Be aware of compatibility issues between different versions of LangGraph and LangChain. +- Consult the documentation and release notes for any version-specific issues. +- Pin dependencies to specific versions to avoid unexpected behavior. + +### 6.4. Compatibility Concerns + +- Ensure compatibility between LangGraph and other technologies used in the application. +- Test the integration with external APIs and services. +- Use a consistent set of libraries and dependencies. + +### 6.5. Debugging Strategies + +- Use logging to track the execution flow and identify errors. +- Use a debugger to step through the code and inspect variables. +- Use a testing framework to write unit tests and integration tests. +- Use monitoring tools to track performance and identify bottlenecks. +- Visualize the graph structure to understand the flow of execution. + +## 7. Tooling and Environment + +### 7.1. Recommended Tools + +- **IDE**: PyCharm, Visual Studio Code with Python extension. +- **Virtual Environment Manager**: venv, conda. +- **Testing Framework**: pytest. +- **Mocking Framework**: pytest-mock. +- **Linting and Formatting**: pylint, black. +- **Module Bundler**: esbuild via a plugin. +- **CI/CD**: GitHub Actions, GitLab CI. +- **Secrets Management**: HashiCorp Vault, AWS Secrets Manager, Azure Key Vault. +- **Monitoring**: LangSmith, Prometheus, Grafana. + +### 7.2. Build Configuration + +- Use a build system such as `poetry` or `pip` to manage dependencies. +- Use a configuration file such as `pyproject.toml` or `setup.py` to define project metadata and build settings. + +### 7.3. Linting and Formatting + +- Use a linter such as `pylint` or `flake8` to enforce code style and identify potential errors. +- Use a code formatter such as `black` or `autopep8` to automatically format the code. +- Configure the linter and formatter to use a consistent set of rules and settings. + +### 7.4. Deployment + +- Containerize the application using Docker. +- Deploy the application to a cloud platform such as AWS, Azure, or Google Cloud. +- Use a deployment tool such as Terraform or Ansible to automate the deployment process. +- Implement a blue-green deployment strategy to minimize downtime. + +### 7.5. CI/CD + +- Use a CI/CD tool such as GitHub Actions or GitLab CI to automate the testing, building, and deployment processes. +- Configure the CI/CD pipeline to run tests, linters, and formatters. +- Use a code review process to ensure code quality and security. + +## Conclusion + +By following these best practices and coding standards, developers can build robust, maintainable, and scalable LangGraph applications. This will also help with collaboration amongst team members working in the same codebase. \ No newline at end of file diff --git a/.cursor/rules/core.mdc b/.cursor/rules/laravel.mdc similarity index 99% rename from .cursor/rules/core.mdc rename to .cursor/rules/laravel.mdc index 929349fe5..233d212ba 100644 --- a/.cursor/rules/core.mdc +++ b/.cursor/rules/laravel.mdc @@ -1,7 +1,4 @@ --- -alwaysApply: true ---- ---- description: This rule outlines comprehensive best practices for Laravel development, covering coding standards, security, performance, and testing to ensure maintainable, efficient, and secure applications. It provides guidelines for code organization, common patterns, performance considerations, security best practices, testing approaches, common pitfalls, and tooling. globs: *.php --- diff --git a/.cursor/rules/lightgbm.mdc b/.cursor/rules/lightgbm.mdc new file mode 100644 index 000000000..00a947350 --- /dev/null +++ b/.cursor/rules/lightgbm.mdc @@ -0,0 +1,297 @@ +--- +description: This rule file provides comprehensive best practices for LightGBM, covering code organization, performance, security, testing, and common pitfalls to avoid. Adhering to these guidelines will improve the efficiency, reliability, and maintainability of your LightGBM projects. +globs: *.py,*.ipynb +--- +# LightGBM Best Practices + +This document outlines best practices for developing and maintaining LightGBM-based machine learning projects. It covers various aspects, from code organization to performance optimization and security considerations. + +## Library Information: + +- Name: LightGBM +- Tags: ai, ml, machine-learning, python, gradient-boosting + +## 1. Code Organization and Structure + +### 1.1 Directory Structure + + +project_root/ +├── data/ +│ ├── raw/ +│ ├── processed/ +│ └── external/ +├── notebooks/ +│ ├── exploratory_data_analysis.ipynb +│ └── model_evaluation.ipynb +├── src/ +│ ├── __init__.py +│ ├── data/ +│ │ ├── __init__.py +│ │ ├── dataset.py # Data loading and preprocessing logic +│ │ └── features.py # Feature engineering functions +│ ├── models/ +│ │ ├── __init__.py +│ │ ├── train.py # Model training script +│ │ ├── predict.py # Prediction script +│ │ └── model.py # Model definition (if applicable) +│ ├── utils/ +│ │ ├── __init__.py +│ │ ├── logger.py # Logging utilities +│ │ └── helper.py # General helper functions +│ └── visualization/ +│ ├── __init__.py +│ └── plots.py # Custom plotting functions +├── tests/ +│ ├── __init__.py +│ ├── data/ +│ ├── models/ +│ └── utils/ +├── configs/ +│ └── config.yaml +├── reports/ +│ └── figures/ +├── .gitignore +├── README.md +├── pyproject.toml +└── requirements.txt + + +* **data/**: Contains raw, processed, and external data. +* **notebooks/**: Jupyter notebooks for exploration and experimentation. +* **src/**: Source code for data loading, feature engineering, model training, and prediction. +* **tests/**: Unit and integration tests. +* **configs/**: Configuration files (e.g., YAML). +* **reports/**: Generated reports and figures. + +### 1.2 File Naming Conventions + +* Python files: `lowercase_with_underscores.py` +* Jupyter notebooks: `descriptive_name.ipynb` +* Configuration files: `config.yaml` or `config.json` +* Data files: `data_description.csv` or `data_description.parquet` + +### 1.3 Module Organization + +* Group related functions and classes into modules. +* Use clear and descriptive module names. +* Minimize dependencies between modules. + +### 1.4 Component Architecture + +* **Data Layer:** Handles data loading, preprocessing, and feature engineering. +* **Model Layer:** Encapsulates model training, evaluation, and prediction. +* **Service Layer:** Exposes model functionality through an API or interface. +* **Configuration Layer:** Manages configuration parameters and settings. + +### 1.5 Code Splitting + +* Break down complex tasks into smaller, more manageable functions. +* Use classes to encapsulate related data and behavior. +* Avoid long functions and deeply nested code blocks. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns + +* **Factory Pattern:** To create different LightGBM models based on configuration. +* **Strategy Pattern:** To implement different feature engineering techniques. +* **Observer Pattern:** To monitor training progress and log metrics. + +### 2.2 Recommended Approaches + +* **Data Preparation:** Utilize LightGBM's native support for missing values and categorical features. Convert categorical features to the `categorical` data type in pandas before splitting the data. +* **Hyperparameter Tuning:** Use techniques like grid search, random search, or Bayesian optimization (e.g., Optuna) to optimize hyperparameters. Implement early stopping to prevent overfitting. +* **Model Monitoring:** Track training and validation performance metrics to detect overfitting and adjust model complexity. +* **Feature Selection:** Use feature importance to identify and select relevant features. +* **Cross-Validation:** Use k-fold cross-validation for robust model evaluation. The `cv` function provides mean metrics and standard deviations across folds. + +### 2.3 Anti-patterns and Code Smells + +* **Hardcoding hyperparameters:** Avoid hardcoding; use configuration files. +* **Ignoring missing values:** LightGBM handles missing values, so explicitly using them is fine, but ensure you understand their impact. +* **Overfitting:** Monitor training vs. validation performance and use regularization. +* **Large, monolithic functions:** Break down into smaller, testable units. +* **Ignoring feature importances:** Use feature importance to help drive feature selection and understand your model. + +### 2.4 State Management + +* Use configuration files to manage model parameters and training settings. +* Store model artifacts (e.g., trained models, scalers) in a designated directory. +* Use version control to track changes to code and data. + +### 2.5 Error Handling + +* Use `try-except` blocks to handle potential exceptions. +* Log errors and warnings using a logging library (e.g., `logging`). +* Provide informative error messages to the user. +* Implement retry mechanisms for transient errors. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +* **Hyperparameter Tuning:** Optimize hyperparameters such as learning rate, number of leaves, and tree depth. +* **Early Stopping:** Implement early stopping to prevent overfitting and reduce training time. +* **Parallel Training:** Enable parallel training for faster computations (data/feature/voting parallel). +* **GPU Acceleration:** Enable GPU usage for accelerated training when possible. +* **Feature Selection:** Remove irrelevant or redundant features to improve performance. +* **Reduce data size:** Consider downcasting numerical types (e.g., float64 to float32) if precision loss is acceptable. +* **Histogram-based algorithms:** LightGBM uses histogram-based algorithms for faster training on large datasets. + +### 3.2 Memory Management + +* **`max_bin` Parameter:** Reduce `max_bin` to decrease memory usage (may impact accuracy). +* **`save_binary` Parameter:** Use `save_binary` to save data in a binary format for faster loading and reduced memory usage. +* **Data Types:** Use appropriate data types to minimize memory footprint (e.g., `int32` instead of `int64`). +* **Garbage Collection:** Explicitly call `gc.collect()` to free up unused memory. + +### 3.3 Bundle Size Optimization (If applicable for deployment) + +* Remove unnecessary dependencies. +* Use code minification and compression techniques. +* Optimize image assets. + +### 3.4 Lazy Loading + +* Load data and models only when needed. +* Use generators to process large datasets in chunks. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities + +* **Untrusted Input:** Vulnerable to injection attacks if model is used directly on user-provided data without sanitization. +* **Model Poisoning:** If training data is sourced from untrusted sources, the model can be poisoned. +* **Denial of Service (DoS):** Malicious input crafted to consume excessive resources. + +### 4.2 Input Validation + +* Validate input data to ensure it conforms to expected types and ranges. +* Sanitize input data to prevent injection attacks. +* Use a schema validation library (e.g., `cerberus`, `jsonschema`). + +### 4.3 Authentication and Authorization + +* Implement authentication to verify the identity of users. +* Use authorization to control access to resources and functionality. +* Use secure protocols (e.g., HTTPS) for API communication. + +### 4.4 Data Protection + +* Encrypt sensitive data at rest and in transit. +* Use data masking to protect sensitive information. +* Implement access controls to restrict access to data. + +### 4.5 Secure API Communication + +* Use HTTPS for all API communication. +* Implement input validation and sanitization. +* Use rate limiting to prevent abuse. +* Monitor API traffic for suspicious activity. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +* Test individual functions and classes in isolation. +* Use mocking and stubbing to isolate dependencies. +* Write tests for different input scenarios and edge cases. +* Verify expected outputs and side effects. + +### 5.2 Integration Testing + +* Test interactions between different components. +* Verify data flow and consistency. +* Test API endpoints and data pipelines. + +### 5.3 End-to-End Testing + +* Test the entire application flow from start to finish. +* Simulate real-world user scenarios. +* Verify that the application meets all requirements. + +### 5.4 Test Organization + +* Organize tests into separate directories based on component. +* Use clear and descriptive test names. +* Follow a consistent testing style. + +### 5.5 Mocking and Stubbing + +* Use mocking to replace external dependencies with controlled substitutes. +* Use stubbing to provide predefined responses for function calls. +* Use a mocking library (e.g., `unittest.mock`, `pytest-mock`). + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +* **Data Leakage:** Accidentally using future information during training. +* **Incorrect Feature Scaling:** Using inappropriate scaling methods. +* **Ignoring Categorical Features:** Not treating categorical features correctly. +* **Not tuning for imbalanced classes:** Ignoring the need to adjust for imbalanced datasets. +* **Improper Cross-Validation:** Setting up cross-validation incorrectly. + +### 6.2 Edge Cases + +* **Rare Categories:** Handling infrequent categorical values. +* **Missing Data Patterns:** Understanding the nature and impact of missing data. +* **Outliers:** Detecting and handling extreme values in your dataset. + +### 6.3 Version-Specific Issues + +* Refer to the LightGBM release notes for known issues and bug fixes. +* Stay up-to-date with the latest version of LightGBM. + +### 6.4 Compatibility Concerns + +* Ensure compatibility between LightGBM and other libraries (e.g., scikit-learn, pandas). +* Address potential conflicts between different versions of dependencies. + +### 6.5 Debugging Strategies + +* Use a debugger (e.g., `pdb`) to step through code and inspect variables. +* Log intermediate values and execution paths. +* Use assertions to verify expected behavior. +* Simplify the problem by isolating components. +* Consult the LightGBM documentation and community forums. + +## 7. Tooling and Environment + +### 7.1 Recommended Tools + +* **Python:** The primary language for LightGBM development. +* **pandas:** For data manipulation and analysis. +* **NumPy:** For numerical computations. +* **scikit-learn:** For machine learning algorithms and tools. +* **Jupyter Notebook:** For interactive development and experimentation. +* **IDE:** VSCode, PyCharm, or similar. +* **Optuna/Hyperopt:** For hyperparameter optimization. + +### 7.2 Build Configuration + +* Use a build system (e.g., `setuptools`, `poetry`) to manage dependencies and build packages. +* Create a `requirements.txt` file to list project dependencies. +* Use a virtual environment to isolate project dependencies. + +### 7.3 Linting and Formatting + +* Use a linter (e.g., `flake8`, `pylint`) to enforce code style and identify potential errors. +* Use a formatter (e.g., `black`, `autopep8`) to automatically format code. +* Configure the IDE to automatically run linters and formatters. + +### 7.4 Deployment Best Practices + +* Containerize the application using Docker. +* Deploy the application to a cloud platform (e.g., AWS, Azure, GCP). +* Use a deployment tool (e.g., Kubernetes, Docker Compose). +* Monitor the application for performance and errors. + +### 7.5 CI/CD Integration + +* Use a CI/CD pipeline (e.g., Jenkins, GitLab CI, GitHub Actions) to automate testing and deployment. +* Run unit and integration tests in the CI/CD pipeline. +* Deploy code to staging and production environments. + +By adhering to these best practices, you can develop robust, efficient, and maintainable LightGBM-based machine learning projects. \ No newline at end of file diff --git a/.cursor/rules/llama-index.mdc b/.cursor/rules/llama-index.mdc new file mode 100644 index 000000000..fa0f6e149 --- /dev/null +++ b/.cursor/rules/llama-index.mdc @@ -0,0 +1,158 @@ +--- +description: This rule outlines best practices and coding standards for developing with LlamaIndex, covering code organization, performance, security, testing, and common pitfalls. It aims to ensure maintainable, efficient, and secure LlamaIndex applications. +globs: *.py +--- +# LlamaIndex Best Practices and Coding Standards + +This document provides comprehensive guidance on developing high-quality applications using LlamaIndex. It covers various aspects of development, including code organization, performance optimization, security considerations, and testing strategies. + +## 1. Code Organization and Structure + +* **Directory Structure Best Practices:** + * `data/`: Store data sources (e.g., documents, PDFs) used by LlamaIndex. + * `indices/`: Contains index definitions and configurations. + * `queries/`: Defines query engines and query logic. + * `models/`: Place custom LLM or embedding model configurations. + * `utils/`: Utility functions and helper classes. + * `tests/`: Unit, integration, and end-to-end tests. + * `scripts/`: Scripts for data ingestion, index building, or other automation tasks. +* **File Naming Conventions:** + * Data loaders: `*_loader.py` (e.g., `pdf_loader.py`) + * Index definitions: `*_index.py` (e.g., `vector_index.py`) + * Query engines: `*_query_engine.py` (e.g., `knowledge_graph_query_engine.py`) + * Models: `*_model.py` (e.g., `custom_llm_model.py`) + * Utilities: `*_utils.py` (e.g., `text_processing_utils.py`) + * Tests: `test_*.py` (e.g., `test_vector_index.py`) +* **Module Organization:** + * Group related functionalities into modules (e.g., `data_ingestion`, `indexing`, `querying`). + * Use clear and descriptive module names. + * Minimize dependencies between modules to improve maintainability. +* **Component Architecture:** + * **Data Connectors:** Abstract data loading logic into reusable connectors. + * **Index Structures:** Use appropriate index structures (e.g., `VectorStoreIndex`, `KnowledgeGraphIndex`) based on data characteristics and query requirements. + * **Query Engines:** Decouple query logic from index structures. + * **LLM Abstraction:** Abstract LLM calls using interfaces for flexibility and testability. +* **Code Splitting:** + * Break down large functions into smaller, well-defined functions. + * Use classes to encapsulate related data and behavior. + * Extract reusable code into separate modules or packages. + +## 2. Common Patterns and Anti-patterns + +* **Design Patterns:** + * **Factory Pattern:** For creating different types of indexes or query engines. + * **Strategy Pattern:** For choosing different retrieval or ranking algorithms. + * **Decorator Pattern:** For adding pre-processing or post-processing steps to queries. +* **Recommended Approaches:** + * **Data Ingestion:** Use `SimpleDirectoryReader` or custom data connectors to load data. + * **Indexing:** Choose the appropriate index type based on your data and query needs. Consider `VectorStoreIndex` for semantic search, `KnowledgeGraphIndex` for knowledge graph-based queries, and `ComposableGraph` for combining multiple indexes. + * **Querying:** Use `as_query_engine()` to create a query engine from an index. Customize the query engine with different retrieval and response synthesis modules. + * **Evaluation:** Use LlamaIndex's evaluation modules to measure the performance of your LLM application (e.g., retrieval and LLM response quality). +* **Anti-patterns and Code Smells:** + * **Tight Coupling:** Avoid tight coupling between components. Use interfaces and dependency injection to promote loose coupling. + * **God Classes:** Avoid creating large classes that do too much. Break them down into smaller, more focused classes. + * **Code Duplication:** Avoid duplicating code. Extract common code into reusable functions or classes. + * **Ignoring Errors:** Don't ignore errors. Handle them gracefully or raise exceptions. +* **State Management:** + * Use LlamaIndex's `StorageContext` to persist indexes to disk. + * Consider using a database to store application state. +* **Error Handling:** + * Use `try-except` blocks to handle exceptions. + * Log errors for debugging purposes. + * Provide informative error messages to the user. + +## 3. Performance Considerations + +* **Optimization Techniques:** + * **Indexing:** Optimize index construction by using appropriate chunk sizes and overlap. + * **Querying:** Optimize query performance by using appropriate retrieval and ranking algorithms. + * **Caching:** Cache query results to improve performance. + * **Parallelization:** Parallelize data loading and indexing tasks. +* **Memory Management:** + * Use generators to process large datasets in chunks. + * Release memory when it is no longer needed. +* **Bundle Size Optimization:** (Not directly applicable to LlamaIndex as it is a backend library, but relevant if building a web UI on top) + * Remove unused code. + * Use code splitting to load only the code that is needed. +* **Lazy Loading:** + * Load data and models only when they are needed. + * Use lazy initialization to defer the creation of objects until they are first used. + +## 4. Security Best Practices + +* **Common Vulnerabilities:** + * **Prompt Injection:** Prevent prompt injection attacks by carefully sanitizing user input and using appropriate prompt engineering techniques. + * **Data Leaks:** Protect sensitive data by using appropriate access control and encryption. + * **API Key Exposure:** Avoid exposing API keys in your code. Use environment variables or a secure configuration management system to store API keys. +* **Input Validation:** + * Validate all user input to prevent injection attacks. + * Sanitize input to remove potentially harmful characters. +* **Authentication and Authorization:** + * Implement authentication and authorization to control access to your application. + * Use strong passwords and multi-factor authentication. +* **Data Protection:** + * Encrypt sensitive data at rest and in transit. + * Use appropriate access control to protect data. +* **Secure API Communication:** + * Use HTTPS to encrypt communication between your application and the LlamaIndex API. + * Validate the server certificate to prevent man-in-the-middle attacks. + +## 5. Testing Approaches + +* **Unit Testing:** + * Write unit tests for all core components, including data connectors, index structures, and query engines. + * Use mocking and stubbing to isolate components during testing. +* **Integration Testing:** + * Write integration tests to verify that different components work together correctly. + * Test the integration between LlamaIndex and other libraries or frameworks. +* **End-to-end Testing:** + * Write end-to-end tests to verify that the entire application works as expected. + * Test the application with real data and user scenarios. +* **Test Organization:** + * Organize tests into separate directories for unit, integration, and end-to-end tests. + * Use clear and descriptive test names. +* **Mocking and Stubbing:** + * Use mocking and stubbing to isolate components during testing. + * Use a mocking framework such as `unittest.mock` or `pytest-mock`. + +## 6. Common Pitfalls and Gotchas + +* **Frequent Mistakes:** + * Using the wrong index type for the data. + * Not optimizing query performance. + * Not handling errors gracefully. + * Exposing API keys. + * Not validating user input. +* **Edge Cases:** + * Handling large documents. + * Handling noisy or incomplete data. + * Handling complex queries. +* **Version-Specific Issues:** + * Be aware of breaking changes in LlamaIndex releases. + * Refer to the LlamaIndex documentation for version-specific information. +* **Compatibility Concerns:** + * Ensure that LlamaIndex is compatible with the other libraries and frameworks that you are using. + * Test your application thoroughly to identify any compatibility issues. +* **Debugging Strategies:** + * Use logging to track the execution of your application. + * Use a debugger to step through your code and inspect variables. + * Use LlamaIndex's debugging tools to diagnose issues. + +## 7. Tooling and Environment + +* **Recommended Development Tools:** + * **IDE:** VS Code, PyCharm + * **Package Manager:** Poetry, pip + * **Testing Framework:** pytest + * **Linting and Formatting:** flake8, black +* **Build Configuration:** + * Use a build system such as `poetry` to manage dependencies. + * Create a `requirements.txt` file to list dependencies. +* **Linting and Formatting:** + * Use a linter such as `flake8` to enforce code style. + * Use a formatter such as `black` to automatically format code. +* **Deployment Best Practices:** + * Use a containerization technology such as Docker to package your application. + * Use a cloud platform such as AWS, Azure, or GCP to deploy your application. +* **CI/CD Integration:** + * Use a CI/CD system such as GitHub Actions or Jenkins to automate the build, test, and deployment process. \ No newline at end of file diff --git a/.cursor/rules/llamaindex-js.mdc b/.cursor/rules/llamaindex-js.mdc new file mode 100644 index 000000000..4c3eed29b --- /dev/null +++ b/.cursor/rules/llamaindex-js.mdc @@ -0,0 +1,294 @@ +--- +description: This rule provides comprehensive guidelines for developing AI applications with LlamaIndex-JS, covering code organization, performance, security, and testing best practices. It aims to ensure robust, efficient, and secure LLM-powered applications. +globs: *.js,*.ts,*.jsx,*.tsx +--- +# LlamaIndex-JS Best Practices and Coding Standards + +This document outlines the best practices and coding standards for developing AI and ML applications using the LlamaIndex-JS library. Following these guidelines will help you build robust, efficient, secure, and maintainable applications. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure + +Adopt a modular directory structure to organize your LlamaIndex-JS project. Here’s a recommended structure: + + +my-llamaindex-app/ +├── src/ +│ ├── components/ # Reusable UI components (if applicable) +│ │ ├── MyComponent.tsx +│ │ └── ... +│ ├── services/ # API services and data fetching logic +│ │ ├── llamaIndexService.ts # LlamaIndex specific functionalities +│ │ └── ... +│ ├── utils/ # Utility functions and helpers +│ │ ├── dataProcessing.ts +│ │ └── ... +│ ├── index/ # Index management +│ │ ├── documentLoaders.ts # Custom document loaders +│ │ ├── indexBuilders.ts # Logic for building indices +│ │ └── ... +│ ├── prompts/ # Custom prompt templates +│ │ ├── summarizationPrompt.ts +│ │ └── ... +│ ├── models/ # Data models and interfaces +│ │ ├── document.ts +│ │ └── ... +│ ├── app.ts # Main application entry point +│ └── ... +├── tests/ +│ ├── unit/ +│ │ └── ... +│ ├── integration/ +│ │ └── ... +│ └── ... +├── .env # Environment variables +├── package.json +├── tsconfig.json # TypeScript configuration +├── README.md +└── ... + + +### 1.2. File Naming Conventions + +* Use descriptive and consistent file names. +* For components, use PascalCase (e.g., `MyComponent.tsx`). +* For utility functions and services, use camelCase (e.g., `llamaIndexService.ts`, `dataProcessing.ts`). +* For configuration files, use kebab-case (e.g., `tsconfig.json`). + +### 1.3. Module Organization + +* **Encapsulation:** Group related functions, classes, and interfaces into modules. +* **Single Responsibility Principle:** Each module should have a clear and specific purpose. +* **Explicit Exports:** Use explicit `export` statements to define the public API of each module. +* **Avoid Circular Dependencies:** Be mindful of circular dependencies between modules, as they can lead to runtime errors and make code harder to understand. + +### 1.4. Component Architecture (If Applicable) + +* If your LlamaIndex-JS application includes a user interface (e.g., using React), follow a component-based architecture. +* **Presentational Components:** Focus on rendering UI elements and receiving data as props. +* **Container Components:** Handle data fetching, state management, and logic. +* **Component Composition:** Build complex UIs by composing smaller, reusable components. + +### 1.5. Code Splitting + +* Use dynamic imports (`import()`) to split your code into smaller chunks. +* Lazy-load components or modules that are not immediately needed. +* This can significantly improve initial load time and reduce the overall bundle size. + +typescript +// Example of lazy loading a module +async function loadMyModule() { + const myModule = await import('./myModule'); + myModule.doSomething(); +} + + +## 2. Common Patterns and Anti-Patterns + +### 2.1. Design Patterns + +* **Retrieval-Augmented Generation (RAG):** Implement RAG to enhance LLM responses by retrieving relevant data from external sources. +* **Factory Pattern:** Use factory functions to create instances of LlamaIndex objects, abstracting the creation logic. +* **Strategy Pattern:** Employ different indexing or query strategies based on the specific use case. +* **Observer Pattern:** Use this to react to changes in the underlying data or model. + +### 2.2. Recommended Approaches for Common Tasks + +* **Data Loading:** Use `Document` objects and `BaseReader` classes for loading data from various sources. +* **Indexing:** Choose appropriate index types (e.g., `VectorStoreIndex`, `SummaryIndex`) based on your data and query requirements. +* **Querying:** Use `QueryEngine` to orchestrate complex queries and retrieve relevant information. +* **Evaluation:** Implement evaluation metrics to measure the performance of your RAG pipeline. + +### 2.3. Anti-Patterns and Code Smells + +* **Tight Coupling:** Avoid tight coupling between LlamaIndex-JS components and other parts of your application. +* **Global State:** Minimize the use of global state, as it can make your application harder to reason about. +* **Ignoring Errors:** Always handle errors gracefully and provide informative error messages. +* **Over-Complicating Queries:** Keep queries simple and focused on retrieving the most relevant information. +* **Not using `async/await`:** Use `async/await` when dealing with asynchronous operations to avoid callback hell. + +### 2.4. State Management + +* Choose a state management library or pattern that fits your application's needs (e.g., React Context, Redux, Zustand). +* Keep state minimal and derive values as needed. +* Use immutable data structures to simplify state updates. + +### 2.5. Error Handling + +* Use `try...catch` blocks to handle exceptions. +* Provide informative error messages to the user. +* Log errors for debugging purposes. +* Consider using a centralized error handling mechanism. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +* **Efficient Indexing:** Optimize indexing by selecting appropriate chunk sizes and embedding models. Experiment to find optimal values. +* **Caching:** Cache frequently accessed data to reduce latency. +* **Asynchronous Operations:** Use asynchronous operations to avoid blocking the main thread. +* **Vector Store Selection:** Choose vector stores that support efficient similarity search (e.g., FAISS, Annoy, Qdrant). + +### 3.2. Memory Management + +* Be mindful of memory usage, especially when dealing with large datasets. +* Use garbage collection effectively. +* Avoid creating unnecessary objects. +* Use streams for processing large files. + +### 3.3. Rendering Optimization (If Applicable) + +* Use virtualization techniques to render large lists efficiently. +* Memoize components to prevent unnecessary re-renders. +* Optimize images and other assets. + +### 3.4. Bundle Size Optimization + +* Use tree shaking to remove unused code. +* Minify and compress your code. +* Use code splitting to load only the code that is needed for each page. +* Analyze bundle size using tools like Webpack Bundle Analyzer. + +### 3.5. Lazy Loading + +* Implement lazy loading for non-critical components or modules. +* This can improve initial load time and reduce the overall bundle size. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities + +* **Prompt Injection:** Protect against prompt injection attacks by sanitizing user input and validating LLM responses. Consider using libraries such as LLM Guard by Protect AI. +* **Data Leakage:** Prevent data leakage by carefully controlling access to sensitive information. +* **Cross-Site Scripting (XSS):** Sanitize user input to prevent XSS attacks (if applicable, in UI components). +* **API Key Exposure:** Protect API keys and other sensitive credentials by storing them securely and avoiding hardcoding them in your code. + +### 4.2. Input Validation + +* Validate all user input to prevent malicious data from entering your application. +* Use appropriate validation techniques for each type of input. +* Sanitize input to remove potentially harmful characters. + +### 4.3. Authentication and Authorization + +* Implement authentication to verify the identity of users. +* Implement authorization to control access to resources. +* Use strong passwords and encryption. +* Follow the principle of least privilege. + +### 4.4. Data Protection + +* Encrypt sensitive data at rest and in transit. +* Use secure storage mechanisms. +* Implement data loss prevention (DLP) measures. +* Regularly back up your data. + +### 4.5. Secure API Communication + +* Use HTTPS for all API communication. +* Validate API responses to ensure data integrity. +* Implement rate limiting to prevent abuse. +* Use API keys or other authentication mechanisms. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +* Write unit tests to verify the functionality of individual components and functions. +* Use mocking and stubbing to isolate units of code. +* Aim for high code coverage. + +### 5.2. Integration Testing + +* Write integration tests to verify the interaction between different components and modules. +* Test the integration with external services and APIs. + +### 5.3. End-to-End Testing + +* Write end-to-end tests to verify the overall functionality of the application. +* Simulate user interactions and verify the expected behavior. + +### 5.4. Test Organization + +* Organize your tests into separate directories for unit, integration, and end-to-end tests. +* Use descriptive test names. +* Keep tests concise and focused. + +### 5.5. Mocking and Stubbing + +* Use mocking and stubbing to isolate units of code during testing. +* Use a mocking library such as Jest or Sinon. +* Avoid over-mocking, as it can make your tests less effective. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +* Incorrectly configuring the LlamaIndex-JS client. +* Using the wrong index type for your data. +* Not handling errors properly. +* Ignoring performance considerations. +* Not securing your application against vulnerabilities. + +### 6.2. Edge Cases + +* Handling large documents or datasets. +* Dealing with complex queries. +* Handling different data formats. +* Dealing with rate limits from external APIs. + +### 6.3. Version-Specific Issues + +* Be aware of breaking changes between LlamaIndex-JS versions. +* Consult the release notes and migration guides when upgrading. +* Test your application thoroughly after upgrading. + +### 6.4. Compatibility Concerns + +* Ensure compatibility between LlamaIndex-JS and other libraries or frameworks that you are using. +* Test your application in different environments. + +### 6.5. Debugging Strategies + +* Use a debugger to step through your code and inspect variables. +* Log messages to the console to track the execution flow. +* Use error monitoring tools to track errors in production. +* Use the LlamaIndex-JS documentation and community forums to find solutions to common problems. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +* Node.js +* npm or yarn +* TypeScript (recommended) +* VS Code or other IDE +* LlamaIndex Studio for visualization +* Postman or Insomnia for testing APIs + +### 7.2. Build Configuration + +* Use a build tool such as Webpack or Parcel to bundle your code. +* Configure your build tool to optimize your code for production. +* Use environment variables to configure your application. + +### 7.3. Linting and Formatting + +* Use a linter such as ESLint to enforce coding standards. +* Use a formatter such as Prettier to format your code consistently. +* Configure your IDE to automatically lint and format your code. + +### 7.4. Deployment + +* Choose a deployment platform that meets your needs (e.g., Vercel, Netlify, AWS). +* Configure your deployment environment to use environment variables. +* Set up monitoring and logging to track the performance of your application. + +### 7.5. CI/CD Integration + +* Use a CI/CD platform such as GitHub Actions or Jenkins to automate your build, test, and deployment processes. +* Configure your CI/CD pipeline to run your tests automatically. +* Use automated deployment to deploy your application to production. + +By following these best practices, you can build robust, efficient, secure, and maintainable AI applications using LlamaIndex-JS. \ No newline at end of file diff --git a/.cursor/rules/llvm.mdc b/.cursor/rules/llvm.mdc new file mode 100644 index 000000000..9f63ab6df --- /dev/null +++ b/.cursor/rules/llvm.mdc @@ -0,0 +1,131 @@ +--- +description: This rule enforces LLVM's coding standards and promotes best practices for writing efficient, maintainable, and robust code within the LLVM ecosystem. It covers style, language usage, optimization strategies, and more. +globs: *.h,*.hpp,*.c,*.cc,*.cpp,*.ll,*.mlir +--- +- Adhere to LLVM's coding standards to ensure code consistency, readability, and maintainability. +- Always follow the [LLVM Coding Standards](https://llvm.org/docs/CodingStandards.html) and [LLVM Developer Policy](https://llvm.org/docs/DeveloperPolicy.html). + +## Languages, Libraries, and Standards + +- Use modern, standard-conforming, and portable C++ code (C++17 or later, as supported by major toolchains). +- Prefer LLVM's own libraries (e.g., `llvm::DenseMap`, `llvm::SmallVector`) over the C++ standard library when appropriate for performance or integration benefits. Consult the LLVM Programmer's Manual for details. +- Use Python (with the minimum version specified in the Getting Started documentation) for automation, build systems, and utility scripts. +- Format Python code with `black` and `darker` to adhere to PEP 8 standards. + +## Mechanical Source Issues + +- Write clear diagnostic messages using succinct and correct English. +- Follow the recommended `#include` style: Main module header first, followed by LLVM project headers (most specific to least specific), then system headers. +- Limit source code width to 80 columns. +- Prefer spaces to tabs in source files. +- Avoid trailing whitespace. +- Format multi-line lambdas like blocks of code. +- Use braced initializer lists as if the braces were parentheses in a function call. + +## Language and Compiler Issues + +- Treat compiler warnings as errors. +- Write portable code, encapsulating non-portable code behind well-defined interfaces. +- Do not use RTTI or exceptions. +- Prefer C++-style casts (`static_cast`, `reinterpret_cast`, `const_cast`) over C-style casts, except when casting to `void` to suppress unused variable warnings or between integral types. +- Do not use static constructors (global variables with constructors or destructors). +- Use `struct` when all members are public, and `class` otherwise. +- Avoid using braced initializer lists to call constructors with non-trivial logic. +- Use `auto` type deduction to make code more readable, but be mindful of unnecessary copies. +- Beware of non-determinism due to the ordering of pointers in unordered containers. +- Beware of non-deterministic sorting order of equal elements; use `llvm::sort` instead of `std::sort`. + +## Style Issues + +- Layer libraries correctly to avoid circular dependencies. Use Unix linker principles to enforce this. +- `#include` as little as possible, especially in header files. Use forward declarations when possible. +- Use namespace qualifiers to implement previously declared functions in source files; avoid opening namespace blocks. +- Use early exits and `continue` statements to simplify code and reduce indentation. +- Don't use `else` or `else if` after control flow interrupting statements like `return`, `break`, `continue`, or `goto`. +- Turn predicate loops into predicate functions. + +## The Low-Level Issues + +- Name types, functions, variables, and enumerators properly, using descriptive and consistent names. +- Assert liberally to check preconditions and assumptions. Include informative error messages in assertions. Use `llvm_unreachable` to mark code that should never be reached. Avoid `assert(false)`. When assertions produce “unused value” warnings, move the call into the assert or cast the value to void. +- Do not use `using namespace std`. Explicitly qualify identifiers from the standard namespace. +- Don't use default labels in fully covered switches over enumerations to catch new enumeration values. Use `llvm_unreachable` after the switch to suppress GCC warnings about reaching the end of a non-void function. +- Use range-based for loops wherever possible. +- Don’t evaluate `end()` every time through a loop; evaluate it once before the loop starts. +- `#include ` is forbidden. Use `raw_ostream` instead. +- Avoid `std::endl`; use `\n` instead. +- Don’t use `inline` when defining a function in a class definition; it's implicit. + +## Microscopic Details + +- Put a space before an open parenthesis only in control flow statements, but not in normal function call expressions and function-like macros. +- Prefer preincrement (`++X`) over postincrement (`X++`). +- Don't indent namespaces; add comments indicating which namespace is being closed when helpful. + +## Code Organization and Structure + +- **Directory Structure:** Follow the existing LLVM directory structure conventions. New components should typically reside in a subdirectory within an existing high-level category (e.g., `lib/Transforms`, `include/llvm/Analysis`). Consider the intended audience (end-user tool vs. internal library) when choosing a location. +- **File Naming:** Use descriptive names for files, matching the class or functionality they contain. Header files should have a `.h` or `.hpp` extension, and implementation files should have a `.cpp`, `.cc`, or `.c` extension (depending on whether they are C or C++). MLIR files should have `.mlir` extension. +- **Module Organization:** Break down large components into smaller, more manageable modules. Each module should have a well-defined purpose and interface. Minimize dependencies between modules. +- **Component Architecture:** Design components with clear separation of concerns. Use interfaces and abstract classes to promote loose coupling and enable polymorphism. Follow the principle of least privilege. +- **Code Splitting:** Decompose large functions into smaller, well-named helper functions. Use namespaces to group related functions and classes. + +## Common Patterns and Anti-patterns + +- **Visitor Pattern:** LLVM frequently uses the Visitor pattern for traversing and manipulating data structures (e.g., the LLVM IR). Implement visitor classes for custom analysis or transformations. +- **Pass Infrastructure:** Leverage LLVM's pass infrastructure for implementing optimization and analysis passes. Create new pass types (e.g., analysis pass, transformation pass) as needed. +- **Factory Pattern:** Use the Factory pattern to create instances of classes based on runtime parameters or configuration settings. +- **Singleton Pattern (Avoid):** Minimize the use of the Singleton pattern, as it can lead to tight coupling and make testing difficult. Consider dependency injection instead. +- **Global Variables (Avoid):** Avoid global variables whenever possible, as they can introduce unexpected side effects and make code harder to reason about. Use configuration objects or dependency injection instead. + +## Performance Considerations + +- **Inline Hints:** Use `inline` judiciously to guide the compiler to inline frequently called functions. Avoid inlining large or complex functions. +- **Profile-Guided Optimization (PGO):** Use PGO to optimize code based on runtime profiles. This can significantly improve performance for frequently executed code paths. +- **LTO (Link-Time Optimization):** Enable LTO to allow the compiler to optimize code across module boundaries. +- **Memory Management:** Use LLVM's memory management facilities (e.g., `BumpPtrAllocator`, `ScopedHashTable`) for efficient memory allocation. Avoid excessive memory allocation and deallocation. +- **Avoid Unnecessary Copies:** Pass objects by reference or pointer to avoid unnecessary copying. +- **Use `SmallVector`:** Use `llvm::SmallVector` for small, fixed-size vectors to avoid dynamic memory allocation. +- **Cache Locality:** Consider cache locality when designing data structures and algorithms. + +## Security Best Practices + +- **Input Validation:** Validate all external inputs to prevent vulnerabilities such as buffer overflows and code injection. Use LLVM's API for parsing various file formats. +- **Sanitizers:** Use AddressSanitizer (ASan), MemorySanitizer (MSan), and UndefinedBehaviorSanitizer (UBSan) during development and testing to detect memory errors and undefined behavior. +- **Avoid Code Injection:** Do not construct code by concatenating strings or using other techniques that could lead to code injection vulnerabilities. +- **Data Validation:** Make sure to validate any data being retrieved from memory or other storage locations. Use `llvm::isSafeToLoadUnconditionally` to check if data is safe to load. + +## Testing Approaches + +- **Unit Tests:** Write unit tests for individual components and functions. Use the LLVM testing framework (e.g., `lit`) to automate test execution. +- **Integration Tests:** Write integration tests to verify the interaction between different components. Create realistic test cases that exercise common use scenarios. +- **Regression Tests:** Add regression tests for any bug fixes to prevent regressions in future versions. +- **Fuzzing:** Use fuzzing techniques to identify corner cases and potential vulnerabilities. Integrate fuzzing into the CI/CD pipeline. +- **Code Coverage:** Measure code coverage to ensure that tests exercise all important code paths. Use `llvm-cov` to generate code coverage reports. + +## Common Pitfalls and Gotchas + +- **Incorrect Use of APIs:** Carefully read the documentation for LLVM APIs and ensure they are used correctly. Pay attention to preconditions and postconditions. +- **Memory Leaks:** Ensure that all allocated memory is properly deallocated. Use memory leak detection tools to identify memory leaks. +- **Dangling Pointers:** Avoid dangling pointers by ensuring that objects are not deleted while they are still being referenced. +- **Undefined Behavior:** Avoid undefined behavior, as it can lead to unpredictable results. Use sanitizers to detect undefined behavior. +- **Version Compatibility:** Be aware of version compatibility issues when using LLVM with other libraries or tools. Test your code with different versions of LLVM. + +## Tooling and Environment + +- **Clang:** Use Clang as the primary compiler for LLVM projects. Clang provides excellent support for C++ standards and LLVM extensions. +- **CMake:** Use CMake to manage the build process for LLVM projects. CMake is a cross-platform build system generator that is well-supported by LLVM. +- **LLVM Tools:** Utilize the LLVM tools (e.g., `llvm-as`, `llvm-dis`, `opt`, `lli`) for various tasks such as assembling and disassembling LLVM IR, optimizing code, and executing LLVM bitcode. +- **Git:** Use Git for version control. Follow the LLVM Git commit message conventions. +- **CI/CD:** Integrate CI/CD pipelines into LLVM development. Use test suites as part of your development cycle. + +## Additional Notes + +- When extending or modifying existing code, adhere to the style already in use. +- Document your code clearly and concisely. Provide comments explaining the purpose of functions, classes, and variables. +- Keep pull requests small and focused. This makes it easier for reviewers to understand and approve your changes. +- Be responsive to feedback from reviewers and address any issues promptly. +- Engage with the LLVM community to ask questions, share knowledge, and contribute to the project. +- Refer to existing documentation and examples in the LLVM codebase for guidance on how to implement new features and functionality. + +By following these guidelines, you can write high-quality LLVM code that is efficient, maintainable, and robust. This ensures consistent code quality and facilitates collaboration within the LLVM community. \ No newline at end of file diff --git a/.cursor/rules/material-ui.mdc b/.cursor/rules/material-ui.mdc new file mode 100644 index 000000000..66b0c3425 --- /dev/null +++ b/.cursor/rules/material-ui.mdc @@ -0,0 +1,273 @@ +--- +description: Comprehensive guide to best practices when developing with Material-UI/MUI, covering code organization, performance, security, testing, and common pitfalls. It focuses on creating maintainable, scalable, and performant React applications using MUI components. +globs: *.js,*.jsx,*.ts,*.tsx +--- +# Material-UI/MUI Best Practices and Coding Standards + +This document outlines best practices and coding standards for developing React applications using Material-UI (MUI). Following these guidelines will help you create maintainable, scalable, performant, and secure applications. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure Best Practices + +Adopt a clear and consistent directory structure to improve code maintainability and collaboration. A common approach is to organize code by feature or domain. + + +src/ + components/ + [ComponentName]/ + [ComponentName].jsx # Component implementation + [ComponentName].module.css # Component-specific styles (CSS Modules) + [ComponentName].test.jsx # Unit tests + index.js # (Optional) Export the component + pages/ + [PageName]/ + [PageName].jsx # Page component + index.js # (Optional) Export the page component + styles/ + theme.js # MUI theme configuration + global.css # Global styles + utils/ + api.js # API client + helpers.js # Utility functions + App.jsx # Main application component + index.jsx # Entry point + + +### 1.2 File Naming Conventions + +- **Components:** Use PascalCase for component file names (e.g., `MyButton.jsx`). +- **Styles:** Use camelCase or kebab-case for style file names (e.g., `myButton.module.css` or `my-button.module.css`). Prefer CSS modules. +- **Utilities:** Use camelCase for utility file names (e.g., `api.js`, `helpers.js`). +- **Tests:** Use the `.test.jsx` or `.spec.jsx` suffix for test files (e.g., `MyComponent.test.jsx`). +- **Indexes**: `index.js` should export the main entity contained within its parent folder. + +### 1.3 Module Organization + +- **Component-Specific Modules:** Encapsulate styles, logic, and tests within a component's directory to promote modularity and reusability. +- **Theme Module:** Centralize MUI theme customization in a dedicated module (`theme.js`). +- **Utility Modules:** Group related utility functions into separate modules (e.g., `api.js` for API calls, `helpers.js` for data manipulation). Import selectively only the parts you need from larger modules. + +### 1.4 Component Architecture + +- **Presentational and Container Components:** Separate concerns by creating presentational (UI-focused) and container (data-fetching and state management) components. Consider using hooks for simpler components. +- **Composition over Inheritance:** Favor component composition over inheritance to create flexible and reusable UI elements. +- **Controlled Components:** Use controlled components with explicit state management for better control and predictability. +- **Small Components:** Create smaller, focused components that do one thing well. This promotes reuse and testability. + +### 1.5 Code Splitting Strategies + +- **Route-Based Splitting:** Use React.lazy and Suspense to split your application into smaller chunks that are loaded on demand based on the current route. +- **Component-Based Splitting:** Lazy-load less critical components to reduce the initial bundle size. Useful for complex dialogs, or infrequently used features. +- **Library Splitting:** If certain libraries are used only in specific parts of your application, consider splitting them into separate chunks. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns Specific to Material-UI/MUI + +- **Theming:** Use the `ThemeProvider` to customize the MUI theme globally. Define your color palette, typography, and other theme values in `theme.js`. Use `createTheme` function to extend the default theme. +- **Styling with `sx` prop:** Employ the `sx` prop for simple, one-off style customizations. +- **Styling with `styled` API:** Use the `styled` API for creating reusable, theme-aware components. This is the recommended approach for component styling in MUI v5 and above. +- **Grid System:** Leverage the `Grid` component for creating responsive layouts. +- **Hooks**: Use React hooks extensively for state management and side effects. MUI components work seamlessly with hooks. + +### 2.2 Recommended Approaches for Common Tasks + +- **Form Handling:** Use `TextField` components with proper validation and state management libraries like Formik or React Hook Form. +- **Data Display:** Use `Table`, `List`, and `Card` components to display data in a structured and visually appealing manner. +- **Navigation:** Use `AppBar`, `Drawer`, and `BottomNavigation` components for application navigation. +- **Notifications:** Implement notifications using the `Snackbar` component. +- **Dialogs/Modals**: Use the `Dialog` component to display modal content. + +### 2.3 Anti-patterns and Code Smells to Avoid + +- **Inline Styles:** Avoid inline styles as they are difficult to maintain and do not support theming. Use the `sx` prop or `styled` API instead. While the `sx` prop is quick, prefer `styled` components for reusability. +- **Direct DOM Manipulation:** Avoid directly manipulating the DOM. Use React's state management and component lifecycle methods to update the UI. +- **Over-nesting Components:** Avoid deeply nested component structures as they can impact performance and readability. Refactor into smaller, more focused components. +- **Mutating Theme Directly**: Don't directly mutate the theme object. Use `createTheme` and `ThemeProvider` to apply changes. +- **Ignoring Accessibility:** Ensure your components are accessible by providing proper ARIA attributes and keyboard navigation support. + +### 2.4 State Management Best Practices + +- **Local Component State:** Use `useState` and `useReducer` for managing component-specific state. +- **Global Application State:** Use Context API, Redux, Zustand, or Jotai for managing global application state. +- **Lifting State Up:** Lift state up to the nearest common ancestor component when multiple components need to share state. +- **Immutable Data:** Treat state as immutable and use immutable data structures to prevent unexpected side effects. Libraries like Immer can help. + +### 2.5 Error Handling Patterns + +- **Error Boundaries:** Use error boundaries to catch JavaScript errors in components and prevent the entire application from crashing. +- **Centralized Error Handling:** Implement a centralized error handling mechanism to log errors and display user-friendly error messages. +- **Try-Catch Blocks:** Use try-catch blocks to handle potential errors in asynchronous operations or API calls. +- **Defensive Programming**: Validate props, check for null/undefined values, and handle potential edge cases. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +- **Memoization:** Use `React.memo` to memoize functional components and prevent unnecessary re-renders. Use `useMemo` and `useCallback` hooks to memoize expensive computations and function references. +- **Virtualization:** Use virtualization libraries like `react-window` or `react-virtualized` to efficiently render large lists or tables. +- **Debouncing and Throttling:** Use debouncing and throttling to limit the frequency of expensive operations like API calls or event handlers. +- **Code Splitting:** Implement code splitting to reduce the initial bundle size and improve load times. +- **Image Optimization:** Optimize images by compressing them and using appropriate formats (e.g., WebP). Use lazy loading for images below the fold. +- **CDN**: Host static assets on a CDN. + +### 3.2 Memory Management + +- **Avoid Memory Leaks:** Be mindful of memory leaks, especially in event listeners and subscriptions. Clean up resources in the `useEffect` hook's cleanup function. +- **Garbage Collection:** Understand how JavaScript's garbage collection works and avoid creating unnecessary objects or closures that can lead to memory leaks. + +### 3.3 Rendering Optimization + +- **ShouldComponentUpdate (Class Components):** Implement `shouldComponentUpdate` (or `React.memo` in functional components) to prevent unnecessary re-renders when the props or state have not changed. +- **PureComponent (Class Components):** Extend `PureComponent` for components that rely solely on props for rendering, as it provides a shallow prop comparison. +- **Key Prop:** Always provide a unique `key` prop when rendering lists of components. This helps React efficiently update the DOM. +- **Minimize DOM Updates:** Reduce the number of DOM updates by batching state updates and using techniques like requestAnimationFrame. + +### 3.4 Bundle Size Optimization + +- **Tree Shaking:** Ensure your build process supports tree shaking to remove unused code from your bundle. +- **Minification:** Minify your code to reduce the bundle size. +- **Compression:** Use gzip or Brotli compression to reduce the size of your assets during transmission. +- **Dependency Analysis:** Analyze your dependencies to identify and remove unnecessary libraries. + +### 3.5 Lazy Loading Strategies + +- **React.lazy and Suspense:** Use `React.lazy` and `Suspense` to lazy load components and improve initial load times. +- **Intersection Observer API:** Use the Intersection Observer API to lazy load components when they become visible in the viewport. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and How to Prevent Them + +- **Cross-Site Scripting (XSS):** Sanitize user input to prevent XSS attacks. Use libraries like DOMPurify to sanitize HTML. +- **Cross-Site Request Forgery (CSRF):** Implement CSRF protection by using anti-CSRF tokens. +- **SQL Injection:** Use parameterized queries or ORMs to prevent SQL injection attacks. +- **Authentication and Authorization Issues:** Implement secure authentication and authorization mechanisms to protect sensitive data and resources. +- **Denial of Service (DoS):** Implement rate limiting and other security measures to prevent DoS attacks. + +### 4.2 Input Validation + +- **Client-Side Validation:** Implement client-side validation to provide immediate feedback to the user. +- **Server-Side Validation:** Always validate user input on the server-side to prevent malicious data from being stored in your database. +- **Sanitization:** Sanitize user input to remove or encode potentially harmful characters or code. + +### 4.3 Authentication and Authorization Patterns + +- **JSON Web Tokens (JWT):** Use JWTs for authentication and authorization. +- **OAuth 2.0:** Use OAuth 2.0 for delegating authorization to third-party applications. +- **Role-Based Access Control (RBAC):** Implement RBAC to control access to resources based on user roles. + +### 4.4 Data Protection Strategies + +- **Encryption:** Encrypt sensitive data at rest and in transit. +- **Data Masking:** Mask sensitive data to protect it from unauthorized access. +- **Data Minimization:** Collect only the necessary data to minimize the risk of data breaches. + +### 4.5 Secure API Communication + +- **HTTPS:** Use HTTPS to encrypt communication between the client and the server. +- **API Keys:** Use API keys to authenticate API requests. +- **Rate Limiting:** Implement rate limiting to prevent abuse of your API. +- **Input Validation**: Perform extensive input validation on API endpoints. + +## 5. Testing Approaches + +### 5.1 Unit Testing Strategies + +- **Test Driven Development (TDD):** Consider using TDD to write tests before implementing the code. +- **Component Testing:** Unit test individual components in isolation to ensure they function correctly. +- **Mocking Dependencies:** Mock external dependencies like API calls or third-party libraries to isolate the component under test. +- **Test Coverage:** Aim for high test coverage to ensure that most of your code is tested. + +### 5.2 Integration Testing + +- **Test Component Interactions:** Integration tests verify the interactions between different components. +- **Test Data Flow:** Test the flow of data between components to ensure that data is passed correctly. +- **Mock API Endpoints:** Mock API endpoints to simulate real-world scenarios. + +### 5.3 End-to-End Testing + +- **Simulate User Interactions:** End-to-end tests simulate user interactions to verify that the application functions correctly from the user's perspective. +- **Test Critical User Flows:** Focus on testing critical user flows like login, registration, and checkout. +- **Use Testing Frameworks:** Use testing frameworks like Cypress or Playwright to automate end-to-end tests. + +### 5.4 Test Organization + +- **Colocate Tests with Components:** Place test files in the same directory as the component they test. +- **Use Descriptive Test Names:** Use descriptive test names to clearly indicate what the test is verifying. +- **Organize Tests by Feature:** Organize tests by feature to improve maintainability. + +### 5.5 Mocking and Stubbing + +- **Mock API Calls:** Mock API calls to isolate components during testing. +- **Stub External Dependencies:** Stub external dependencies to control their behavior during testing. +- **Use Mocking Libraries:** Use mocking libraries like Jest or Sinon to create mocks and stubs. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes Developers Make + +- **Not Using the ThemeProvider:** Neglecting to use the `ThemeProvider` for consistent styling. +- **Overriding Styles Incorrectly:** Incorrectly overriding MUI component styles (e.g., using CSS specificity issues). +- **Ignoring Responsiveness:** Failing to design responsive layouts using the `Grid` component. +- **Not Using the Latest Version:** Using older versions of MUI that may have known bugs or security vulnerabilities. +- **Over-reliance on `any` type in Typescript**: Not defining accurate types can lead to runtime errors. + +### 6.2 Edge Cases to Be Aware Of + +- **Server-Side Rendering (SSR):** MUI requires special configuration for SSR to prevent CSS hydration issues. +- **Accessibility (a11y):** Ensure components are accessible by providing proper ARIA attributes and keyboard navigation support. +- **Browser Compatibility:** Test your application in different browsers to ensure compatibility. +- **Internationalization (i18n):** Consider internationalization when designing your application. + +### 6.3 Version-Specific Issues + +- **Breaking Changes:** Be aware of breaking changes when upgrading MUI versions. Refer to the migration guide for each version. +- **Deprecated Features:** Avoid using deprecated features as they may be removed in future versions. + +### 6.4 Compatibility Concerns + +- **React Version:** Ensure that your MUI version is compatible with your React version. +- **Third-Party Libraries:** Be aware of compatibility issues between MUI and other third-party libraries. + +### 6.5 Debugging Strategies + +- **Use Browser Developer Tools:** Use browser developer tools to inspect the DOM, debug JavaScript code, and profile performance. +- **Use React Developer Tools:** Use the React Developer Tools to inspect the component tree, view component props and state, and profile performance. +- **Use Logging Statements:** Use logging statements to trace the execution of your code and identify potential issues. +- **Use a Debugger:** Use a debugger to step through your code and inspect variables. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +- **IDE:** Visual Studio Code (VS Code) with extensions like ESLint, Prettier, and React Developer Tools. +- **Package Manager:** npm, yarn, or pnpm. +- **Build Tool:** Webpack, Parcel, or Rollup. +- **Testing Framework:** Jest or Mocha. +- **Linting and Formatting:** ESLint and Prettier. + +### 7.2 Build Configuration + +- **Webpack Configuration:** Configure Webpack to optimize your bundle size, enable code splitting, and handle assets. +- **Babel Configuration:** Configure Babel to transpile your code to older versions of JavaScript for browser compatibility. + +### 7.3 Linting and Formatting + +- **ESLint:** Configure ESLint to enforce code style and prevent common errors. Use a shared ESLint configuration like Airbnb or Standard. +- **Prettier:** Configure Prettier to automatically format your code. +- **Husky and Lint-Staged:** Use Husky and Lint-Staged to run linters and formatters before committing code. + +### 7.4 Deployment Best Practices + +- **Choose a Hosting Provider:** Choose a reliable hosting provider like Netlify, Vercel, or AWS. +- **Configure Environment Variables:** Configure environment variables for different environments (e.g., development, staging, production). +- **Optimize Assets:** Optimize assets like images and fonts before deploying your application. +- **Use a CDN:** Use a CDN to serve static assets. + +### 7.5 CI/CD Integration + +- **Choose a CI/CD Tool:** Choose a CI/CD tool like GitHub Actions, Jenkins, or CircleCI. +- **Automate Tests:** Automate tests to run automatically on every commit or pull request. +- **Automate Deployment:** Automate deployment to automatically deploy your application to different environments. \ No newline at end of file diff --git a/.cursor/rules/matplotlib.mdc b/.cursor/rules/matplotlib.mdc new file mode 100644 index 000000000..34817371c --- /dev/null +++ b/.cursor/rules/matplotlib.mdc @@ -0,0 +1,306 @@ +--- +description: This rule provides guidelines and best practices for developing robust, maintainable, and performant data visualizations using Matplotlib in Python. It covers aspects from code organization to testing and security considerations. +globs: *.py +--- +# Matplotlib Best Practices + +This document outlines best practices for using Matplotlib, a powerful Python library for creating static, interactive, and animated visualizations. Adhering to these guidelines will help you write cleaner, more maintainable, and performant code for your data visualization projects. + +Library Information: +- Name: matplotlib +- Tags: ai, ml, data-science, python, data-visualization + +## 1. Code Organization and Structure + +Proper code organization is essential for maintainability and collaboration. For visualization projects using Matplotlib, the organization becomes crucial due to the complexity that can arise from numerous plotting options and data manipulation. + +### 1.1 Directory Structure + +Adopt a structured directory layout to separate source code, data, and output. A typical project structure might look like this: + + +project_name/ +├── src/ +│ ├── __init__.py +│ ├── data_processing.py # Data loading, cleaning, and transformation functions +│ ├── plotting_functions.py # Reusable plotting functions using Matplotlib +│ ├── main.py # Main script to orchestrate the visualization +├── data/ +│ ├── raw/ # Original, unmodified data +│ ├── processed/ # Data that has been cleaned and transformed +├── output/ +│ ├── images/ # Saved plot images +│ ├── reports/ # Generated reports or summaries +├── notebooks/ # Jupyter notebooks for exploration +├── tests/ # Testing directory +│ ├── __init__.py +│ ├── test_data_processing.py +│ ├── test_plotting_functions.py +├── README.md +├── requirements.txt +├── .gitignore + + +### 1.2 File Naming Conventions + +* Use descriptive and consistent file names. +* Python files: `snake_case.py` (e.g., `data_processing.py`, `plotting_functions.py`). +* Image files: `descriptive_name.png`, `descriptive_name.jpg`, or `descriptive_name.svg`. +* Data files: `descriptive_name.csv`, `descriptive_name.json`. + +### 1.3 Module Organization + +* Group related functions and classes into modules. +* Use meaningful module names that reflect their purpose (e.g., `data_processing`, `plotting_utils`). +* Import modules using explicit relative imports when working within the same package (e.g., `from . import data_processing`). +* Avoid circular dependencies between modules. + +### 1.4 Component Architecture + +* **Data Abstraction Layer:** Create modules dedicated to data loading, cleaning, and transformation. This isolates the visualization logic from the data source. +* **Plotting Abstraction Layer:** Develop reusable plotting functions that encapsulate common Matplotlib configurations. This promotes code reuse and consistency. +* **Configuration Management:** Store plot configurations (colors, styles, labels) in a separate configuration file (e.g., a JSON or YAML file). This allows for easy modification of plot aesthetics without changing the code. +* **Orchestration Layer:** A main script responsible for calling the data loading, processing and plotting functions and assembling the final visualization, like `main.py` in the example structure above. + +### 1.5 Code Splitting Strategies + +* **Functional Decomposition:** Break down complex plotting tasks into smaller, manageable functions. +* **Class-Based Organization:** Use classes to represent complex plot structures (e.g., a custom chart type) or to manage plot state (see section on State Management below). +* **Separate Data Handling:** Ensure functions which load and clean data are distinct from plotting functionalities. +* **Configuration Driven:** Define plot styles and settings in external configuration files (JSON, YAML) which are then read by plotting functions, allowing easy style modifications. + +## 2. Common Patterns and Anti-patterns + +Recognizing and applying appropriate design patterns can significantly improve the quality of your Matplotlib code. Similarly, awareness of anti-patterns can help you avoid common pitfalls. + +### 2.1 Design Patterns + +* **Object-Oriented API:** Prioritize using Matplotlib's object-oriented API (`fig, ax = plt.subplots()`) over the stateful `pyplot` interface. The object-oriented API provides greater flexibility and control, especially for complex plots. +* **Template Method:** Create a base class for common plot types with abstract methods for data loading and customization. Subclasses can then implement these methods to create specific plots. +* **Strategy Pattern:** Define different plotting strategies as separate classes. The main plotting function can then dynamically choose the appropriate strategy based on the data or user input. +* **Observer Pattern:** Useful in interactive plots where changes in one element (e.g., a slider) trigger updates in other elements of the plot. Matplotlib's event handling system can be leveraged for this pattern. + +### 2.2 Recommended Approaches for Common Tasks + +* **Creating Subplots:** Use `plt.subplots()` to create figures with multiple subplots. Specify the number of rows and columns, and unpack the returned figure and axes objects for individual manipulation. +* **Customizing Plots:** Use the `ax.set()` method to set plot titles, axis labels, legends, and other properties. For more fine-grained control, use methods like `ax.set_xlabel()`, `ax.set_ylabel()`, `ax.set_title()`, etc. +* **Adding Legends:** Use `ax.legend()` to add a legend to the plot. Customize the legend appearance and location as needed. +* **Saving Plots:** Use `fig.savefig()` to save plots to files. Specify the desired file format, resolution (dpi), and other options. +* **Working with Color Maps:** Use `cmap` argument in plotting functions to apply color maps to your data. Consider using perceptually uniform color maps to avoid visual distortions. +* **Handling Date Data:** Matplotlib provides excellent support for date data. Use `matplotlib.dates` module to format and manipulate dates on the axes. Use `dateutil.parser` for flexible date parsing. +* **Interactive plots:** Utilize Matplotlib's interactive capabilities with `plt.ion()` and `plt.ioff()` to update plots dynamically in a loop. Use widgets such as sliders and buttons for user interaction. + +### 2.3 Anti-patterns and Code Smells + +* **Excessive Use of `pyplot` Interface:** Relying heavily on the `pyplot` interface can lead to tightly coupled and less maintainable code. Embrace the object-oriented API for better structure and flexibility. +* **Hardcoding Plot Configurations:** Avoid hardcoding plot configurations (colors, styles, labels) directly in the code. Use configuration files or dictionaries to manage these settings. +* **Copy-Pasting Plotting Code:** Duplicating plotting code across multiple scripts leads to redundancy and makes it difficult to maintain consistency. Create reusable plotting functions instead. +* **Ignoring Performance Considerations:** Creating very complex plots with huge amount of data without considering optimization can result in slow rendering and high memory usage. +* **Not Handling Exceptions:** Failure to handle potential exceptions (e.g., file not found, invalid data) can lead to unexpected program termination. +* **Overplotting:** In scatter plots, overlapping data points can obscure the underlying distribution. Use techniques like transparency (`alpha`) or jitter to mitigate overplotting. + +### 2.4 State Management + +* **Encapsulate Plot State:** Use classes or functions to encapsulate the state of a plot. This makes it easier to manage complex plot configurations and ensures that plots are rendered consistently. +* **Avoid Global State:** Minimize the use of global variables to store plot state. This can lead to unexpected side effects and make it difficult to reason about the code. +* **Use Configuration Objects:** Create configuration objects to store plot settings. This allows you to easily modify the appearance of your plots without changing the code. + +### 2.5 Error Handling + +* **Use `try...except` Blocks:** Wrap potentially problematic code (e.g., file I/O, data processing) in `try...except` blocks to handle exceptions gracefully. +* **Log Errors:** Use the `logging` module to log errors and warnings. This helps you identify and diagnose problems in your code. +* **Provide Informative Error Messages:** When raising exceptions, provide informative error messages that help the user understand the cause of the problem. +* **Validate Inputs:** Before using data in your plots, validate the data to ensure it is in the expected format and range. Handle invalid data appropriately. +* **Custom Exceptions:** Create custom exception classes for specific error conditions in your plotting code. This improves code readability and makes it easier to handle errors. + +## 3. Performance Considerations + +Performance is a crucial aspect when dealing with large datasets or complex visualizations. Optimizing your code can significantly improve rendering speed and reduce memory consumption. + +### 3.1 Optimization Techniques + +* **Vectorization:** Use NumPy's vectorized operations to perform calculations on entire arrays instead of looping through individual elements. This can significantly improve performance. +* **Data Aggregation:** Aggregate data before plotting it to reduce the number of data points. This can be especially helpful for large datasets. +* **Use Efficient Plot Types:** Choose plot types that are appropriate for the data and the visualization goal. For example, use histograms to visualize distributions instead of scatter plots. +* **Limit Data Points:** Plot only the necessary data points. Avoid plotting unnecessary details that do not contribute to the visualization. +* **Caching:** Cache intermediate results to avoid redundant calculations. This can be helpful for computationally intensive tasks. +* **Simplify Complex Geometries:** Reduce the complexity of plot elements by simplifying geometries or using approximations. + +### 3.2 Memory Management + +* **Release Unused Memory:** Explicitly release memory occupied by large data structures when they are no longer needed. Use `del` statement to remove references to objects. +* **Use Generators:** Use generators to process large datasets in chunks instead of loading the entire dataset into memory. This can significantly reduce memory consumption. +* **Avoid Creating Copies:** Avoid creating unnecessary copies of data. Use in-place operations whenever possible. +* **Sparse Data Structures:** For sparse datasets, consider using sparse data structures to reduce memory consumption. + +### 3.3 Rendering Optimization + +* **Use Blitting:** Use blitting to redraw only the parts of the plot that have changed. This can significantly improve rendering speed for interactive plots. +* **Reduce Figure Size:** Reduce the size of the figure to improve rendering speed. Smaller figures require less memory and processing power. +* **Rasterize Vector Graphics:** For very complex plots, consider rasterizing vector graphics to reduce rendering time. Use the `rasterized=True` option in plotting functions. +* **Use Hardware Acceleration:** Ensure that Matplotlib is using hardware acceleration if available. This can significantly improve rendering speed. + +### 3.4 Bundle Size Optimization + +* This is generally less relevant for Matplotlib as it's a backend library, not typically bundled for frontend use. +* However, if integrating Matplotlib plots into web applications, pre-render the images on the server-side and serve static images to the client. + +### 3.5 Lazy Loading + +* Delay loading large datasets until they are needed. This can improve the startup time of your application. +* Load only the data that is visible in the plot. As the user zooms or pans, load additional data as needed. +* Use lazy loading techniques to create thumbnails or previews of plots without rendering the full plot. + +## 4. Security Best Practices + +While Matplotlib is primarily a visualization library, security considerations are important when dealing with untrusted data or integrating Matplotlib into web applications. + +### 4.1 Common Vulnerabilities + +* **Code Injection:** If user-provided data is used to construct plot commands, it could lead to code injection vulnerabilities. Always sanitize user input before using it in plot commands. +* **Denial of Service (DoS):** Creating extremely complex plots with large datasets can consume excessive resources and lead to DoS attacks. Implement resource limits and input validation to prevent this. +* **Cross-Site Scripting (XSS):** If integrating Matplotlib plots into web applications, be careful about how the plots are displayed. Sanitize any user-provided data that is displayed in the plot to prevent XSS attacks. +* **Data Leakage:** Be cautious about displaying sensitive data in plots. Ensure that the data is properly anonymized or obfuscated before plotting it. + +### 4.2 Input Validation + +* **Validate Data Types:** Ensure that the data used for plotting is of the expected type (e.g., numeric, date). +* **Validate Data Ranges:** Ensure that the data falls within the expected range. This can prevent unexpected behavior and potential errors. +* **Sanitize User Input:** If user input is used to generate plot commands, sanitize the input to prevent code injection vulnerabilities. +* **Limit Input Size:** Limit the size of the input data to prevent DoS attacks. + +### 4.3 Authentication and Authorization + +* This section is generally less relevant for Matplotlib, as it doesn't typically handle user authentication directly. +* If the visualizations are part of a larger application that has authentication, ensure that the correct users are authenticated before showing the plots. +* Implement authorization to control which users have access to specific plots or data. + +### 4.4 Data Protection + +* **Anonymize Sensitive Data:** Before plotting sensitive data, anonymize or obfuscate it to protect user privacy. +* **Use Secure Storage:** Store sensitive data securely, using encryption and access controls. +* **Comply with Regulations:** Ensure that your data handling practices comply with relevant regulations (e.g., GDPR, HIPAA). + +### 4.5 Secure API Communication + +* If integrating Matplotlib plots into web applications, use secure API communication protocols (e.g., HTTPS). +* Validate and sanitize data received from APIs before using it in plots. +* Implement rate limiting to prevent API abuse. + +## 5. Testing Approaches + +Thorough testing is essential to ensure the quality and reliability of your Matplotlib code. Unit tests, integration tests, and end-to-end tests all play a role in validating different aspects of your code. + +### 5.1 Unit Testing + +* **Test Individual Functions and Classes:** Write unit tests to verify the behavior of individual functions and classes in your plotting code. +* **Use Assertions:** Use assertions to check that the output of your functions is as expected. +* **Test Edge Cases:** Test edge cases and boundary conditions to ensure that your code handles unexpected input gracefully. +* **Test Error Handling:** Test that your code handles exceptions correctly. +* **Use Mocking:** Use mocking to isolate units of code from their dependencies. This makes it easier to test units of code in isolation. +* **Parameterize Tests:** Write parameterized tests to test the same function with different inputs and expected outputs. This reduces code duplication and improves test coverage. + +### 5.2 Integration Testing + +* **Test Interactions Between Modules:** Write integration tests to verify that different modules in your plotting code work together correctly. +* **Test Data Flow:** Test the flow of data through your plotting pipeline to ensure that data is processed correctly. +* **Test Plot Rendering:** Test that plots are rendered correctly and that they contain the expected elements. +* **Test Data Integration:** Test the integration with data sources, ensuring that the data is loaded correctly. + +### 5.3 End-to-End Testing + +* **Test the Entire Plotting Workflow:** Write end-to-end tests to verify that the entire plotting workflow works correctly, from data loading to plot rendering. +* **Simulate User Interactions:** Simulate user interactions (e.g., zooming, panning, clicking) to test the behavior of interactive plots. +* **Verify Visual Output:** Use visual testing tools to compare the rendered plots against baseline images. This can help you detect visual regressions. + +### 5.4 Test Organization + +* **Create a Separate Test Directory:** Create a separate directory for your tests. +* **Use Descriptive Test Names:** Use descriptive test names that indicate what the test is verifying. +* **Organize Tests by Module:** Organize your tests by module to make it easier to find and run tests. +* **Use Test Runners:** Use test runners (e.g., `pytest`, `unittest`) to run your tests and generate reports. + +### 5.5 Mocking and Stubbing + +* **Mock External Dependencies:** Mock external dependencies (e.g., data sources, APIs) to isolate units of code from their dependencies. +* **Stub Function Calls:** Stub function calls to control the behavior of functions that are called by the code under test. +* **Use Mocking Libraries:** Use mocking libraries (e.g., `unittest.mock`, `pytest-mock`) to simplify the process of creating mocks and stubs. + +## 6. Common Pitfalls and Gotchas + +Knowing common pitfalls and gotchas can save you a lot of time and frustration when working with Matplotlib. + +### 6.1 Frequent Mistakes + +* **Incorrect Axis Limits:** Forgetting to set the correct axis limits can result in plots that are difficult to interpret. +* **Missing Labels and Titles:** Forgetting to add labels and titles to plots makes them less informative and harder to understand. +* **Overlapping Labels:** Overlapping labels can make plots difficult to read. Use techniques like rotation or alignment to avoid overlapping labels. +* **Incorrect Color Maps:** Using inappropriate color maps can distort the data and make it difficult to interpret the plots. +* **Not Handling Missing Data:** Not handling missing data can result in unexpected errors or incorrect plots. Use appropriate techniques to handle missing data (e.g., imputation, filtering). +* **Ignoring Aspect Ratio:** Ignoring the aspect ratio of the plot can result in distorted visualizations. Use `ax.set_aspect('equal')` to ensure that the aspect ratio is correct. + +### 6.2 Edge Cases + +* **Empty Datasets:** Handle the case where the dataset is empty gracefully. +* **Non-Numeric Data:** Handle the case where the data contains non-numeric values. +* **Infinite Values:** Handle the case where the data contains infinite values. +* **NaN Values:** Handle the case where the data contains NaN (Not a Number) values. +* **Large Datasets:** Handle the case where the dataset is very large. Use techniques like data aggregation and lazy loading to improve performance. + +### 6.3 Version-Specific Issues + +* **API Changes:** Be aware of API changes between different versions of Matplotlib. Consult the documentation for the specific version you are using. +* **Compatibility Issues:** Be aware of compatibility issues between Matplotlib and other libraries (e.g., NumPy, Pandas). +* **Bug Fixes:** Be aware of bug fixes in newer versions of Matplotlib. Upgrading to the latest version may resolve known issues. + +### 6.4 Compatibility Concerns + +* **NumPy Version:** Ensure that your NumPy version is compatible with your Matplotlib version. +* **Pandas Version:** Ensure that your Pandas version is compatible with your Matplotlib version. +* **Operating System:** Be aware of potential compatibility issues between Matplotlib and different operating systems (e.g., Windows, macOS, Linux). +* **Backend Rendering Engine:** Be aware of potential compatibility issues between Matplotlib and different backend rendering engines (e.g., Agg, TkAgg, WebAgg). + +### 6.5 Debugging Strategies + +* **Use Print Statements:** Use print statements to inspect the values of variables and data structures. +* **Use Debuggers:** Use debuggers (e.g., `pdb`, `ipdb`) to step through your code and examine the state of the program. +* **Use Logging:** Use logging to record events and errors in your code. This can help you identify and diagnose problems. +* **Simplify the Plot:** Simplify the plot to isolate the source of the problem. Remove unnecessary elements or reduce the amount of data being plotted. +* **Check the Documentation:** Consult the Matplotlib documentation for information about specific functions and methods. +* **Search Online:** Search online for solutions to common problems. Stack Overflow and other online forums can be valuable resources. +* **Isolate the issue**: Comment out portions of the plotting routine to isolate which sections are causing unexpected behavior. + +## 7. Tooling and Environment + +Using the right tools and environment can significantly improve your productivity and the quality of your Matplotlib code. + +### 7.1 Recommended Development Tools + +* **IDE:** Use an Integrated Development Environment (IDE) such as VS Code, PyCharm, or Spyder. These tools provide features such as code completion, debugging, and testing. +* **Jupyter Notebooks:** Use Jupyter Notebooks for interactive exploration and development. +* **Virtual Environments:** Use virtual environments to isolate project dependencies. +* **Version Control:** Use version control systems (e.g., Git) to track changes to your code. + +### 7.2 Build Configuration + +* **Use `requirements.txt`:** Create a `requirements.txt` file to specify project dependencies. Use `pip freeze > requirements.txt` to generate the file. +* **Use `setup.py`:** Create a `setup.py` file to package your plotting code into a reusable library. This file defines the metadata about the library (name, version, author, etc.) and the dependencies required to install it. +* **Use `pyproject.toml`:** For modern projects, consider using `pyproject.toml` to manage build dependencies and configurations. It offers a more standardized and flexible approach compared to `setup.py`. + +### 7.3 Linting and Formatting + +* **Use Linters:** Use linters (e.g., `flake8`, `pylint`) to identify potential errors and style violations in your code. +* **Use Formatters:** Use formatters (e.g., `black`, `autopep8`) to automatically format your code according to PEP 8 style guidelines. +* **Configure IDE:** Configure your IDE to automatically run linters and formatters when you save your code. + +### 7.4 Deployment Best Practices + +* **Containerization:** Use containerization technologies (e.g., Docker) to create portable and reproducible deployment environments. This ensures that your plotting code runs consistently across different platforms. +* **Cloud Platforms:** Deploy your plotting code to cloud platforms (e.g., AWS, Azure, Google Cloud) for scalability and reliability. +* **Serverless Functions:** Consider using serverless functions to deploy individual plotting tasks as independent units. This can be a cost-effective way to run plotting code on demand. + +### 7.5 CI/CD Integration + +* **Use CI/CD Pipelines:** Set up Continuous Integration/Continuous Deployment (CI/CD) pipelines to automate the build, test, and deployment process. This helps ensure that your code is always in a releasable state. +* **Run Tests Automatically:** Configure your CI/CD pipeline to automatically run tests whenever you commit code. This helps you catch errors early. +* **Deploy Automatically:** Configure your CI/CD pipeline to automatically deploy your code to production environments after it has passed all tests. \ No newline at end of file diff --git a/.cursor/rules/maven.mdc b/.cursor/rules/maven.mdc new file mode 100644 index 000000000..e6156b7b2 --- /dev/null +++ b/.cursor/rules/maven.mdc @@ -0,0 +1,177 @@ +--- +description: Comprehensive guidelines for effective Maven project management, covering code organization, dependency management, performance optimization, and security best practices. This rule provides actionable advice to avoid common pitfalls and promote maintainable, scalable Maven projects. +globs: **/pom.xml +--- +--- +- **Introduction** + This guide provides comprehensive best practices for effectively using Maven in Java development, focusing on code organization, dependency management, performance, security, testing, and tooling. Adhering to these guidelines will result in more maintainable, scalable, and robust Maven projects. + +- **Code Organization and Structure** + + - **Standard Directory Structure:** + - Follow the standard Maven directory structure for consistency and ease of understanding: + + project-root/ + ├── src/ + │ ├── main/ + │ │ ├── java/ # Source code + │ │ ├── resources/ # Resources (e.g., properties files) + │ ├── test/ + │ │ ├── java/ # Test code + │ │ ├── resources/ # Test resources + ├── pom.xml # Maven project definition + + - **File Naming Conventions:** + - Java source files: `YourClass.java` + - Resource files: `application.properties`, `log4j2.xml` + - POM files: `pom.xml` + - **Module Organization (Multi-Module Projects):** + - For larger projects, consider using a multi-module structure: + + parent-project/ + ├── pom.xml # Parent POM (defines dependencies, plugins, versions) + ├── module1/ + │ └── pom.xml # Module-specific POM + ├── module2/ + │ └── pom.xml + └── ... + + - Parent POM handles shared configurations. + - Each module has its specific functionality and dependencies. + - **Component Architecture:** + - Organize code into logical components (e.g., data access, business logic, UI). + - Use interfaces to define contracts between components. + - Employ dependency injection to manage component dependencies. + - **Code Splitting Strategies:** + - Split large classes into smaller, more manageable units based on functionality. + - Create utility classes for reusable code. + - Break down complex processes into separate methods. + +- **Common Patterns and Anti-patterns** + + - **Design Patterns:** + - **Factory Pattern:** Use factories to create instances of objects, especially when the exact type of object needed is not known at compile time. + - **Singleton Pattern:** Ensure only one instance of a class exists (use with caution). + - **Strategy Pattern:** Define a family of algorithms, encapsulate each one, and make them interchangeable. + - **Recommended Approaches:** + - **Dependency Injection:** Use frameworks like Spring to manage dependencies and improve testability. + - **Configuration Management:** Externalize configuration using properties files or environment variables. + - **Logging:** Use a logging framework like Log4j 2 or SLF4J for consistent logging practices. + - **Anti-Patterns:** + - **God Classes:** Avoid large, monolithic classes that handle too many responsibilities. + - **Copy-Paste Programming:** Extract common code into reusable methods or classes. + - **Ignoring Exceptions:** Always handle exceptions properly; don't just catch and ignore them. + - **Hardcoding Values:** Externalize configuration values to avoid hardcoding. + - **State Management:** + - For web applications, use session management carefully. + - Avoid storing sensitive data in sessions. + - Use appropriate scoping for managed beans (e.g., request, session, application). + - **Error Handling:** + - Use try-catch blocks to handle exceptions gracefully. + - Log exceptions with sufficient context. + - Provide user-friendly error messages. + - Use custom exception classes to represent specific error conditions. + +- **Performance Considerations** + + - **Optimization Techniques:** + - **Profiling:** Use profiling tools (e.g., VisualVM, YourKit) to identify performance bottlenecks. + - **Caching:** Implement caching (e.g., using Ehcache or Redis) to reduce database load and improve response times. + - **Connection Pooling:** Use connection pooling to manage database connections efficiently. + - **Efficient Data Structures:** Choose appropriate data structures for your specific needs (e.g., HashMap, ArrayList). + - **Optimize Database Queries:** Use indexes, avoid full table scans, and optimize SQL queries. + - **Memory Management:** + - **Garbage Collection:** Understand how garbage collection works and tune JVM settings if necessary. + - **Object Pooling:** Use object pooling for frequently created and destroyed objects. + - **Avoid Memory Leaks:** Ensure resources are properly released to prevent memory leaks. + - **Bundle Size Optimization:** + - Use ProGuard or other code shrinking tools to reduce the size of JAR files. + - Remove unused dependencies. + - Optimize resource files (e.g., compress images). + - **Lazy Loading:** + - Use lazy loading for resources or data that are not immediately needed. + - Implement pagination for large datasets. + +- **Security Best Practices** + + - **Common Vulnerabilities:** + - **Dependency Vulnerabilities:** Regularly scan dependencies for known vulnerabilities (using Snyk, OWASP Dependency-Check). + - **SQL Injection:** Use parameterized queries or ORM frameworks to prevent SQL injection. + - **Cross-Site Scripting (XSS):** Encode user input properly to prevent XSS attacks. + - **Cross-Site Request Forgery (CSRF):** Implement CSRF protection mechanisms. + - **Authentication and Authorization Flaws:** Use strong authentication and authorization mechanisms. + - **Input Validation:** + - Validate all user input to prevent malicious data from entering the system. + - Use whitelisting to define allowed values. + - Sanitize input to remove potentially harmful characters. + - **Authentication and Authorization:** + - Use a secure authentication mechanism (e.g., OAuth 2.0, OpenID Connect). + - Implement role-based access control (RBAC) to restrict access to resources. + - Use strong password policies. + - **Data Protection:** + - Encrypt sensitive data at rest and in transit. + - Use HTTPS to secure communication. + - Store passwords securely using hashing algorithms (e.g., bcrypt). + - **Secure API Communication:** + - Use API keys or OAuth 2.0 to authenticate API requests. + - Validate API requests to prevent malicious input. + - Limit the rate of API requests to prevent abuse. + +- **Testing Approaches** + + - **Unit Testing:** + - Use JUnit or TestNG for unit testing. + - Write tests for all critical components and methods. + - Use mocking frameworks (e.g., Mockito, EasyMock) to isolate units under test. + - Aim for high code coverage. + - **Integration Testing:** + - Test the interaction between different components or services. + - Use embedded databases or mock services for testing. + - Test data access layers, API endpoints, and message queues. + - **End-to-End Testing:** + - Simulate real user scenarios to test the entire application flow. + - Use testing frameworks like Selenium or Cypress. + - Test the user interface, business logic, and data access layers. + - **Test Organization:** + - Place test code in the `src/test/java` directory. + - Organize tests into packages that mirror the source code structure. + - Use descriptive test names. + - **Mocking and Stubbing:** + - Use mocking frameworks to create mock objects for dependencies. + - Use stubs to provide predefined responses for method calls. + - Avoid over-mocking; test the actual behavior when possible. + +- **Common Pitfalls and Gotchas** + + - **Dependency Conflicts:** Use `mvn dependency:tree` to identify and resolve dependency conflicts. Use `` to control versions. + - **Version Mismatches:** Ensure all modules in a multi-module project use compatible versions of dependencies. + - **Transitive Dependencies:** Be aware of transitive dependencies and their potential impact on your project. + - **Incorrect Scopes:** Use the correct dependency scopes (e.g., compile, runtime, test, provided) to avoid including unnecessary dependencies in the final artifact. + - **Snapshot Dependencies:** Avoid using snapshot dependencies in production environments. + +- **Tooling and Environment** + + - **Recommended Development Tools:** + - **IDE:** IntelliJ IDEA, Eclipse, NetBeans + - **Build Tool:** Apache Maven + - **Version Control:** Git + - **Dependency Scanning:** Snyk, OWASP Dependency-Check + - **Static Analysis:** SonarQube, FindBugs + - **Build Configuration:** + - Use properties to define reusable values (e.g., versions, file paths). + - Configure plugins properly and use appropriate versions. + - Use profiles to manage different build configurations (e.g., development, production). + - **Linting and Formatting:** + - Use code formatters (e.g., IntelliJ IDEA code style, Eclipse code formatter) to ensure consistent code formatting. + - Use static analysis tools to identify potential issues. + - **Deployment:** + - Use Maven plugins (e.g., `maven-deploy-plugin`) to deploy artifacts to a repository. + - Use configuration management tools (e.g., Ansible, Chef, Puppet) to automate deployment. + - Follow best practices for deploying Java applications (e.g., using application servers, containers). + - **CI/CD Integration:** + - Integrate Maven builds with CI/CD pipelines (e.g., Jenkins, GitLab CI, GitHub Actions). + - Automate testing, code analysis, and deployment. + - Use build automation tools to manage the build process. + +- **Conclusion** + By adhering to these best practices, you can build robust, maintainable, and secure Maven projects. Regular code reviews, continuous integration, and proactive dependency management will further enhance the quality and reliability of your software. \ No newline at end of file diff --git a/.cursor/rules/microsoft-teams.mdc b/.cursor/rules/microsoft-teams.mdc new file mode 100644 index 000000000..ae644fd27 --- /dev/null +++ b/.cursor/rules/microsoft-teams.mdc @@ -0,0 +1,241 @@ +--- +description: This rule provides best practices for developing with Microsoft Teams, covering code organization, performance, security, testing, and common pitfalls. Adhering to these guidelines will ensure robust, maintainable, and secure Teams applications. +globs: *.js,*.jsx,*.ts,*.tsx +--- +- # Code Organization and Structure + - Effective code organization is critical for maintainability and scalability in Microsoft Teams application development. + + - ## Directory Structure Best Practices + - **src/**: Contains all source code. + - **components/**: Reusable UI components. + - **modules/**: Independent, reusable modules with specific functionalities (e.g., authentication, data fetching). + - **pages/** or **views/**: Top-level components representing different application views/pages. + - **services/**: Code interacting with Microsoft Teams APIs or other external services. + - **utils/** or **helpers/**: Utility functions, constants, and helper classes. + - **styles/**: CSS, SCSS, or other styling-related files. + - **types/**: TypeScript type definitions (if using TypeScript). + - **public/**: Static assets (images, fonts, etc.). + - **tests/**: Unit, integration, and end-to-end tests. + - **.cursor/**: Stores Cursor-specific project rules. + + - ## File Naming Conventions + - **Components:** PascalCase (e.g., `MyTeamsComponent.tsx`). + - **Modules:** camelCase (e.g., `teamsAuthentication.ts`). + - **Styles:** kebab-case (e.g., `my-teams-component.scss`). + - **Tests:** `.test.ts` or `.spec.ts` (e.g., `MyTeamsComponent.test.tsx`). + - **Interfaces/Types:** Capitalized with a leading `I` if an interface (e.g., `IUser.ts`). + + - ## Module Organization + - Favor small, focused modules with clear responsibilities. + - Use dependency injection to manage dependencies between modules. + - Consider using an inversion of control (IoC) container for more complex applications. + + - ## Component Architecture + - **Presentational Components:** Responsible for rendering UI; receive data and callbacks as props. + - **Container Components:** Fetch data, manage state, and pass data to presentational components. + - Consider using a state management library like Redux, Zustand, or Jotai for complex applications. + + - ## Code Splitting + - Use dynamic imports to lazy-load modules and components. + - Implement route-based code splitting to reduce the initial bundle size. + - Analyze bundle size using tools like Webpack Bundle Analyzer to identify large dependencies. + +- # Common Patterns and Anti-patterns + - Employing established design patterns and avoiding anti-patterns are crucial for writing clean, maintainable code in Microsoft Teams development. + + - ## Design Patterns + - **Observer Pattern:** Useful for subscribing components to changes in Teams state (e.g., user presence, channel messages). + - **Factory Pattern:** Abstract component creation to simplify management of different Teams context variables or user settings. + - **Strategy Pattern:** To handle different API authentication approaches and use appropriate logic depending on where code is run (webpage or Teams client) + - **Facade Pattern:** Simplify complex Teams API interactions by providing a higher-level interface. + + - ## Recommended Approaches + - Use the Microsoft Teams Client SDK for interacting with the Teams client. + - Prefer asynchronous operations to avoid blocking the UI thread. + - Implement proper error handling for API calls and user input. + - Adopt responsive design principles to ensure your application works well on different screen sizes. + + - ## Anti-patterns and Code Smells + - **Deeply Nested Components:** Leads to performance issues and difficulty in understanding the component hierarchy. + - **God Objects:** Modules or components with too many responsibilities. + - **Copy-Pasted Code:** Indicates a lack of proper abstraction and code reuse. + - **Ignoring Errors:** Can lead to unexpected behavior and difficult-to-debug issues. + - **Over-reliance on `any` Type (TypeScript):** Reduces the benefits of TypeScript's type safety. + + - ## State Management + - For simple components, use React's built-in `useState` hook. + - For complex applications, consider using a state management library: + - **Redux:** Predictable state container with a centralized store and unidirectional data flow. + - **Zustand:** Small, fast, and scalable bearbones state-management solution. + - **Jotai:** Primitive and flexible state management with an atomic model. + - Avoid mutating state directly; use immutability techniques. + + - ## Error Handling + - Use `try...catch` blocks to handle exceptions gracefully. + - Implement global error handling to catch unhandled exceptions. + - Log errors to a central location for monitoring and debugging. + - Provide informative error messages to the user. + +- # Performance Considerations + - Optimizing performance is crucial for providing a smooth user experience in Microsoft Teams applications. + + - ## Optimization Techniques + - **Memoization:** Cache the results of expensive function calls to avoid redundant computations. + - **Debouncing and Throttling:** Limit the rate at which functions are executed in response to user input. + - **Virtualization:** Render only the visible portion of large lists or tables. + - **Code Splitting:** As mentioned earlier, divide code into smaller chunks to reduce initial load time. + - **Efficient Data Structures:** Use appropriate data structures for optimal performance (e.g., Maps instead of Objects for frequent lookups). + + - ## Memory Management + - Avoid memory leaks by properly cleaning up resources (e.g., event listeners, timers). + - Use weak references when necessary to avoid preventing garbage collection. + - Monitor memory usage using browser developer tools to identify potential issues. + + - ## Rendering Optimization + - Use React.memo or similar techniques to prevent unnecessary re-renders. + - Optimize images and other assets to reduce file size. + - Minimize DOM manipulations. + + - ## Bundle Size Optimization + - Use tree shaking to remove unused code from dependencies. + - Minify and compress code using tools like Terser and Gzip. + - Analyze bundle size to identify large dependencies and consider alternatives. + + - ## Lazy Loading + - Lazy-load images and other assets that are not immediately visible. + - Use Intersection Observer API to detect when elements are visible. + - Lazy load modules by incorporating `React.lazy` and `Suspense`. + +- # Security Best Practices + - Security is paramount when developing Microsoft Teams applications to protect user data and prevent vulnerabilities. + + - ## Common Vulnerabilities + - **Cross-Site Scripting (XSS):** Injecting malicious scripts into the application. + - **Cross-Site Request Forgery (CSRF):** Attacking users into performing actions without their consent. + - **Authentication and Authorization Flaws:** Improperly securing user authentication and authorization mechanisms. + - **Data Injection:** SQL injection, LDAP injection, etc. + + - ## Prevention Strategies + - **Input Validation:** Sanitize and validate all user input to prevent injection attacks. + - **Output Encoding:** Escape output to prevent XSS attacks. + - **Content Security Policy (CSP):** Configure CSP to restrict the sources of content that the browser is allowed to load. + - **Regular Security Audits:** Conduct regular security audits to identify and address vulnerabilities. + + - ## Authentication and Authorization + - Use OAuth 2.0 for secure authentication and authorization. + - Implement proper role-based access control (RBAC) to restrict access to sensitive data and functionality. + - Store credentials securely using appropriate encryption and hashing techniques. + + - ## Data Protection + - Encrypt sensitive data at rest and in transit. + - Implement data loss prevention (DLP) policies to prevent sensitive information from being shared inappropriately. + - Follow privacy best practices to protect user data. + + - ## Secure API Communication + - Use HTTPS for all API communication. + - Validate server-side certificate. + - Implement proper authentication and authorization for API endpoints. + - Use rate limiting to prevent denial-of-service attacks. + +- # Testing Approaches + - Comprehensive testing is essential for ensuring the quality and reliability of Microsoft Teams applications. + + - ## Unit Testing + - Test individual components and modules in isolation. + - Use mocking and stubbing to isolate dependencies. + - Aim for high code coverage to ensure that all parts of the code are tested. + + - ## Integration Testing + - Test the interaction between different components and modules. + - Verify that data flows correctly between different parts of the application. + - Test integrations with Microsoft Teams APIs and other external services. + + - ## End-to-End Testing + - Test the entire application flow from the user's perspective. + - Simulate real-world user scenarios. + - Use tools like Cypress or Playwright to automate end-to-end tests. + + - ## Test Organization + - Organize tests into logical groups based on functionality or component. + - Use clear and descriptive test names. + - Keep tests small and focused. + + - ## Mocking and Stubbing + - Use mocking libraries like Jest or Sinon.js to create mocks and stubs. + - Mock external dependencies to isolate the code under test. + - Use stubs to provide controlled responses from dependencies. + +- # Common Pitfalls and Gotchas + - Being aware of common pitfalls and gotchas can save developers time and effort when developing Microsoft Teams applications. + + - ## Frequent Mistakes + - Improperly handling asynchronous operations. + - Not validating user input. + - Exposing sensitive data in the client-side code. + - Failing to handle errors gracefully. + - Neglecting performance optimization. + - Not localizing application for different locales + + - ## Edge Cases + - Handling different Teams client versions and platforms. + - Dealing with network connectivity issues. + - Handling unexpected API responses. + - Managing different user roles and permissions. + - Handling large data sets. + + - ## Version-Specific Issues + - Be aware of breaking changes in Microsoft Teams API updates. + - Test your application with different Teams client versions. + - Use feature detection to adapt to different API capabilities. + + - ## Compatibility Concerns + - Ensure that your application is compatible with different browsers and devices. + - Test your application with different operating systems. + - Use polyfills to support older browsers. + + - ## Debugging Strategies + - Use browser developer tools to debug client-side code. + - Use logging to track application flow and identify errors. + - Use remote debugging to debug server-side code. + - Use a debugger to step through code and inspect variables. + +- # Tooling and Environment + - Choosing the right tools and environment can significantly improve the development experience for Microsoft Teams applications. + + - ## Recommended Development Tools + - **Visual Studio Code:** Code editor with excellent TypeScript support and debugging capabilities. + - **Microsoft Teams Toolkit:** Provides templates and tools for creating and deploying Teams applications. + - **Node.js:** JavaScript runtime environment for server-side development. + - **NPM or Yarn:** Package managers for managing dependencies. + - **Webpack or Parcel:** Module bundlers for bundling code and assets. + - **React Developer Tools:** Browser extension for debugging React components. + + - ## Build Configuration + - Use a build system like Webpack or Parcel to automate the build process. + - Configure the build system to optimize code for production (e.g., minification, compression). + - Use environment variables to manage different configuration settings for different environments. + + - ## Linting and Formatting + - Use ESLint or TSLint to enforce coding style and prevent errors. + - Use Prettier to automatically format code. + - Configure the code editor to automatically format code on save. + + - ## Deployment + - Deploy your application to Azure or another cloud provider. + - Use Microsoft Teams App Studio to create and manage your Teams app package. + - Submit your app to the Microsoft Teams app store for wider distribution. + + - ## CI/CD Integration + - Use a CI/CD system like Azure DevOps, GitHub Actions, or Jenkins to automate the build, test, and deployment process. + - Configure the CI/CD system to run tests and linters automatically. + - Use automated deployments to deploy changes to production quickly and reliably. + +- # Additional Best Practices + - **Accessibility:** Ensure your application is accessible to users with disabilities. + - **Localization:** Localize your application for different languages and regions. + - **Documentation:** Write clear and concise documentation for your code. + - **Collaboration:** Use a version control system like Git to collaborate with other developers. + - **Stay Up-to-Date:** Keep up with the latest Microsoft Teams API updates and best practices. + + +By adhering to these best practices, developers can create robust, maintainable, secure, and high-performing Microsoft Teams applications that provide a great user experience. \ No newline at end of file diff --git a/.cursor/rules/mkdocs.mdc b/.cursor/rules/mkdocs.mdc new file mode 100644 index 000000000..9fe510350 --- /dev/null +++ b/.cursor/rules/mkdocs.mdc @@ -0,0 +1,270 @@ +--- +description: Comprehensive guidelines for mkdocs development, covering code organization, best practices, performance, security, testing, common pitfalls, and tooling. This rule aims to ensure maintainable, performant, and secure documentation using mkdocs. +globs: *.md +--- +# mkdocs Best Practices + +This document provides comprehensive guidelines for developing documentation using mkdocs. It covers various aspects including code organization, common patterns, performance, security, testing, common pitfalls, and tooling. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure Best Practices + +- **docs/**: All documentation source files should reside within this directory. This is the default and recommended location for mkdocs. + - `index.md`: The project homepage. + - `about.md`: An example of another page. + - `user-guide/`: A subdirectory for a section of documentation. + - `getting-started.md`: A sub-page within the user guide. + - `configuration-options.md`: Another sub-page. + - `img/`: A directory to store images used in the documentation. Store your images as close as possible to the document that references it. + - `screenshot.png` +- **mkdocs.yml**: The main configuration file for mkdocs, located at the root of the project alongside the `docs/` directory. + +Example: + + +mkdocs.yml +docs/ + index.md + about.md + license.md + img/ + screenshot.png +user-guide/ + getting-started.md + configuration-options.md + + +### 1.2. File Naming Conventions + +- Use `.md` as the extension for Markdown files. +- Name the homepage `index.md` or `README.md`. +- Use descriptive names for other pages (e.g., `getting-started.md`, `configuration-options.md`). +- Use lowercase and hyphens for file names to create clean URLs. + +### 1.3. Module Organization + +- mkdocs itself doesn't have modules in the traditional programming sense. Structure your documentation content logically into sections and sub-sections using directories and files. +- Use the `nav` configuration in `mkdocs.yml` to define the structure of the navigation menu. + +Example: + + +nav: + - Home: index.md + - User Guide: + - Getting Started: user-guide/getting-started.md + - Configuration Options: user-guide/configuration-options.md + - About: about.md + - License: license.md + + +### 1.4. Component Architecture + +- mkdocs is not a component-based system like React or Vue.js. +- Instead, think of each Markdown file as a page or section of your documentation. +- Use includes or macros (via plugins) to reuse content across multiple pages (see Snippets and Includes section). + +### 1.5. Code Splitting Strategies + +- Divide your documentation into logical sections and sub-sections. +- Create separate Markdown files for each section. +- Use the `nav` configuration to structure the navigation menu. +- Use `include-markdown` plugin to avoid repeating content in multiple pages. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns Specific to mkdocs + +- **Navigation as Code**: Managing the navigation structure directly within `mkdocs.yml` is a fundamental pattern. It allows for centralized control over the site's hierarchy. +- **Content Reusability**: Utilizing plugins like `include-markdown` to reuse common elements like disclaimers, notices, or standard procedures. +- **Theming Customization**: Overriding the default theme templates to tailor the look and feel of the documentation to match branding or specific aesthetic requirements. +- **Plugin Extension**: Extending the functionality of mkdocs by using and customizing existing plugins or creating custom ones to fulfill specific documentation needs, such as generating API documentation or integrating with external tools. + +### 2.2. Recommended Approaches for Common Tasks + +- **Linking to Pages**: Use relative paths for internal links to avoid issues when deploying (e.g., `[link to about](about.md)`). +- **Linking to Sections**: Use anchor links to link to specific sections within a page (e.g., `[link to license](about.md#license)`). +- **Including Images**: Place images in the `docs/img/` directory and link to them using relative paths (e.g., `![Screenshot](img/screenshot.png)`). +- **Adding Meta-Data**: Add YAML or MultiMarkdown style meta-data to the beginning of Markdown files to control page templates or add information such as authors or descriptions. + +### 2.3. Anti-patterns and Code Smells to Avoid + +- **Absolute Paths in Links**: Avoid using absolute paths for internal links. Use relative paths instead. +- **Large Markdown Files**: Avoid creating very large Markdown files. Break them into smaller, more manageable files and use the `nav` configuration to link them. +- **Ignoring the `nav` Configuration**: Relying on the automatic navigation generation instead of explicitly defining it in `mkdocs.yml` can lead to a disorganized navigation menu. +- **Over-Customization**: Excessively customizing the theme or adding too many plugins can make the documentation difficult to maintain and update. +- **Lack of a Clear File Structure**: An unstructured or inconsistent file structure makes it difficult to navigate and understand the documentation. + +### 2.4. State Management Best Practices + +- mkdocs is a static site generator, so it doesn't have a traditional concept of state management. +- However, you can use plugins to add dynamic content or interact with external data sources. + +### 2.5. Error Handling Patterns + +- mkdocs doesn't have runtime error handling in the traditional sense. +- Errors typically occur during the build process (e.g., invalid configuration, broken links). +- Use a CI/CD pipeline to automatically build and test the documentation and catch errors early. +- Utilize linters and validators (plugins) to ensure the integrity of your Markdown and configuration files. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +- **Optimize Images**: Use optimized images to reduce file sizes and improve loading times. +- **Minify HTML, CSS, and JavaScript**: Use the `mkdocs-minify-plugin` to minify the generated HTML, CSS, and JavaScript files. +- **Lazy Loading Images**: Lazy load images below the fold to improve initial page load time. +- **Use a CDN**: Use a CDN (Content Delivery Network) to serve static assets (images, CSS, JavaScript) from multiple locations around the world. + +### 3.2. Memory Management + +- mkdocs itself doesn't have specific memory management considerations. +- However, if you're using plugins that generate dynamic content, be mindful of memory usage. + +### 3.3. Rendering Optimization + +- mkdocs generates static HTML files, so rendering performance is generally not a concern. +- However, complex Markdown structures can slow down the build process. Keep your Markdown files as simple as possible. + +### 3.4. Bundle Size Optimization + +- Optimize the size of your images and other static assets. +- Use the `mkdocs-minify-plugin` to minify the generated HTML, CSS, and JavaScript files. +- Avoid including unnecessary CSS or JavaScript in your theme. + +### 3.5. Lazy Loading Strategies + +- Implement lazy loading for images using JavaScript. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities and How to Prevent Them + +- **Cross-Site Scripting (XSS)**: Since mkdocs generates static sites, the risk of XSS is minimal, but if you incorporate user-generated content or external data, sanitize inputs properly. +- **Injection Attacks**: Avoid using untrusted data to generate content. If you must use untrusted data, sanitize it properly. + +### 4.2. Input Validation + +- If your mkdocs site incorporates forms or user input via plugins, validate all inputs to prevent injection attacks. + +### 4.3. Authentication and Authorization Patterns + +- mkdocs doesn't provide built-in authentication or authorization. +- If you need to protect certain pages, you can implement authentication at the web server level or use a plugin. + +### 4.4. Data Protection Strategies + +- Since mkdocs generates static sites, there is no sensitive data stored on the server. +- However, be careful not to include sensitive information in your documentation source files. + +### 4.5. Secure API Communication + +- If your mkdocs site communicates with external APIs, use HTTPS to encrypt the communication. +- Verify the API server's SSL certificate. + +## 5. Testing Approaches + +### 5.1. Unit Testing Strategies + +- mkdocs itself doesn't have unit tests in the traditional programming sense. +- However, if you create custom plugins, you should write unit tests for them. + +### 5.2. Integration Testing + +- Create integration tests to verify that your mkdocs site is built correctly and that all links are working. +- Use a tool like `htmlproofer` to validate URLs in the rendered HTML files. + +### 5.3. End-to-end Testing + +- Use an end-to-end testing framework to verify that your mkdocs site is working as expected in a browser. + +### 5.4. Test Organization + +- Organize your tests into separate directories (e.g., `tests/`). +- Use descriptive names for your test files and test functions. + +### 5.5. Mocking and Stubbing + +- If you're testing custom plugins that interact with external APIs, use mocking and stubbing to isolate your tests. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes Developers Make + +- **Incorrect File Paths**: Using incorrect file paths in links or image references. +- **Ignoring the `nav` Configuration**: Relying on the automatic navigation generation instead of explicitly defining it in `mkdocs.yml`. +- **Over-Customization**: Customizing the theme too much can make the documentation difficult to maintain and update. +- **Not Keeping Documentation Up-to-Date**: Forgetting to update the documentation when the codebase changes. + +### 6.2. Edge Cases to Be Aware Of + +- **Long File Paths**: Long file paths can cause issues on some operating systems. +- **Special Characters in File Names**: Avoid using special characters in file names. +- **Conflicting Plugin Options**: Some plugins may have conflicting options. + +### 6.3. Version-Specific Issues + +- mkdocs is constantly evolving, so be aware of version-specific issues. +- Always test your documentation with the latest version of mkdocs before deploying. + +### 6.4. Compatibility Concerns + +- Ensure that your documentation is compatible with different browsers and devices. +- Test your documentation on different platforms to ensure that it looks good everywhere. + +### 6.5. Debugging Strategies + +- **Check the mkdocs Build Log**: The mkdocs build log contains valuable information about errors and warnings. +- **Use a Debugger**: Use a debugger to step through the mkdocs build process and identify issues. +- **Simplify Your Configuration**: Simplify your `mkdocs.yml` file to isolate the issue. +- **Disable Plugins**: Disable plugins one by one to identify the plugin causing the issue. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +- **Text Editor**: VS Code, Sublime Text, Atom +- **Markdown Editor**: Typora, Mark Text +- **Web Browser**: Chrome, Firefox, Safari +- **mkdocs CLI**: The mkdocs command-line interface. + +### 7.2. Build Configuration + +- Use a `mkdocs.yml` file to configure your mkdocs site. +- The `mkdocs.yml` file should be located at the root of your project. +- The `mkdocs.yml` file should contain the following information: + - `site_name`: The name of your site. + - `site_description`: A description of your site. + - `site_author`: The author of your site. + - `docs_dir`: The directory containing your documentation source files. + - `theme`: The theme to use for your site. + - `nav`: The navigation menu for your site. + - `markdown_extensions`: A list of Markdown extensions to enable. + - `plugins`: A list of plugins to enable. + +### 7.3. Linting and Formatting + +- Use a linter to enforce code style and identify potential issues. +- Use a formatter to automatically format your Markdown files. + +### 7.4. Deployment Best Practices + +- **Use a CI/CD Pipeline**: Use a CI/CD pipeline to automatically build and deploy your mkdocs site. +- **Deploy to a CDN**: Deploy your mkdocs site to a CDN to improve performance. +- **Use HTTPS**: Use HTTPS to encrypt communication with your site. +- **Configure a Custom Domain**: Configure a custom domain for your site. + +### 7.5. CI/CD Integration + +- Use a CI/CD tool like GitHub Actions, GitLab CI, or Travis CI to automatically build and deploy your mkdocs site. +- Configure your CI/CD pipeline to run tests and linters. +- Configure your CI/CD pipeline to deploy your site to a CDN. + +## Additional Notes + +- Always refer to the official mkdocs documentation for the most up-to-date information. +- Consider contributing to the mkdocs community by creating plugins or themes. +- Stay informed about the latest best practices and security vulnerabilities. + +This comprehensive guide should help you create maintainable, performant, and secure documentation using mkdocs. \ No newline at end of file diff --git a/.cursor/rules/mlx.mdc b/.cursor/rules/mlx.mdc new file mode 100644 index 000000000..7b9ca00f8 --- /dev/null +++ b/.cursor/rules/mlx.mdc @@ -0,0 +1,282 @@ +--- +description: This rule provides comprehensive best practices for the MLX library, covering code organization, performance, security, testing, and common pitfalls. It aims to promote consistent, efficient, and maintainable code when working with MLX on Apple platforms. +globs: *.swift +--- +# MLX Library Best Practices + +This document outlines the recommended best practices and coding standards for developing applications using the MLX library on Apple platforms, primarily using Swift. Following these guidelines will help ensure code quality, maintainability, performance, and security. + +## 1. Code Organization and Structure + +A well-structured project is crucial for scalability and maintainability. Here's how to organize your MLX projects effectively: + +### 1.1. Directory Structure + +Adopt a modular directory structure that separates concerns and promotes reusability. A typical project structure might look like this: + + +MyMLXProject/ +├── Data/ +│ ├── Datasets.swift # Classes/structs for handling datasets +│ ├── Preprocessing.swift # Data preprocessing utilities +├── Models/ +│ ├── ModelDefinitions.swift # MLX Model Definitions (structs/classes) +│ ├── Layers.swift # Custom layers for your models +│ ├── LossFunctions.swift # Loss functions for training +├── Training/ +│ ├── Trainer.swift # Training loop and optimization logic +│ ├── Metrics.swift # Evaluation metrics +├── Inference/ +│ ├── InferenceEngine.swift # Model inference and prediction code +├── Utilities/ +│ ├── Helpers.swift # Helper functions and extensions +│ ├── Configuration.swift # Project configuration settings +├── UI/ +│ ├── ViewControllers.swift # UI-related code (if applicable) +│ ├── Views.swift # Custom UI views +├── Tests/ +│ ├── UnitTests.swift # Unit tests for individual components +│ ├── IntegrationTests.swift # Integration tests for the application +├── MyMLXProject.xcodeproj # Xcode project file +├── Podfile # CocoaPods dependencies (if using) +├── Package.swift # Swift Package Manager manifest (if using) + + +### 1.2. File Naming Conventions + +- Use descriptive and consistent names for your files. +- Follow the `PascalCase` convention for Swift files (e.g., `ModelDefinitions.swift`). +- Name files according to the primary type or functionality they contain. +- Use suffixes like `View`, `Controller`, `Model`, `Helper`, `Manager` to clearly indicate the role of the file. + +### 1.3. Module Organization + +- Organize your code into logical modules or Swift packages using Swift Package Manager (SPM). +- Each module should encapsulate a specific set of functionalities, such as data processing, model definitions, or training logic. +- Modules should have well-defined interfaces and minimize dependencies between them to promote reusability and testability. + +### 1.4. Component Architecture + +Consider using established architectural patterns like MVVM (Model-View-ViewModel) or VIPER (View-Interactor-Presenter-Entity-Router), especially for applications with a UI. However, even in non-UI projects: + +- **Data Layer**: Responsible for data loading, preprocessing, and storage. This could interface with Core Data, files, or network resources. +- **Model Layer**: Holds the MLX model definitions, custom layers, and any associated logic. Focus on making these types lightweight and easily serializable (if needed). +- **Service Layer**: Manages model training, inference, and evaluation. This layer orchestrates the interaction between the data and model layers. +- **Utility Layer**: Contains helper functions, extensions, and common utilities used throughout the project. + +### 1.5. Code Splitting Strategies + +- Break down large files into smaller, more manageable units based on functionality or responsibility. +- Use extensions to logically group related methods within a class or struct (e.g., `// MARK: - Data Loading`). +- Employ protocols to define clear interfaces between components and enable loose coupling. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +- **Factory Pattern**: Use factory methods or classes to create instances of MLX models, layers, or optimizers, especially when dealing with complex configurations or dependencies. +- **Strategy Pattern**: Implement different optimization algorithms or loss functions as strategies that can be swapped in and out during training. +- **Observer Pattern**: Employ the observer pattern to notify components about training progress or model updates. + +### 2.2. Recommended Approaches for Common Tasks + +- **Data Loading**: Use `MLXData` or custom data loaders to efficiently load and preprocess your training data. Consider using memory mapping for large datasets. +- **Model Definition**: Define your models using `MLX` layers and sequential models. Leverage functional programming techniques for building complex model architectures. +- **Training Loop**: Implement a well-structured training loop that iterates over the data, performs forward and backward passes, updates model parameters, and evaluates performance. +- **Inference**: Use `MLXModel.predict` or custom inference engines to make predictions on new data. +- **Evaluation**: Implement robust evaluation metrics and visualizations to monitor model performance and identify potential issues. + +### 2.3. Anti-Patterns and Code Smells + +- **Massive View Controllers**: Avoid placing all MLX-related logic directly within view controllers. Delegate tasks to service layers or dedicated MLX components. +- **Global State**: Minimize the use of global variables or singletons to manage MLX model instances or training data. Instead, pass these objects as dependencies. +- **Hardcoded Values**: Avoid hardcoding model hyperparameters or data paths. Use configuration files or dependency injection to manage these values. +- **Ignoring Errors**: Handle potential errors during data loading, model training, or inference gracefully, rather than ignoring or suppressing them. +- **Synchronous Operations on the Main Thread**: Offload time-consuming operations like model training or inference to background threads to prevent blocking the main thread and freezing the UI. + +### 2.4. State Management + +- For simple applications, pass state directly between components or use a simple state container object. +- For more complex applications, consider using state management frameworks like Combine or SwiftUI's `@StateObject` and `@EnvironmentObject`. Or a dedicated state management framework for ML models. +- Store model parameters and training progress in a dedicated state object and update it during the training loop. +- Make sure MLX-related state is properly handled and persisted across application lifecycle events (e.g., app termination, backgrounding). + +### 2.5. Error Handling + +- Use Swift's `Error` protocol and `do-catch` blocks to handle potential errors during data loading, model training, or inference. +- Create custom error types to represent specific MLX-related errors. +- Log error messages with sufficient context to aid debugging. +- Provide user-friendly error messages or feedback to the user when appropriate. + +## 3. Performance Considerations + +ML workloads are inherently performance-sensitive. Optimize your MLX code for speed and efficiency: + +### 3.1. Optimization Techniques + +- **Vectorization**: Leverage `MLX`'s vectorized operations to perform computations on entire arrays or matrices rather than iterating over individual elements. +- **Data Type Optimization**: Use the appropriate data types for your model parameters and training data (e.g., `Float16` or `Float32` instead of `Float64`). +- **Memory Optimization**: Minimize memory allocations and deallocations during the training loop. Re-use buffers and avoid unnecessary data copies. +- **Graph Compilation**: If MLX supports it, compile your model graph to optimize its execution. This can significantly improve inference speed. +- **GPU Acceleration**: Ensure your MLX code is running on the GPU for faster computations (if available). Verify GPU usage using system monitoring tools. + +### 3.2. Memory Management + +- Be mindful of memory usage, especially when dealing with large datasets or complex models. +- Use memory profiling tools to identify memory leaks or excessive memory allocations. +- Release unused memory explicitly when possible. +- Consider using techniques like data streaming or batch processing to reduce memory footprint. + +### 3.3. Rendering Optimization (If Applicable) + +- MLX is focused on numerical computation, but if you are visualizing outputs, use techniques like Metal shaders to optimize rendering if you are drawing visualizations of ML data. + +### 3.4. Bundle Size Optimization + +- If using dependencies, consider the impact of external dependencies on the application bundle size. +- Use code stripping and dead code elimination to remove unused code and resources. +- Compress images, models, and other assets to reduce their size. + +### 3.5. Lazy Loading + +- Lazy load large models or datasets only when they are needed to reduce startup time and memory footprint. +- Use placeholder data or loading indicators while lazy-loading data in the background. + +## 4. Security Best Practices + +Even though MLX focuses on numerical computation, security is still important: + +### 4.1. Common Vulnerabilities and Prevention + +- **Model Poisoning**: Prevent malicious actors from injecting adversarial data into your training set to degrade model performance or inject biases. + - **Mitigation**: Implement data validation and sanitization techniques. Monitor data sources and investigate anomalies. +- **Model Extraction**: Protect your trained models from being stolen or reverse-engineered by unauthorized parties. + - **Mitigation**: Use model encryption or obfuscation techniques. Implement access controls and authentication mechanisms. +- **Adversarial Attacks**: Defend against adversarial attacks that can trick your models into making incorrect predictions. + - **Mitigation**: Implement adversarial training techniques or input validation mechanisms. + +### 4.2. Input Validation + +- Validate all external inputs, including user data, API responses, and configuration files, to prevent malicious data from compromising your application or models. +- Sanitize input data to remove or escape potentially harmful characters or code. +- Use type checking and data validation libraries to ensure data conforms to expected formats and ranges. + +### 4.3. Authentication and Authorization + +- Implement robust authentication and authorization mechanisms to protect access to your models, training data, and API endpoints. +- Use secure password storage techniques like hashing and salting. +- Implement role-based access control (RBAC) to restrict access to sensitive resources based on user roles. + +### 4.4. Data Protection + +- Encrypt sensitive data at rest and in transit to prevent unauthorized access. +- Use secure storage options like the iOS Keychain to store sensitive information like API keys or passwords. +- Implement data masking or anonymization techniques to protect user privacy. + +### 4.5. Secure API Communication + +- Use HTTPS to encrypt communication between your application and external APIs or services. +- Implement certificate pinning to prevent man-in-the-middle attacks. +- Validate API responses to ensure data integrity and prevent data injection attacks. + +## 5. Testing Approaches + +Thorough testing is essential for ensuring the reliability and correctness of your MLX code: + +### 5.1. Unit Testing + +- Write unit tests to verify the functionality of individual components, such as model layers, loss functions, or data preprocessing utilities. +- Use mocking and stubbing techniques to isolate components and simulate dependencies. +- Test edge cases and boundary conditions to ensure robust behavior. + +### 5.2. Integration Testing + +- Write integration tests to verify the interaction between different components or modules. +- Test the data flow through the application pipeline, from data loading to model inference. +- Verify that the application integrates correctly with external APIs or services. + +### 5.3. End-to-End Testing + +- Write end-to-end tests to simulate user interactions and verify the overall application functionality. +- Test the user interface (if applicable) to ensure it behaves as expected. +- Verify that the application meets the specified performance and security requirements. + +### 5.4. Test Organization + +- Organize your tests into separate test suites or folders based on the component or functionality being tested. +- Use descriptive names for your test methods to clearly indicate their purpose. +- Follow a consistent naming convention for your test files (e.g., `MyComponentTests.swift`). + +### 5.5. Mocking and Stubbing + +- Use mocking frameworks to create mock objects that simulate the behavior of dependencies during unit testing. +- Use stubbing techniques to provide pre-defined responses or values for dependencies during testing. +- Avoid over-mocking or stubbing, as it can make tests brittle and less reliable. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +- **Incorrect Data Shapes**: Ensure that your input data and model parameters have the correct shapes and dimensions. Incorrect shapes can lead to errors or unexpected behavior. +- **Gradient Vanishing/Exploding**: Be aware of gradient vanishing or exploding problems during training. Use techniques like gradient clipping or batch normalization to mitigate these issues. +- **Overfitting**: Monitor your model's performance on a validation set to prevent overfitting. Use regularization techniques or dropout to reduce overfitting. +- **Not Freezing the graph before production**: After training and validation, ensure that MLX is not attempting to recalculate the graph each time inference is called. Freeze the graph before deploying to production. + +### 6.2. Edge Cases + +- **Handling Missing Data**: Implement strategies for handling missing data in your datasets. Consider using imputation techniques or creating custom data loaders. +- **Dealing with Imbalanced Datasets**: Address imbalanced datasets by using techniques like oversampling, undersampling, or weighted loss functions. +- **Handling Out-of-Memory Errors**: Be prepared to handle out-of-memory errors when dealing with large models or datasets. Reduce batch sizes, use memory mapping, or distribute training across multiple GPUs. + +### 6.3. Version-Specific Issues + +- Be aware of potential compatibility issues between different versions of MLX and other dependencies. +- Consult the MLX documentation and release notes for information on known issues or breaking changes. +- Use dependency management tools like CocoaPods or SPM to manage dependencies and ensure compatibility. + +### 6.4. Compatibility Concerns + +- Consider compatibility between MLX and other technologies used in your project, such as Core ML or Metal. +- Use feature detection or conditional compilation to handle different platform versions or hardware capabilities. + +### 6.5. Debugging Strategies + +- Use Xcode's debugger to step through your code and inspect variables. +- Use logging statements to track the execution flow and data values. +- Use visualization tools to inspect model architectures, data distributions, and training progress. +- Leverage MLX's error messages and stack traces to identify and resolve issues. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +- **Xcode**: Use Xcode as your primary IDE for developing MLX applications on Apple platforms. +- **Metal Performance Shaders**: Use Metal Performance Shaders (MPS) to optimize performance on Apple GPUs. +- **Instruments**: Use Instruments to profile your application's performance and identify bottlenecks. +- **TensorBoard**: Use TensorBoard to visualize model architectures, training progress, and evaluation metrics. + +### 7.2. Build Configuration + +- Configure your Xcode project to optimize build settings for performance (e.g., enable optimization levels, disable debugging symbols). +- Use build configurations (e.g., Debug, Release) to manage different build settings for development and production environments. +- Use environment variables to configure application behavior based on the environment. + +### 7.3. Linting and Formatting + +- Use SwiftLint to enforce coding style guidelines and identify potential code quality issues. +- Use SwiftFormat to automatically format your code according to a consistent style. +- Configure Xcode to automatically run SwiftLint and SwiftFormat during the build process. + +### 7.4. Deployment + +- Prepare your MLX models and data for deployment. Convert models to efficient formats (e.g., Core ML) if necessary. +- Use code signing and provisioning profiles to ensure secure application distribution. +- Deploy your application to the App Store or distribute it using enterprise deployment mechanisms. + +### 7.5. CI/CD Integration + +- Integrate your MLX project with a continuous integration and continuous delivery (CI/CD) system like Jenkins, Travis CI, or GitHub Actions. +- Configure CI/CD pipelines to automatically build, test, and deploy your application on every code change. +- Use automated testing frameworks to verify the correctness of your MLX code and models. + +By following these best practices, you can develop robust, efficient, and secure MLX applications that leverage the power of machine learning on Apple platforms. \ No newline at end of file diff --git a/.cursor/rules/mobx.mdc b/.cursor/rules/mobx.mdc new file mode 100644 index 000000000..d33f79582 --- /dev/null +++ b/.cursor/rules/mobx.mdc @@ -0,0 +1,429 @@ +--- +description: This rule provides comprehensive guidance for using MobX effectively, covering best practices for code organization, performance, testing, and common pitfalls. It aims to ensure efficient and maintainable state management in React and other JavaScript applications using MobX. +globs: *.js,*.jsx,*.ts,*.tsx +--- +# MobX Best Practices and Coding Standards + +This document outlines the best practices for using MobX in your projects. Following these guidelines will help you write more maintainable, performant, and robust code. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure Best Practices + +* **Feature-Based Organization:** Organize your code by feature rather than by file type (e.g., components, stores, utils). This promotes better modularity and easier navigation. + + + src/ + ├── features/ + │ ├── user-profile/ + │ │ ├── components/ + │ │ │ ├── UserProfile.jsx + │ │ │ └── UserDetails.jsx + │ │ ├── stores/ + │ │ │ ├── userStore.js + │ │ ├── api/ + │ │ │ └── userApi.js + │ │ └── utils/ + │ │ └── userUtils.js + │ ├── product-listing/ + │ │ └── ... + ├── app.js + └── ... + + +* **Dedicated `stores` Directory:** Place all your MobX stores in a dedicated `stores` directory to clearly separate state management logic from the rest of your application. + +* **Shared Utilities:** Create a `utils` directory for reusable utility functions. + +### 1.2 File Naming Conventions + +* **Descriptive Names:** Use descriptive names for files and modules that clearly indicate their purpose. +* **Consistent Case:** Maintain a consistent naming convention (e.g., camelCase for JavaScript files, PascalCase for React components). + +### 1.3 Module Organization + +* **Single Responsibility Principle:** Each module should have a single, well-defined responsibility. +* **Clear Exports:** Clearly define the exports of each module (e.g., named exports for individual functions, default export for the main component or store). +* **Avoid Circular Dependencies:** Ensure that modules do not have circular dependencies to prevent runtime errors and improve code maintainability. Use dependency injection if necessary. + +### 1.4 Component Architecture + +* **Presentational and Container Components:** Separate presentational (UI-focused) components from container (data-fetching and logic-handling) components. Presentational components receive data via props, while container components connect to MobX stores and pass data down. + + jsx + // Container component + import React from 'react'; + import { observer } from 'mobx-react-lite'; + import { useUserStore } from './stores/userStore'; + import UserProfile from './UserProfile'; + + const UserProfileContainer = observer(() => { + const userStore = useUserStore(); + + return ; // Pass data as props + }); + + export default UserProfileContainer; + + // Presentational component + import React from 'react'; + + const UserProfile = ({ user }) => { + return ( +
+

{user.name}

+

{user.email}

+
+ ); + }; + + export default UserProfile; + + +* **Functional Components with Hooks:** Use functional components with the `useObserver` hook (or `observer` from `mobx-react-lite`) for better performance and readability. + + jsx + import React from 'react'; + import { observer } from 'mobx-react-lite'; + import { useUserStore } from './stores/userStore'; + + const UserProfile = observer(() => { + const userStore = useUserStore(); + + return ( +
+

{userStore.user.name}

+

{userStore.user.email}

+
+ ); + }); + + export default UserProfile; + + +* **Component Composition:** Favor component composition over deep inheritance to create reusable and flexible components. + +### 1.5 Code Splitting Strategies + +* **Route-Based Splitting:** Split your application into chunks based on routes or pages. This allows users to download only the code they need for the current view. +* **Component-Based Splitting:** Split large components into smaller chunks that can be loaded on demand. Use `React.lazy` and `Suspense` for lazy loading components. +* **Vendor Splitting:** Separate your vendor dependencies (e.g., libraries) into a separate chunk. This allows browsers to cache vendor code separately from your application code. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns Specific to MobX + +* **Observable State:** Use `@observable` to define the state that MobX should track for changes. Ensure that only necessary data is made observable to optimize performance. +* **Computed Properties:** Use `@computed` to derive values from observable state. Computed properties are automatically updated when their dependencies change and are cached for performance. + + javascript + import { makeObservable, observable, computed } from 'mobx'; + + class Cart { + items = []; + + constructor() { + makeObservable(this, { + items: observable, + totalPrice: computed + }); + } + + get totalPrice() { + return this.items.reduce((sum, item) => sum + item.price, 0); + } + } + + +* **Actions:** Use `@action` to modify the state. Actions ensure that state changes are batched and tracked by MobX. All state modifications should happen within actions to maintain predictability. + + javascript + import { makeObservable, observable, computed, action } from 'mobx'; + + class Cart { + items = []; + + constructor() { + makeObservable(this, { + items: observable, + totalPrice: computed, + addItem: action + }); + } + + get totalPrice() { + return this.items.reduce((sum, item) => sum + item.price, 0); + } + + addItem(item) { + this.items.push(item); + } + } + + +* **Reactions:** Use `reaction`, `autorun`, and `when` to react to state changes. Use `reaction` for side effects that depend on specific observable values, `autorun` for side effects that depend on any observable value, and `when` for one-time side effects. + +### 2.2 Recommended Approaches for Common Tasks + +* **Form Handling:** Use MobX to manage form state. Create observable properties for each form field and use actions to update them. + + jsx + import React from 'react'; + import { observer } from 'mobx-react-lite'; + import { makeObservable, observable, action } from 'mobx'; + + class FormStore { + name = ''; + email = ''; + + constructor() { + makeObservable(this, { + name: observable, + email: observable, + setName: action, + setEmail: action + }); + } + + setName(value) { + this.name = value; + } + + setEmail(value) { + this.email = value; + } + } + + const formStore = new FormStore(); + + const Form = observer(() => { + return ( +
+ formStore.setName(e.target.value)} /> + formStore.setEmail(e.target.value)} /> +
+ ); + }); + + export default Form; + + +* **Asynchronous Operations:** Use actions to handle asynchronous operations such as API calls. Use `async/await` syntax to simplify asynchronous code. + + javascript + import { makeObservable, observable, action } from 'mobx'; + + class UserStore { + user = null; + loading = false; + + constructor() { + makeObservable(this, { + user: observable, + loading: observable, + fetchUser: action + }); + } + + async fetchUser(id) { + this.loading = true; + try { + const response = await fetch(`/api/users/${id}`); + this.user = await response.json(); + } finally { + this.loading = false; + } + } + } + + +### 2.3 Anti-patterns and Code Smells to Avoid + +* **Mutating Observables Directly:** Avoid directly mutating observable values outside of actions. This can lead to unexpected behavior and make it difficult to track state changes. Always use actions to modify observable state. +* **Over-Observing:** Avoid making everything observable. Only observe the data that needs to be tracked for changes. Over-observing can lead to performance issues. +* **Complex Computed Properties:** Keep computed properties simple and focused. Avoid complex logic in computed properties, as this can make your code harder to understand and debug. +* **Using `autorun` Excessively:** Be careful when using `autorun`, as it can easily lead to performance issues if not used correctly. Prefer `reaction` when you need to react to specific observable values. +* **Forgetting to Dispose Reactions:** Always dispose of reactions when they are no longer needed to prevent memory leaks. Use the `dispose` function returned by `autorun` and `reaction`. + +### 2.4 State Management Best Practices + +* **Single Source of Truth:** Maintain a single source of truth for your application's state. Avoid duplicating state across multiple stores. +* **Normalized State:** Normalize your state to reduce redundancy and improve performance. Store data in a flat, relational structure. +* **Immutability (with MobX's Mutability):** While MobX embraces mutability for performance, try to treat your data as immutable as possible, especially when working with arrays and objects. Instead of directly modifying arrays, use methods like `concat`, `slice`, and `filter` to create new arrays. +* **Centralized State Management:** Use MobX to manage all your application's state in a centralized location. This makes it easier to reason about and debug your code. + +### 2.5 Error Handling Patterns + +* **Try-Catch Blocks:** Use `try-catch` blocks to handle errors in asynchronous operations and other code that might throw exceptions. +* **Error Stores:** Create dedicated error stores to manage application-wide errors. This allows you to display error messages to the user and log errors for debugging. +* **Global Error Handling:** Implement global error handling to catch unhandled exceptions and prevent your application from crashing. Use `window.onerror` or `React Error Boundaries`. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +* **Use `mobx-react-lite`:** Use `mobx-react-lite` instead of `mobx-react` for smaller bundle size and improved performance. `mobx-react-lite` provides hooks-based integration with React. +* **`useMemo` and `useCallback`:** Use `useMemo` and `useCallback` to optimize rendering performance by memoizing expensive calculations and preventing unnecessary re-renders. + + jsx + import React, { useMemo } from 'react'; + import { observer } from 'mobx-react-lite'; + + const MyComponent = observer(() => { + const expensiveValue = useMemo(() => { + // Perform expensive calculation + return computeExpensiveValue(); + }, []); + + return
{expensiveValue}
; + }); + + export default MyComponent; + + +* **`shouldComponentUpdate` (Class Components):** If you are using class components, implement `shouldComponentUpdate` to prevent unnecessary re-renders. Compare the previous and current props and state to determine if a re-render is necessary. Consider using `PureComponent`. +* **Minimize Re-renders:** Minimize the number of re-renders by optimizing your component structure and using techniques like `useMemo` and `useCallback`. + +### 3.2 Memory Management + +* **Dispose Reactions:** Always dispose of reactions when they are no longer needed to prevent memory leaks. Use the `dispose` function returned by `autorun` and `reaction`. +* **Avoid Creating Unnecessary Objects:** Avoid creating unnecessary objects, especially in computed properties and reactions. This can lead to memory leaks and performance issues. +* **Garbage Collection:** Be aware of JavaScript's garbage collection mechanism and avoid creating circular references that can prevent garbage collection. + +### 3.3 Rendering Optimization + +* **Virtualization:** Use virtualization techniques to render large lists efficiently. Virtualization renders only the visible items in the list, which can significantly improve performance. +* **Debouncing and Throttling:** Use debouncing and throttling to limit the frequency of updates to the UI. This can improve performance by preventing excessive re-renders. + +### 3.4 Bundle Size Optimization + +* **Code Splitting:** Use code splitting to reduce the initial bundle size of your application. This allows users to download only the code they need for the current view. +* **Tree Shaking:** Use tree shaking to remove dead code from your bundle. Tree shaking is a technique that removes unused code from your bundle, which can significantly reduce its size. +* **Minification:** Use minification to reduce the size of your code. Minification removes whitespace and comments from your code, which can reduce its size. + +### 3.5 Lazy Loading Strategies + +* **Lazy Loading Components:** Use `React.lazy` and `Suspense` to lazy load components. This allows you to load components on demand, which can improve the initial load time of your application. +* **Lazy Loading Images:** Use lazy loading for images to improve the initial load time of your application. This can be done using the `loading` attribute on the `img` element or using a library like `react-lazyload`. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and How to Prevent Them + +* **Cross-Site Scripting (XSS):** Prevent XSS attacks by sanitizing user input and escaping output. Use libraries like `DOMPurify` to sanitize HTML. +* **Cross-Site Request Forgery (CSRF):** Prevent CSRF attacks by using CSRF tokens. CSRF tokens are unique, secret values that are included in requests to prevent attackers from forging requests on behalf of users. +* **SQL Injection:** Prevent SQL injection attacks by using parameterized queries or an ORM (Object-Relational Mapper). Parameterized queries escape user input, which prevents attackers from injecting malicious SQL code. + +### 4.2 Input Validation + +* **Server-Side Validation:** Validate user input on the server-side to prevent malicious data from being stored in your database. +* **Client-Side Validation:** Validate user input on the client-side to provide immediate feedback to the user and improve the user experience. However, always validate on the server-side as well, since client-side validation can be bypassed. +* **Regular Expressions:** Use regular expressions to validate user input. Regular expressions are a powerful tool for validating data against specific patterns. + +### 4.3 Authentication and Authorization Patterns + +* **Authentication:** Use a secure authentication mechanism to verify the identity of users. Use libraries like `Passport.js` or `Auth0` to simplify the authentication process. +* **Authorization:** Implement authorization to control access to resources based on the user's role. Use role-based access control (RBAC) or attribute-based access control (ABAC) to manage access permissions. +* **JSON Web Tokens (JWT):** Use JWTs to securely transmit user information between the client and the server. JWTs are digitally signed, which makes them tamper-proof. + +### 4.4 Data Protection Strategies + +* **Encryption:** Encrypt sensitive data at rest and in transit. Use HTTPS to encrypt data in transit and libraries like `bcrypt` to encrypt passwords. +* **Data Masking:** Mask sensitive data to protect it from unauthorized access. Data masking replaces sensitive data with fictitious data, which allows developers to work with the data without exposing the actual sensitive information. +* **Data Anonymization:** Anonymize data to remove personally identifiable information (PII). Data anonymization is a technique that removes or modifies PII to prevent it from being linked to a specific individual. + +### 4.5 Secure API Communication + +* **HTTPS:** Always use HTTPS to encrypt communication between the client and the server. HTTPS uses TLS/SSL to encrypt data in transit, which prevents eavesdropping. +* **API Keys:** Use API keys to authenticate requests to your API. API keys are unique, secret values that are used to identify the client making the request. +* **Rate Limiting:** Implement rate limiting to prevent abuse of your API. Rate limiting limits the number of requests that a client can make within a given time period. + +## 5. Testing Approaches + +### 5.1 Unit Testing Strategies + +* **Test Stores in Isolation:** Unit test MobX stores in isolation to verify that their state and actions behave as expected. Use mocking and stubbing to isolate the stores from external dependencies. +* **Test Computed Properties:** Test computed properties to ensure that they correctly derive values from observable state. +* **Test Actions:** Test actions to ensure that they correctly modify the state. + +### 5.2 Integration Testing + +* **Test Components with Stores:** Integration test components with MobX stores to verify that they interact correctly. Use a testing library like `React Testing Library` to render the components and simulate user interactions. +* **Test API Interactions:** Test API interactions to ensure that data is correctly fetched from and sent to the server. Use mocking to isolate the components from the actual API. + +### 5.3 End-to-End Testing + +* **Automated Browser Tests:** Use end-to-end testing frameworks like Cypress or Selenium to automate browser tests. End-to-end tests verify that the entire application works correctly from the user's perspective. + +### 5.4 Test Organization + +* **Separate Test Files:** Create separate test files for each module or component. This makes it easier to find and run the tests. +* **Descriptive Test Names:** Use descriptive names for your tests that clearly indicate what they are testing. +* **Test Suites:** Organize your tests into test suites based on functionality or module. + +### 5.5 Mocking and Stubbing + +* **Mock Dependencies:** Use mocking to replace external dependencies with mock objects. This allows you to isolate the code being tested and control its behavior. +* **Stub Functions:** Use stubbing to replace functions with predefined return values or behavior. This allows you to control the behavior of the code being tested without actually executing it. +* **Mock API Calls:** Mock API calls to avoid making real API requests during testing. This makes your tests faster and more reliable. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes Developers Make + +* **Forgetting to Wrap Components with `observer`:** Forgetting to wrap React components with `observer` (or using `useObserver`) prevents them from reacting to changes in the MobX store. +* **Directly Modifying Observable Arrays/Objects:** Directly modifying observable arrays or objects (e.g., `myArray[0] = 'new value'`) won't trigger reactivity. Always use the methods provided by MobX (e.g., `myArray.splice(0, 1, 'new value')` or `myObject.set('key', 'value')`). +* **Not Using `useLocalObservable` in Components:** Using `useLocalObservable` is crucial for creating isolated, component-specific stores, preventing unintended state sharing. + +### 6.2 Edge Cases to Be Aware Of + +* **React Strict Mode:** Be aware that React Strict Mode can cause MobX to re-run reactions multiple times. This can be useful for debugging, but it can also lead to performance issues. +* **Large Datasets:** Be careful when working with large datasets in MobX. Consider using virtualization techniques to improve performance. + +### 6.3 Version-Specific Issues + +* **MobX 5 vs. MobX 6:** Be aware of the differences between MobX 5 and MobX 6. MobX 6 introduced several breaking changes, including the removal of implicit observability. Make sure your code is compatible with the version of MobX you are using. +* **React Compatibility:** Ensure that your version of `mobx-react` or `mobx-react-lite` is compatible with your version of React. + +### 6.4 Compatibility Concerns + +* **Browser Compatibility:** Ensure that your code is compatible with the browsers you are targeting. Use polyfills to support older browsers. +* **Node.js Compatibility:** Ensure that your code is compatible with the version of Node.js you are using. Use a tool like `nvm` to manage multiple Node.js versions. + +### 6.5 Debugging Strategies + +* **MobX DevTools:** Use the MobX DevTools to inspect your application's state and track changes. The MobX DevTools is a browser extension that allows you to visualize your MobX stores and track changes in real-time. +* **Logging:** Use logging to track the execution of your code and identify errors. Use a logging library like `debug` to simplify the logging process. +* **Breakpoints:** Use breakpoints to pause the execution of your code and inspect its state. Breakpoints are a powerful tool for debugging complex code. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +* **VS Code:** Use VS Code as your IDE. VS Code has excellent support for JavaScript, TypeScript, and React, and it has a wide range of extensions that can improve your development workflow. +* **ESLint:** Use ESLint to enforce coding standards and identify potential errors. ESLint is a linter that can be configured to enforce a wide range of coding standards. +* **Prettier:** Use Prettier to automatically format your code. Prettier is a code formatter that can be configured to automatically format your code according to a set of rules. + +### 7.2 Build Configuration + +* **Webpack:** Use Webpack to bundle your code. Webpack is a module bundler that can be used to bundle your code and its dependencies into a single file. +* **Babel:** Use Babel to transpile your code to older versions of JavaScript. Babel is a transpiler that can be used to convert your code to older versions of JavaScript, which allows it to run on older browsers. +* **TypeScript:** Use TypeScript to add static typing to your code. TypeScript is a superset of JavaScript that adds static typing to the language. This can help you catch errors early and improve the maintainability of your code. + +### 7.3 Linting and Formatting + +* **ESLint:** Use ESLint to enforce coding standards and identify potential errors. Configure ESLint to use the recommended rules for React and MobX. +* **Prettier:** Use Prettier to automatically format your code. Configure Prettier to use a consistent code style. +* **Husky:** Use Husky to run linters and formatters before committing code. This ensures that all code committed to the repository meets the required standards. + +### 7.4 Deployment Best Practices + +* **Continuous Integration/Continuous Deployment (CI/CD):** Implement a CI/CD pipeline to automate the deployment process. This ensures that your code is automatically tested and deployed whenever changes are made. +* **Caching:** Use caching to improve the performance of your application. Cache static assets like images and JavaScript files to reduce the load on your server. +* **Content Delivery Network (CDN):** Use a CDN to distribute your static assets across multiple servers. This improves the performance of your application by serving assets from the server closest to the user. + +### 7.5 CI/CD Integration + +* **GitHub Actions:** Use GitHub Actions to automate your CI/CD pipeline. GitHub Actions is a CI/CD service that is integrated with GitHub. Use Jenkins, CircleCI, or other CI/CD tools. +* **Automated Testing:** Automate your testing process to ensure that your code is thoroughly tested before it is deployed. Use a testing framework like Jest or Mocha to write automated tests. +* **Automated Deployment:** Automate your deployment process to ensure that your code is deployed quickly and reliably. Use a deployment tool like `Capistrano` or `Deployer` to automate the deployment process. \ No newline at end of file diff --git a/.cursor/rules/mockito.mdc b/.cursor/rules/mockito.mdc new file mode 100644 index 000000000..71d490ce0 --- /dev/null +++ b/.cursor/rules/mockito.mdc @@ -0,0 +1,249 @@ +--- +description: This rule provides comprehensive best practices and coding standards for using the Mockito library in Java projects. It covers code organization, patterns, performance, security, testing, and common pitfalls to enhance test reliability and maintainability. +globs: *.java +--- +# Mockito Best Practices and Coding Standards + +This document provides comprehensive guidelines for using the Mockito library effectively in Java projects. It covers various aspects, including code organization, common patterns, performance considerations, security, testing, and common pitfalls. + +## Library Information: + +- Name: Mockito +- Tags: testing, java, mocking, unit-testing + +## 1. Code Organization and Structure + +### 1.1 Directory Structure + +- **Standard Maven/Gradle Structure:** Follow the standard Maven or Gradle project structure. + - `src/main/java`: Production code. + - `src/test/java`: Test code. + - `src/test/resources`: Test resources (e.g., data files). +- **Package Structure:** Mirror the package structure of your production code in your test code. + - Example: If your production code is in `com.example.service`, your test code should be in `com.example.service` (or a subpackage like `com.example.service.tests`). + +### 1.2 File Naming Conventions + +- **Test Class Naming:** + - Use the same name as the class being tested, followed by `Test` or `IT` (for integration tests). + - Examples: `UserServiceTest`, `ProductRepositoryIT`. +- **Test Method Naming:** + - Use descriptive names that clearly indicate what is being tested. + - Consider using the pattern `should_[expectedBehavior]_when_[condition]`. + - Example: `should_returnUser_when_userExists()`. + +### 1.3 Module Organization + +- **Single Module:** For small projects, a single module is usually sufficient. +- **Multi-Module Projects:** For larger projects, consider breaking down the project into modules based on functionality. + - Example: `core`, `service`, `repository`, `api`. + - Ensure that test code for each module resides within that module. + +### 1.4 Component Architecture + +- **Layered Architecture:** A common and effective architecture is layered architecture (e.g., presentation, service, data access). +- **Test Each Layer:** Write unit tests for each layer in isolation using Mockito to mock dependencies. +- **Integration Tests:** Write integration tests to verify the interactions between layers. + +### 1.5 Code Splitting Strategies + +- **Test-Driven Development (TDD):** Write tests before writing production code. This naturally leads to smaller, testable units of code. +- **Refactoring:** Regularly refactor your code to improve its structure and testability. +- **Extract Methods:** If a method is too complex to test easily, extract smaller, more focused methods. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns Specific to Mockito + +- **Mock Object:** The core pattern in Mockito is the Mock Object pattern, where dependencies are replaced with controlled test doubles. +- **Dependency Injection:** Use dependency injection to make your code testable. Mockito is most effective when dependencies are easily replaceable with mocks. + +### 2.2 Recommended Approaches for Common Tasks with Mockito + +- **Mocking Dependencies:** Use `@Mock` annotation for cleaner mock creation. +- **Injecting Mocks:** Use `@InjectMocks` to automatically inject mocks into the class under test. +- **Verifying Interactions:** Use `verify()` to ensure that expected method calls occurred. +- **Stubbing Method Calls:** Use `when(...).thenReturn(...)` to define the return values of mocked method calls. +- **Capturing Arguments:** Use `ArgumentCaptor` to capture and assert arguments passed to mocked methods. + +### 2.3 Anti-patterns and Code Smells to Avoid + +- **Over-mocking:** Avoid mocking classes that are easy to instantiate and use directly (e.g., value objects, simple data structures). +- **Testing Implementation Details:** Focus on testing behavior rather than implementation details. Tests should not break when you refactor code internally. +- **Ignoring Test Failures:** Investigate and fix test failures promptly. Don't ignore failing tests. +- **Writing Untestable Code:** Design code with testability in mind. Avoid tightly coupled code and dependencies. +- **Misusing Spies:** Use spies sparingly. If you need to spy on a class, it might be a sign that the class has too many responsibilities. +- **Chaining `when()` calls excessively:** If you have long chains of `when()` calls, it might indicate that your class under test is doing too much. +- **Using `reset()` frequently:** Frequent use of `reset()` might suggest that your tests are not properly isolated or that your mocks are holding too much state. + +### 2.4 State Management Best Practices + +- **Isolated Tests:** Each test should be independent of other tests. Avoid sharing state between tests. +- **Fresh Mocks:** Create fresh mock objects for each test using `@BeforeEach` or `@Before` (JUnit 4). +- **Avoid Static State:** Minimize the use of static variables or shared mutable state that could affect test results. + +### 2.5 Error Handling Patterns + +- **Expected Exceptions:** Use `@Test(expected = Exception.class)` (JUnit 4) or `assertThrows()` (JUnit 5) to verify that expected exceptions are thrown. +- **Exception Handling in Mocks:** Use `when(...).thenThrow(...)` to simulate exceptions thrown by mocked dependencies. +- **Verification of Exception Handling:** If the code under test catches exceptions, verify that appropriate error handling logic is executed (e.g., logging, retries). + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +- **Lazy Initialization:** Initialize mocks only when they are needed to reduce startup time. +- **Efficient Stubbing:** Stub only the methods that are actually used in the test case. +- **Avoid Excessive Verification:** Verify only the interactions that are relevant to the test case. + +### 3.2 Memory Management + +- **Limited Mock Scope:** Keep the scope of mock objects as narrow as possible (e.g., within a method). This allows them to be garbage collected sooner. +- **Avoid Large Mocks:** Avoid creating mocks that consume large amounts of memory (e.g., mocks with many fields or methods). + +### 3.3 Rendering Optimization (N/A) + +- Mockito is primarily a testing framework and does not directly involve rendering. + +### 3.4 Bundle Size Optimization (N/A) + +- Mockito is a testing dependency and is not included in the production bundle. + +### 3.5 Lazy Loading Strategies (Not Directly Applicable) + +- Lazy loading is typically used for loading data or resources. Mockito doesn't directly use lazy loading patterns, although you can lazily initialize mocks if necessary. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and How to Prevent Them + +- **Unintended Side Effects:** Ensure that mocked methods do not introduce unintended side effects that could compromise security. +- **Sensitive Data Exposure:** Avoid logging or storing sensitive data in mock objects. + +### 4.2 Input Validation Best Practices + +- **Validate Inputs in Production Code:** Mockito tests should verify that input validation is performed correctly in the production code. +- **Simulate Invalid Inputs:** Use Mockito to simulate invalid inputs and verify that the code handles them appropriately. + +### 4.3 Authentication and Authorization Patterns + +- **Mock Authentication/Authorization Services:** Mock authentication and authorization services to test different security scenarios (e.g., authorized vs. unauthorized users). +- **Verify Access Control:** Use Mockito to verify that access control checks are performed correctly in the production code. + +### 4.4 Data Protection Strategies + +- **Encryption:** If sensitive data is used in mock objects, ensure that it is encrypted. +- **Data Masking:** Use data masking techniques to protect sensitive data in mock objects. + +### 4.5 Secure API Communication + +- **Mock API Clients:** Mock API clients to simulate different API responses, including error conditions. +- **Verify Secure Communication:** Use Mockito to verify that secure communication protocols (e.g., HTTPS) are used when interacting with APIs. + +## 5. Testing Approaches + +### 5.1 Unit Testing Strategies + +- **Test Individual Units:** Unit tests should focus on testing individual units of code (e.g., classes, methods) in isolation. +- **Mock Dependencies:** Use Mockito to mock dependencies and control their behavior. +- **Test All Scenarios:** Write tests for all possible scenarios, including normal cases, edge cases, and error conditions. + +### 5.2 Integration Testing Approaches + +- **Test Interactions Between Units:** Integration tests should focus on testing the interactions between different units of code. +- **Use Real Dependencies (If Possible):** Use real dependencies whenever possible to ensure that the integration works correctly. +- **Mock External Systems:** Mock external systems (e.g., databases, APIs) to control their behavior and simulate different scenarios. + +### 5.3 End-to-End Testing Recommendations (Beyond Mockito) + +- **Use UI Testing Frameworks:** Use UI testing frameworks (e.g., Selenium, Cypress) to test the entire application flow. +- **Test User Interactions:** End-to-end tests should simulate real user interactions to ensure that the application works as expected. + +### 5.4 Test Organization Best Practices + +- **Separate Test Classes:** Create separate test classes for each class being tested. +- **Descriptive Test Names:** Use descriptive test names that clearly indicate what is being tested. +- **Arrange, Act, Assert:** Follow the Arrange, Act, Assert (AAA) pattern in your tests. +- **Keep Tests Short and Focused:** Keep tests short and focused on testing a single aspect of the code. + +### 5.5 Mocking and Stubbing Techniques + +- **`@Mock` Annotation:** Use `@Mock` to create mock objects. +- **`@InjectMocks` Annotation:** Use `@InjectMocks` to automatically inject mock objects into the class under test. +- **`when(...).thenReturn(...)`:** Use `when(...).thenReturn(...)` to stub method calls. +- **`verify(...)`:** Use `verify(...)` to verify that method calls occurred. +- **`ArgumentCaptor`:** Use `ArgumentCaptor` to capture and assert arguments passed to mocked methods. +- **`doReturn(...).when(...)`:** Use `doReturn(...).when(...)` when stubbing void methods or methods that are called multiple times. +- **`thenAnswer(...)`:** Use `thenAnswer(...)` to provide custom logic for stubbing method calls. +- **`thenThrow(...)`:** Use `thenThrow(...)` to simulate exceptions thrown by mocked methods. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes Developers Make + +- **Mocking Everything:** Mocking simple classes can lead to brittle tests. Use real objects when possible. +- **Testing Implementation Details:** Focus on testing behavior, not implementation. +- **Ignoring Test Failures:** Always investigate and fix test failures promptly. +- **Not Using Argument Matchers Properly:** When using argument matchers, ensure that all arguments are either exact values or matchers. +- **Forgetting to Initialize Mocks:** Ensure that mocks are initialized using `MockitoAnnotations.initMocks(this)` or `@ExtendWith(MockitoExtension.class)` (JUnit 5). + +### 6.2 Edge Cases to Be Aware Of + +- **Final Classes and Methods:** Mockito cannot mock final classes or methods by default. Consider using the Mockito inline mock maker or PowerMock (though PowerMock should be used as a last resort due to complexity). +- **Static Methods:** Mockito cannot mock static methods by default. Consider using the Mockito inline mock maker or PowerMock. +- **Equals and HashCode:** Mockito can have issues with `equals()` and `hashCode()` methods, especially when using argument matchers. Be careful when mocking classes that rely heavily on these methods. +- **Order of Interactions:** Mockito verifies interactions in the order they occur. Be mindful of the order of method calls when writing tests. + +### 6.3 Version-Specific Issues + +- **Mockito 2 vs. Mockito 3 vs. Mockito 4/5:** Be aware of the differences between Mockito versions. Some features may be deprecated or added in newer versions. +- **Java Version Compatibility:** Ensure that your Mockito version is compatible with your Java version. + +### 6.4 Compatibility Concerns + +- **JUnit Version:** Ensure that your Mockito version is compatible with your JUnit version. +- **Other Testing Frameworks:** Mockito can be used with other testing frameworks, but ensure that there are no compatibility issues. +- **IDE Integration:** Ensure that your IDE has proper support for Mockito and JUnit. + +### 6.5 Debugging Strategies + +- **Print Statements:** Use print statements to debug the code and verify that the mock objects are behaving as expected. +- **IDE Debugger:** Use the IDE debugger to step through the code and inspect the mock objects. +- **Mockito Spy:** Use a Mockito spy to partially mock a class and observe its behavior. +- **Verbose Logging:** Use verbose logging to see all method calls and their return values. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +- **IDE:** IntelliJ IDEA, Eclipse, or Visual Studio Code. +- **Build Tool:** Maven or Gradle. +- **Testing Framework:** JUnit 4 or JUnit 5. +- **Code Coverage Tool:** JaCoCo or SonarQube. + +### 7.2 Build Configuration Best Practices + +- **Dependency Management:** Use Maven or Gradle to manage Mockito dependencies. +- **Test Dependencies:** Declare Mockito as a test dependency to ensure that it is not included in the production code. +- **Plugin Configuration:** Configure Maven or Gradle plugins to run tests and generate code coverage reports. + +### 7.3 Linting and Formatting + +- **Code Style:** Follow a consistent code style (e.g., Google Java Style, Checkstyle). +- **Linting Rules:** Use linting tools (e.g., PMD, SpotBugs) to enforce coding standards and identify potential issues. +- **Formatting:** Use code formatters (e.g., IntelliJ IDEA formatter, Eclipse formatter) to automatically format your code. + +### 7.4 Deployment Best Practices (N/A) + +- Mockito is a testing dependency and is not deployed with the production code. + +### 7.5 CI/CD Integration + +- **Automated Builds:** Configure your CI/CD system (e.g., Jenkins, CircleCI, GitHub Actions) to automatically build and test your code. +- **Test Execution:** Run unit tests and integration tests as part of the CI/CD pipeline. +- **Code Coverage:** Generate code coverage reports as part of the CI/CD pipeline and set thresholds for code coverage. +- **Static Analysis:** Perform static analysis as part of the CI/CD pipeline to identify potential issues. + +## Conclusion + +By following these best practices, you can write more effective and maintainable tests using Mockito. Remember to focus on testing behavior, avoid over-mocking, and keep your tests short and focused. \ No newline at end of file diff --git a/.cursor/rules/modal.mdc b/.cursor/rules/modal.mdc new file mode 100644 index 000000000..4a58ce0d3 --- /dev/null +++ b/.cursor/rules/modal.mdc @@ -0,0 +1,227 @@ +--- +description: This rule outlines best practices for developing and maintaining the Modal library, covering code organization, performance, security, and testing. It aims to ensure high-quality, maintainable, and scalable cloud deployment solutions. +globs: *.py +--- +# Modal Library Best Practices + +This document outlines the best practices for developing and maintaining the Modal library. These guidelines cover various aspects of the library, including code organization, performance, security, testing, and deployment. Following these guidelines will ensure the Modal library remains a high-quality, maintainable, and scalable cloud deployment solution. + +## 1. Code Organization and Structure + +- **Directory Structure Best Practices:** + - Adopt a modular directory structure based on functionality or component type. + - Use clear and descriptive directory names. + - Example: + + modal/ + ├── api/ + │ ├── __init__.py + │ ├── routes.py + │ └── models.py + ├── config/ + │ ├── __init__.py + │ └── settings.py + ├── deployment/ + │ ├── __init__.py + │ ├── deploy.py + │ └── utils.py + ├── exceptions/ + │ ├── __init__.py + │ └── custom_exceptions.py + ├── security/ + │ ├── __init__.py + │ ├── auth.py + │ └── permissions.py + ├── testing/ + │ ├── __init__.py + │ ├── unit/ + │ └── integration/ + ├── utils/ + │ ├── __init__.py + │ └── helpers.py + ├── __init__.py + └── core.py + + +- **File Naming Conventions:** + - Use descriptive and consistent file names. + - Follow a consistent naming convention (e.g., `snake_case` for Python). + - Example: `user_authentication.py`, `database_connection.py` + +- **Module Organization:** + - Group related functions, classes, and data into modules. + - Use `__init__.py` files to define package structure. + - Ensure clear separation of concerns between modules. + - Each module should have a clearly defined purpose. + +- **Component Architecture:** + - Design the library using a component-based architecture. + - Each component should be self-contained and reusable. + - Components should have well-defined interfaces. + - Use dependency injection to manage component dependencies. + +- **Code Splitting Strategies:** + - Divide large modules into smaller, more manageable files. + - Split code based on functionality or feature. + - Consider using lazy loading for infrequently used modules. + +## 2. Common Patterns and Anti-patterns + +- **Design Patterns Specific to Modal:** + - **Singleton:** For managing global resources (e.g., configuration). + - **Factory:** For creating Modal clients with different configurations. + - **Observer:** For notifying subscribers of Modal state changes. + - **Strategy:** For different deployment strategies (e.g., different cloud providers). + +- **Recommended Approaches for Common Tasks:** + - **Configuration Management:** Use a dedicated configuration module to manage settings. + - **Logging:** Implement a robust logging system for debugging and monitoring. + - **Error Handling:** Use exception handling to gracefully handle errors. + - **Asynchronous Operations:** Use `asyncio` or similar libraries for handling asynchronous tasks. + - **Resource Management:** Use context managers (`with` statement) to ensure proper resource cleanup. + +- **Anti-patterns and Code Smells to Avoid:** + - **God Classes:** Avoid creating classes that are too large and complex. + - **Spaghetti Code:** Maintain a clear and well-structured codebase. + - **Magic Numbers:** Use constants instead of hardcoded values. + - **Duplicated Code:** Refactor duplicated code into reusable functions or classes. + - **Ignoring Errors:** Always handle exceptions appropriately. + +- **State Management Best Practices:** + - Prefer immutable data structures to avoid unintended side effects. + - Use a centralized state management system (if needed) for complex state. + - Consider using libraries like `attrs` or `dataclasses` for managing data objects. + +- **Error Handling Patterns:** + - Use try-except blocks to handle potential exceptions. + - Define custom exception classes for specific error conditions. + - Log exceptions with relevant information for debugging. + - Implement retry mechanisms for transient errors. + +## 3. Performance Considerations + +- **Optimization Techniques:** + - Profile code to identify performance bottlenecks. + - Optimize database queries and data access patterns. + - Use caching to reduce the load on external services. + - Avoid unnecessary computations and memory allocations. + +- **Memory Management:** + - Use generators and iterators to process large datasets efficiently. + - Avoid creating large objects in memory unnecessarily. + - Use memory profiling tools to identify memory leaks. + +- **Rendering Optimization (if applicable):** + - N/A (Modal library is primarily backend-focused). + +- **Bundle Size Optimization (if applicable):** + - N/A (Modal library is primarily backend-focused). + +- **Lazy Loading Strategies:** + - Use lazy loading to load modules and resources only when they are needed. + - This can reduce startup time and memory usage. + +## 4. Security Best Practices + +- **Common Vulnerabilities and How to Prevent Them:** + - **Injection Attacks:** Sanitize user inputs to prevent SQL injection, command injection, etc. + - **Cross-Site Scripting (XSS):** N/A (Modal library is primarily backend-focused). + - **Cross-Site Request Forgery (CSRF):** N/A (Modal library is primarily backend-focused). + - **Authentication and Authorization Flaws:** Implement secure authentication and authorization mechanisms. + - **Data Exposure:** Protect sensitive data by encrypting it at rest and in transit. + +- **Input Validation:** + - Validate all user inputs to prevent malicious data from entering the system. + - Use data validation libraries (e.g., `marshmallow`, `pydantic`) to enforce data types and constraints. + +- **Authentication and Authorization Patterns:** + - Use a strong authentication mechanism (e.g., OAuth 2.0, JWT). + - Implement role-based access control (RBAC) to restrict access to sensitive resources. + - Avoid storing passwords in plain text; use password hashing algorithms (e.g., bcrypt, Argon2). + +- **Data Protection Strategies:** + - Encrypt sensitive data at rest and in transit. + - Use HTTPS to secure communication between clients and servers. + - Implement data masking and anonymization techniques to protect sensitive data. + +- **Secure API Communication:** + - Use API keys or tokens to authenticate API requests. + - Implement rate limiting to prevent abuse. + - Use input validation to prevent malicious data from entering the system. + +## 5. Testing Approaches + +- **Unit Testing Strategies:** + - Write unit tests for all critical functions and classes. + - Use mocking and stubbing to isolate units of code. + - Aim for high test coverage. + +- **Integration Testing:** + - Write integration tests to verify the interaction between different components. + - Test the integration with external services (e.g., databases, APIs). + +- **End-to-End Testing:** + - Write end-to-end tests to verify the entire system functionality. + - Simulate real-world scenarios. + +- **Test Organization:** + - Organize tests into separate directories based on functionality or component. + - Use a consistent naming convention for test files and functions. + +- **Mocking and Stubbing:** + - Use mocking libraries (e.g., `unittest.mock`, `pytest-mock`) to isolate units of code. + - Create stubs for external dependencies. + +## 6. Common Pitfalls and Gotchas + +- **Frequent Mistakes Developers Make:** + - Improper error handling. + - Lack of input validation. + - Security vulnerabilities. + - Performance bottlenecks. + - Inadequate testing. + +- **Edge Cases to Be Aware Of:** + - Handling of large datasets. + - Concurrent access to shared resources. + - Network failures and timeouts. + - Unexpected input values. + +- **Version-Specific Issues:** + - Be aware of compatibility issues between different versions of the library and its dependencies. + - Test the library with different versions of Python and other dependencies. + +- **Compatibility Concerns:** + - Ensure the library is compatible with different operating systems and environments. + +- **Debugging Strategies:** + - Use logging to trace the execution flow and identify errors. + - Use debuggers to step through the code and inspect variables. + - Use profiling tools to identify performance bottlenecks. + +## 7. Tooling and Environment + +- **Recommended Development Tools:** + - IDE: VS Code, PyCharm + - Debugger: pdb, ipdb + - Profiler: cProfile, line_profiler + - Testing Framework: pytest, unittest + - Mocking Library: unittest.mock, pytest-mock + - Linting and Formatting: pylint, flake8, black + +- **Build Configuration:** + - Use a build system (e.g., `setuptools`, `poetry`) to manage dependencies and build the library. + - Define a clear build process. + +- **Linting and Formatting:** + - Use linters (e.g., `pylint`, `flake8`) to enforce code style and identify potential errors. + - Use formatters (e.g., `black`) to automatically format the code. + +- **Deployment Best Practices:** + - Use a deployment automation tool (e.g., Ansible, Terraform) to automate the deployment process. + - Use infrastructure-as-code (IaC) to manage infrastructure. + - Monitor the deployment process for errors. + +- **CI/CD Integration:** + - Integrate the library with a CI/CD pipeline to automate testing and deployment. + - Use tools like Jenkins, GitLab CI, or GitHub Actions. \ No newline at end of file diff --git a/.cursor/rules/mongodb.mdc b/.cursor/rules/mongodb.mdc new file mode 100644 index 000000000..c0af569cf --- /dev/null +++ b/.cursor/rules/mongodb.mdc @@ -0,0 +1,148 @@ +--- +description: Comprehensive best practices for developing with MongoDB, covering schema design, code organization, performance optimization, security considerations, and testing strategies. This rule provides actionable guidance to help developers build robust and scalable MongoDB applications. +globs: *.js,*.ts,*.mongodb +--- +- **Understand Schema Differences Between Relational and Document-based Databases:** Recognize that MongoDB's document-oriented model differs significantly from relational databases. Design schemas that reflect the relationships within the data itself, rather than relying on joins. +- **Embed Your Data Instead of Relying on Joins:** Favor embedding related data within a single document to minimize the need for costly join operations. This approach can significantly improve read performance. +- **Use Indexes For Frequent Operations:** Create indexes on fields that are frequently queried to optimize performance. Carefully consider the types of queries your application will perform and create indexes accordingly. Compound indexes are beneficial for queries involving multiple fields. +- **Properly Size Your Servers:** Ensure that your MongoDB server resources are adequately sized for your workload. Monitor resource utilization and scale as needed to maintain optimal performance. Consider CPU, memory, and disk I/O. +- **Use Replication or Sharding:** Implement replication or sharding to enhance scalability and reliability. Replication provides data redundancy and high availability, while sharding distributes data across multiple servers to handle larger datasets and higher traffic volumes. + +### 1. Code Organization and Structure: + +- **Directory Structure Best Practices:** + - `config/`: Contains configuration files for database connections, authentication, and other settings. + - `models/`: Defines data models using Mongoose or other ODM libraries. Each model represents a MongoDB collection. + - `routes/`: Handles API endpoints and routes for interacting with the database. + - `controllers/`: Implements the logic for handling requests, interacting with models, and returning responses. + - `services/`: Contains reusable business logic related to data access and manipulation. + - `utils/`: Provides utility functions for common tasks such as validation, formatting, and error handling. + - `tests/`: Includes unit, integration, and end-to-end tests. +- **File Naming Conventions:** + - Use descriptive names that reflect the purpose of the file (e.g., `user.model.js`, `auth.controller.ts`, `product.service.js`). + - Follow a consistent naming convention across the project (e.g., camelCase or snake_case). +- **Module Organization:** + - Organize code into modules based on functionality or domain (e.g., `user` module, `product` module). + - Use ES modules or CommonJS modules to encapsulate code and manage dependencies. +- **Component Architecture:** + - Design reusable components for common tasks such as data validation, error handling, and authentication. + - Follow the principles of separation of concerns and single responsibility. +- **Code Splitting Strategies:** + - Implement lazy loading of modules or components to improve initial load time. + - Use code splitting to break down large bundles into smaller chunks. + +### 2. Common Patterns and Anti-patterns: + +- **Design Patterns Specific to MongoDB:** + - **Embedded Document Pattern:** Embed related data within a single document to minimize the need for joins. + - **Polymorphic Pattern:** Store different types of documents in the same collection using a common base schema and discriminator fields. + - **Bucket Pattern:** Group data into buckets based on time or other criteria for efficient querying and aggregation. +- **Recommended Approaches for Common Tasks:** + - Use Mongoose or other ODM libraries to simplify data modeling and validation. + - Implement pagination for large result sets. + - Use aggregation pipelines for complex queries and data transformations. +- **Anti-patterns and Code Smells to Avoid:** + - **Over-indexing:** Creating too many indexes can degrade write performance. + - **Ignoring Performance:** Neglecting to analyze query performance can lead to slow response times. + - **Schema Violations:** Allowing inconsistent data to be stored in collections can cause unexpected errors. +- **State Management Best Practices:** + - Use a state management library such as Redux or Zustand to manage application state. + - Store state in a centralized store to ensure consistency and predictability. +- **Error Handling Patterns:** + - Implement robust error handling to catch and handle exceptions gracefully. + - Use try-catch blocks to handle potential errors. + - Log errors for debugging and monitoring. + +### 3. Performance Considerations: + +- **Optimization Techniques:** + - Use indexes to optimize query performance. + - Avoid using `$where` operator, as it can be slow. + - Use projection to retrieve only the necessary fields. +- **Memory Management:** + - Monitor memory usage and identify potential memory leaks. + - Use connection pooling to reuse database connections. +- **Rendering Optimization:** (If applicable for UI-based apps using MongoDB data) + - Implement virtualization for large lists. + - Use memoization to avoid unnecessary re-renders. +- **Bundle Size Optimization:** (If applicable) + - Minify and compress JavaScript and CSS files. + - Remove unused code. +- **Lazy Loading Strategies:** (If applicable) + - Lazy load images and other resources. + - Use code splitting to load modules on demand. + +### 4. Security Best Practices: + +- **Common Vulnerabilities and How to Prevent Them:** + - **NoSQL Injection:** Sanitize user inputs to prevent injection attacks. + - **Authentication Bypass:** Implement strong authentication and authorization mechanisms. + - **Data Exposure:** Protect sensitive data by encrypting it and controlling access. +- **Input Validation:** + - Validate all user inputs to prevent malicious data from being stored in the database. + - Use a validation library such as Joi or Yup to define validation schemas. +- **Authentication and Authorization Patterns:** + - Use a secure authentication protocol such as OAuth 2.0 or JWT. + - Implement role-based access control (RBAC) to restrict access to sensitive data and functionality. +- **Data Protection Strategies:** + - Encrypt sensitive data at rest and in transit. + - Use data masking to protect personally identifiable information (PII). +- **Secure API Communication:** + - Use HTTPS to encrypt communication between the client and server. + - Implement rate limiting to prevent abuse. + +### 5. Testing Approaches: + +- **Unit Testing Strategies:** + - Write unit tests to verify the functionality of individual modules and components. + - Use a testing framework such as Jest or Mocha. +- **Integration Testing:** + - Write integration tests to verify the interaction between different modules and components. + - Test the integration between the application and the database. +- **End-to-End Testing:** + - Write end-to-end tests to verify the functionality of the entire application. + - Use a testing framework such as Cypress or Playwright. +- **Test Organization:** + - Organize tests into separate directories based on functionality or module. + - Use descriptive names for test files and test cases. +- **Mocking and Stubbing:** + - Use mocking and stubbing to isolate units of code during testing. + - Use a mocking library such as Sinon or Jest's built-in mocking. + +### 6. Common Pitfalls and Gotchas: + +- **Frequent Mistakes Developers Make:** + - **Not Understanding MongoDB's Query Language:** Failing to grasp the nuances of MongoDB's query language can lead to inefficient queries. + - **Ignoring Indexes:** Neglecting to create indexes on frequently queried fields can significantly impact performance. + - **Not Handling Errors Properly:** Failing to handle errors gracefully can lead to unexpected application behavior. +- **Edge Cases to Be Aware Of:** + - **Data Type Mismatches:** Ensure that data types are consistent across the application and the database. + - **Concurrency Issues:** Handle concurrency issues carefully to prevent data corruption. +- **Version-Specific Issues:** + - Be aware of compatibility issues between different versions of MongoDB and related libraries. + - Consult the documentation for the specific version you are using. +- **Compatibility Concerns:** + - Ensure that the application is compatible with different operating systems and browsers. + - Test the application on different devices and screen sizes. +- **Debugging Strategies:** + - Use logging to track the execution flow and identify potential issues. + - Use a debugger to step through the code and inspect variables. + +### 7. Tooling and Environment: + +- **Recommended Development Tools:** + - **MongoDB Compass:** A GUI tool for exploring and managing MongoDB databases. + - **MongoDB Shell:** A command-line interface for interacting with MongoDB. + - **VS Code Extension for MongoDB:** Provides syntax highlighting, code completion, and other features for MongoDB development. +- **Build Configuration:** + - Use a build tool such as Webpack or Parcel to bundle and optimize the application. + - Configure the build tool to minify and compress JavaScript and CSS files. +- **Linting and Formatting:** + - Use a linter such as ESLint to enforce coding standards. + - Use a formatter such as Prettier to automatically format code. +- **Deployment Best Practices:** + - Use a containerization technology such as Docker to package the application and its dependencies. + - Deploy the application to a cloud platform such as AWS, Azure, or Google Cloud. +- **CI/CD Integration:** + - Integrate the application with a CI/CD pipeline to automate the build, test, and deployment process. + - Use a CI/CD tool such as Jenkins, Travis CI, or CircleCI. \ No newline at end of file diff --git a/.cursor/rules/mypy.mdc b/.cursor/rules/mypy.mdc new file mode 100644 index 000000000..d53c1961a --- /dev/null +++ b/.cursor/rules/mypy.mdc @@ -0,0 +1,101 @@ +--- +description: This rule file outlines best practices for using mypy in Python projects, emphasizing gradual adoption, consistent configuration, and leveraging advanced features for improved code quality and maintainability. It covers code organization, performance, security, testing, common pitfalls, and tooling. +globs: *.py +--- +# Mypy Best Practices and Coding Standards + +This document outlines the recommended best practices for using Mypy in Python projects. Following these guidelines can lead to more maintainable, robust, and understandable code. + +## 1. Gradual Typing and Adoption + +- **Start Small:** When introducing Mypy to an existing codebase, focus on a manageable subset of the code first. Choose modules or files that are relatively isolated or self-contained. +- **Iterative Annotation:** Gradually increase the coverage by adding type hints as you modify or add new code. Avoid large-scale refactoring solely for the purpose of adding type hints. +- **`# type: ignore` Strategically:** Use `# type: ignore` comments sparingly to temporarily suppress errors in code that is not yet fully typed. Always include a specific error code when using `# type: ignore` to avoid unintentionally ignoring other errors. Review these regularly. +- **Prioritize Widely Imported Modules:** Focus on annotating modules that are imported by many other modules. This will provide the greatest benefit in terms of type checking and error detection. + +## 2. Consistent Configuration and Integration + +- **Standardized Configuration:** Ensure that all developers use the same Mypy configuration and version. Use a `mypy.ini` file or a `pyproject.toml` file to define the project's Mypy settings. This file should be version-controlled. +- **CI Integration:** Integrate Mypy checks into your Continuous Integration (CI) pipeline to catch type errors early in the development process. Use a pre-commit hook to run Mypy before committing code. +- **Editor Integration:** Encourage developers to use Mypy integration in their code editors to get real-time feedback on type errors. Most popular Python editors, such as VS Code, PyCharm, and Sublime Text, have Mypy plugins or extensions. +- **Version Pinning:** Pin the version of mypy in your project's dependencies to ensure consistent behavior across different environments. + +## 3. Leveraging Advanced Features + +- **Strict Mode:** Utilize Mypy's strict mode (enabled with the `--strict` flag) to catch more potential errors. Strict mode enables a collection of stricter type checking options. +- **`--warn-unused-ignores`:** Use this flag to identify `# type: ignore` comments that are no longer necessary because the corresponding errors have been fixed. +- **`--disallow-untyped-defs`:** Use this flag to require type annotations for all function definitions. +- **`--disallow-incomplete-defs`:** Use this flag to disallow function definitions with incomplete type annotations. +- **`--check-untyped-defs`:** Use this flag to type check function bodies even if the signature lacks type annotations. +- **Protocols and Structural Subtyping:** Take advantage of Mypy's support for protocols and structural subtyping to define flexible interfaces and improve code reusability. +- **Generics:** Use generics to write type-safe code that can work with different types of data. +- **TypedDict:** Use `TypedDict` to define the types of dictionaries with known keys and values. This can help to prevent errors when working with data structures. + +## 4. Code Organization and Structure + +- **Directory Structure:** Use a well-defined directory structure to organize your code. A common pattern is the `src` layout, where the project's source code is located in a `src` directory. +- **File Naming Conventions:** Follow consistent file naming conventions. Use lowercase letters and underscores for module names (e.g., `my_module.py`). +- **Module Organization:** Organize your code into logical modules. Each module should have a clear purpose and a well-defined interface. +- **Component Architecture:** Design your application using a component-based architecture. Each component should be responsible for a specific task and should have well-defined inputs and outputs. +- **Code Splitting:** Split large modules into smaller, more manageable files. This can improve code readability and maintainability. + +## 5. Common Patterns and Anti-patterns + +- **Dependency Injection:** Use dependency injection to decouple components and improve testability. +- **Abstract Factories:** Use abstract factories to create families of related objects without specifying their concrete classes. +- **Singletons:** Avoid using singletons excessively. They can make code harder to test and reason about. +- **Global State:** Minimize the use of global state. It can make code harder to understand and debug. +- **Exception Handling:** Use exception handling to gracefully handle errors and prevent the application from crashing. Avoid catching generic exceptions (e.g., `except Exception:`). Catch specific exceptions and handle them appropriately. + +## 6. Performance Considerations + +- **Profiling:** Use profiling tools to identify performance bottlenecks in your code. +- **Caching:** Use caching to store frequently accessed data and reduce the number of expensive operations. +- **Lazy Loading:** Use lazy loading to defer the loading of resources until they are actually needed. +- **Efficient Data Structures:** Choose appropriate data structures for your data. For example, use sets for membership testing and dictionaries for key-value lookups. +- **Avoid Unnecessary Copying:** Avoid making unnecessary copies of data. This can consume memory and slow down your code. + +## 7. Security Best Practices + +- **Input Validation:** Validate all user inputs to prevent injection attacks and other security vulnerabilities. Use type annotations to enforce type constraints on function arguments. +- **Authentication and Authorization:** Implement robust authentication and authorization mechanisms to protect your application from unauthorized access. +- **Data Protection:** Protect sensitive data by encrypting it and storing it securely. +- **Secure API Communication:** Use HTTPS to encrypt communication between your application and external APIs. +- **Dependency Management:** Regularly audit your project's dependencies for security vulnerabilities and update them to the latest versions. + +## 8. Testing Approaches + +- **Unit Testing:** Write unit tests for all components in your application. Unit tests should verify that each component behaves as expected in isolation. +- **Integration Testing:** Write integration tests to verify that different components in your application work together correctly. +- **End-to-End Testing:** Write end-to-end tests to verify that the entire application works as expected from the user's perspective. +- **Test Organization:** Organize your tests into a clear and logical structure. Use separate directories for unit tests, integration tests, and end-to-end tests. +- **Mocking and Stubbing:** Use mocking and stubbing to isolate components during testing and to simulate external dependencies. + +## 9. Common Pitfalls and Gotchas + +- **`Any` Type:** Avoid using the `Any` type excessively. It effectively disables type checking for the corresponding code. +- **Inconsistent Type Annotations:** Ensure that type annotations are consistent throughout your codebase. Inconsistent type annotations can lead to unexpected errors. +- **Ignoring Errors:** Avoid ignoring Mypy errors without a good reason. Mypy errors usually indicate a real problem in your code. +- **Version Compatibility:** Be aware of compatibility issues between different versions of Mypy and other libraries. +- **Circular Dependencies:** Avoid circular dependencies between modules. They can make code harder to understand and test. + +## 10. Tooling and Environment + +- **Development Tools:** Use a good code editor with Mypy integration, such as VS Code, PyCharm, or Sublime Text. +- **Build Configuration:** Use a build system, such as `setuptools` or `poetry`, to manage your project's dependencies and build process. +- **Linting and Formatting:** Use a linter, such as `flake8` or `ruff`, to enforce code style and detect potential errors. Use a code formatter, such as `black` or `ruff`, to automatically format your code. +- **Deployment:** Deploy your application to a production environment using a container system, such as Docker, or a cloud platform, such as AWS or Google Cloud. +- **CI/CD:** Integrate Mypy into your CI/CD pipeline to automatically check your code for type errors before deploying it to production. + +## 11. Additional Best Practices + +- **Use type hints for all function arguments and return values.** This makes it easier to understand what types of data the function expects and returns. +- **Use type aliases to simplify complex type annotations.** This can make your code more readable and maintainable. +- **Use the `typing` module to access advanced type features.** The `typing` module provides a number of useful type-related classes and functions, such as `List`, `Dict`, `Union`, and `Optional`. +- **Use the `reveal_type()` function to inspect the type of an expression.** This can be helpful for debugging type-related issues. +- **Keep your code clean and well-organized.** This makes it easier to understand and maintain. +- **Write clear and concise comments.** This helps others understand your code and how it works. +- **Follow the PEP 8 style guide.** This ensures that your code is consistent and readable. +- **Use a version control system.** This allows you to track changes to your code and collaborate with others. + +By following these best practices, you can improve the quality, maintainability, and robustness of your Python code that uses mypy and other Python tools. \ No newline at end of file diff --git a/.cursor/rules/neo4j.mdc b/.cursor/rules/neo4j.mdc new file mode 100644 index 000000000..d9c091def --- /dev/null +++ b/.cursor/rules/neo4j.mdc @@ -0,0 +1,278 @@ +--- +description: This rule provides guidelines for best practices and coding standards when developing applications with Neo4j. It covers aspects from code organization and performance to security and testing. +globs: *.cypher +--- +# Neo4j Development Best Practices + +This document outlines best practices and coding standards for developing applications using Neo4j. These guidelines are designed to promote maintainability, performance, and security. + +Library Information: +- Name: neo4j +- Tags: database, graph, nosql, relationships + +## 1. Code Organization and Structure + +### 1.1 Directory Structure + +Organize your project with a clear directory structure that separates concerns. A recommended structure is as follows: + + +project_root/ +├── data/ # Contains data files for import/export +├── queries/ # Stores Cypher queries +├── src/ # Source code for the application +│ ├── models/ # Defines data models and graph schemas +│ ├── services/ # Contains business logic and Neo4j interactions +│ ├── utils/ # Utility functions and helper classes +│ ├── config/ # Configuration files +│ └── app.js # Main application file +├── tests/ # Unit, integration, and end-to-end tests +├── .env # Environment variables +├── package.json # Node.js project configuration +├── requirements.txt # Python project dependencies +└── README.md + + +### 1.2 File Naming Conventions + +* **Cypher Queries:** Use descriptive names (e.g., `get_user_friends.cypher`). +* **Models:** Name files according to the entity they represent (e.g., `user.js`, `movie.py`). +* **Services:** Use a service-based naming convention (e.g., `user_service.js`, `movie_service.py`). +* **Tests:** Match test file names to the source file names (e.g., `user_service.test.js`). + +### 1.3 Module Organization + +Break down your application into modules based on functionality. Use well-defined interfaces and avoid circular dependencies. + +* **Node.js:** Use ES modules (`import`, `export`) or CommonJS (`require`, `module.exports`). +* **Python:** Utilize packages and modules for organizing code. + +### 1.4 Component Architecture + +Design a component architecture that promotes reusability and maintainability. Consider using patterns like Model-View-Controller (MVC) or a layered architecture. + +* **Models:** Define data structures and interact with the Neo4j database. +* **Services:** Implement business logic and handle data manipulation. +* **Controllers (or equivalent):** Handle user requests and orchestrate interactions between models and services. + +### 1.5 Code Splitting + +For large applications, use code splitting to improve initial load times. Load modules and components on demand when they are needed. + +* **Node.js:** Use dynamic imports (`import()`) for on-demand loading. +* **Frontend Frameworks (if applicable):** Use framework-specific code-splitting techniques (e.g., React.lazy, Vue.js's async components). + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns + +* **Repository Pattern:** Abstract data access logic behind a repository interface. This makes it easier to switch database implementations or mock data access for testing. +* **Unit of Work:** Group multiple database operations into a single transaction to ensure atomicity. +* **Data Mapper:** Transfer data between domain objects and the database. +* **Graph Traversal Pattern:** Encapsulate common graph traversal logic into reusable functions or classes. + +### 2.2 Recommended Approaches for Common Tasks + +* **Creating Nodes and Relationships:** Use Cypher queries with parameters to avoid SQL injection and improve performance. +* **Querying Data:** Use Cypher's `MATCH` clause for efficient graph traversal. Leverage indexes and constraints for optimal query performance. +* **Data Validation:** Validate data before inserting it into the database. Use constraints to enforce data integrity at the database level. +* **Error Handling:** Implement robust error handling to gracefully handle database errors and prevent application crashes. + +### 2.3 Anti-patterns and Code Smells + +* **Over-fetching Data:** Avoid retrieving unnecessary data from the database. Use projections in Cypher queries to select only the required properties. +* **Long Cypher Queries:** Break down complex Cypher queries into smaller, more manageable queries. +* **Lack of Indexing:** Ensure that frequently queried properties are indexed to improve query performance. +* **Ignoring Constraints:** Define and enforce constraints to maintain data integrity and consistency. +* **Hardcoding Values:** Avoid hardcoding values in Cypher queries. Use parameters instead. +* **Excessive Relationship Traversal in Application Code:** Prefer to execute complex relationship traversals within Cypher rather than in application code which reduces the amount of data transported and is significantly faster. + +### 2.4 State Management + +* **Stateless Services:** Design services to be stateless to improve scalability and testability. +* **Session Management:** Use appropriate session management techniques for web applications. +* **Caching:** Implement caching to reduce database load and improve response times. + +### 2.5 Error Handling + +* **Centralized Error Handling:** Implement a centralized error handling mechanism to handle exceptions consistently. +* **Logging:** Log errors and warnings to help with debugging and monitoring. +* **Retry Logic:** Implement retry logic for transient database errors. +* **Custom Exceptions:** Define custom exceptions for specific error conditions. +* **Graceful Degradation:** Design the application to degrade gracefully in case of database failures. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +* **Indexing:** Create indexes on frequently queried properties. +* **Constraints:** Use constraints to enforce data integrity and improve query performance. +* **Query Optimization:** Analyze Cypher query execution plans and optimize queries for performance. +* **Connection Pooling:** Use connection pooling to reuse database connections and reduce connection overhead. +* **Batch Operations:** Use batch operations to insert or update multiple nodes and relationships in a single transaction. +* **Profile Queries:** Use `PROFILE` or `EXPLAIN` to understand query performance. +* **Use `apoc.periodic.iterate` for batch processing** When dealing with large datasets, `apoc.periodic.iterate` allows for batch processing and avoids exceeding memory limits. + +### 3.2 Memory Management + +* **Limit Result Set Size:** Use `LIMIT` in Cypher queries to restrict the number of returned results. +* **Stream Data:** Stream data from the database to avoid loading large amounts of data into memory. +* **Garbage Collection:** Monitor garbage collection and tune JVM settings for optimal performance (Java-based implementations). + +### 3.3 Bundle Size Optimization + +* **Tree shaking** remove unused code +* **Minification:** Minify code to reduce bundle size. +* **Compression:** Compress bundles to reduce transfer size. + +### 3.4 Lazy Loading + +* **On-Demand Loading:** Load data and components on demand when they are needed. +* **Pagination:** Use pagination to load data in smaller chunks. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities + +* **Cypher Injection:** Prevent Cypher injection by using parameterized queries. +* **Authentication Bypass:** Secure authentication mechanisms and avoid relying on client-side authentication. +* **Data Exposure:** Protect sensitive data by encrypting it at rest and in transit. +* **Authorization Flaws:** Implement robust authorization mechanisms to control access to resources. + +### 4.2 Input Validation + +* **Sanitize Inputs:** Sanitize user inputs to prevent Cross-Site Scripting (XSS) attacks. +* **Validate Inputs:** Validate user inputs to ensure they conform to expected formats and values. +* **Parameterize Queries:** Always use parameterized queries to prevent Cypher injection. + +### 4.3 Authentication and Authorization + +* **Secure Authentication:** Use strong authentication mechanisms such as OAuth 2.0 or JWT. +* **Role-Based Access Control (RBAC):** Implement RBAC to control access to resources based on user roles. +* **Least Privilege Principle:** Grant users only the minimum necessary permissions. +* **Neo4j's built-in security:** Utilize Neo4j's built-in authentication and authorization mechanisms for database access. + +### 4.4 Data Protection + +* **Encryption at Rest:** Encrypt sensitive data at rest using Neo4j's encryption features or third-party encryption solutions. +* **Encryption in Transit:** Use HTTPS to encrypt data in transit. +* **Data Masking:** Mask sensitive data in logs and reports. +* **Regular Backups:** Perform regular backups to protect against data loss. +* **Database Auditing:** Enable database auditing to track access and modifications to data. +* **Avoid Storing Sensitive Data:** Only store necessary sensitive data. Consider tokenization or anonymization where applicable. + +### 4.5 Secure API Communication + +* **HTTPS:** Use HTTPS for all API communication. +* **API Keys:** Use API keys to authenticate API requests. +* **Rate Limiting:** Implement rate limiting to prevent abuse. +* **Input Validation:** Validate API requests to prevent malicious input. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +* **Test Individual Components:** Unit test individual components in isolation. +* **Mock Dependencies:** Use mocking to isolate components from external dependencies. +* **Test Edge Cases:** Test edge cases and boundary conditions. +* **Test Data Validation** Unit tests should cover data validation logic. + +### 5.2 Integration Testing + +* **Test Interactions:** Test the interactions between different components. +* **Test Database Interactions:** Test the interactions between the application and the Neo4j database. +* **Use Test Databases:** Use separate test databases for integration tests. + +### 5.3 End-to-End Testing + +* **Test Full Workflows:** Test the complete end-to-end workflows of the application. +* **Automate Tests:** Automate end-to-end tests to ensure consistent results. + +### 5.4 Test Organization + +* **Organize Tests:** Organize tests in a clear and logical manner. +* **Use Test Suites:** Use test suites to group related tests together. +* **Naming Convention:** Follow a clear naming convention for test files and test methods. + +### 5.5 Mocking and Stubbing + +* **Mock Neo4j Driver:** Mock the Neo4j driver to isolate components from the database. +* **Stub Responses:** Stub database responses to control the data returned by the database. +* **Verify Interactions:** Verify that components interact with the database as expected. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +* **Lack of Planning:** Failing to properly plan the graph schema and data model. +* **Ignoring Performance:** Neglecting to optimize Cypher queries and database configuration. +* **Poor Security:** Failing to implement proper security measures. +* **Insufficient Testing:** Insufficient testing leading to bugs and regressions. +* **Not Utilizing Indexes:** Neglecting to create indexes on frequently queried properties. + +### 6.2 Edge Cases + +* **Large Graphs:** Handling very large graphs with millions or billions of nodes and relationships. +* **Concurrent Access:** Managing concurrent access to the database. +* **Transaction Management:** Properly managing transactions to ensure data consistency. +* **Handling Null Values:** Understanding how Neo4j handles null values and handling them appropriately. + +### 6.3 Version-Specific Issues + +* **API Changes:** Be aware of API changes between different versions of the Neo4j driver and database. +* **Cypher Syntax:** Be aware of changes to the Cypher syntax in different versions of Neo4j. +* **Deprecated Features:** Avoid using deprecated features. + +### 6.4 Compatibility Concerns + +* **Driver Compatibility:** Ensure that the Neo4j driver is compatible with the version of the Neo4j database. +* **Operating System Compatibility:** Ensure that the application is compatible with the target operating system. +* **Java Version Compatibility:** Ensure the Java version is compatible (if using Java-based drivers). + +### 6.5 Debugging Strategies + +* **Logging:** Use logging to track the execution of the application and identify errors. +* **Debuggers:** Use debuggers to step through the code and inspect variables. +* **Neo4j Browser:** Use the Neo4j Browser to visualize the graph and execute Cypher queries. +* **Cypher Profiler:** Use the Cypher profiler to analyze the performance of Cypher queries. +* **APOC Procedures:** Use APOC Procedures to aid with debugging and monitoring. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +* **Neo4j Browser:** A web-based interface for interacting with the Neo4j database. +* **Neo4j Desktop:** A desktop application for managing Neo4j databases. +* **IntelliJ IDEA/PyCharm:** IDEs with excellent support for Neo4j development. +* **VS Code:** Popular code editor with Neo4j extensions. +* **APOC Library:** Provides many helpful stored procedures. + +### 7.2 Build Configuration + +* **Dependency Management:** Use a dependency management tool (e.g., npm, pip) to manage project dependencies. +* **Environment Variables:** Use environment variables to configure the application for different environments. +* **Build Scripts:** Use build scripts to automate the build process. + +### 7.3 Linting and Formatting + +* **ESLint/Pylint:** Use linters to enforce coding standards and identify potential errors. +* **Prettier/Black:** Use formatters to automatically format code. +* **Consistent Style:** Maintain a consistent coding style throughout the project. + +### 7.4 Deployment Best Practices + +* **Containerization:** Use containerization (e.g., Docker) to package the application and its dependencies. +* **Cloud Deployment:** Deploy the application to a cloud platform (e.g., AWS, Azure, GCP). +* **Load Balancing:** Use load balancing to distribute traffic across multiple instances of the application. +* **Monitoring:** Monitor the application to detect and respond to issues. +* **Immutable Infrastructure:** Treat servers as immutable; rebuild instead of modifying. + +### 7.5 CI/CD Integration + +* **Automated Builds:** Automate the build process using a CI/CD pipeline. +* **Automated Tests:** Run automated tests as part of the CI/CD pipeline. +* **Automated Deployments:** Automate the deployment process using a CI/CD pipeline. +* **Version Control:** Use version control (e.g., Git) to manage the codebase. +* **Trunk-Based Development:** Consider trunk-based development for faster feedback cycles. + +By following these best practices, developers can build robust, scalable, and secure Neo4j applications. \ No newline at end of file diff --git a/.cursor/rules/nestjs.mdc b/.cursor/rules/nestjs.mdc new file mode 100644 index 000000000..847fec924 --- /dev/null +++ b/.cursor/rules/nestjs.mdc @@ -0,0 +1,178 @@ +--- +description: This rule provides comprehensive guidance on NestJS best practices, coding standards, and architectural patterns. It aims to help developers build scalable, maintainable, and performant NestJS applications by covering code organization, security, testing, and other essential aspects. +globs: *.ts +--- +- **Code Organization and Structure:** + - **Directory Structure:** + - Adopt a modular structure that reflects the application's domain or features. A common approach includes organizing code into modules, services, controllers, DTOs, and entities, each in their own directory. + - Example: + + src/ + ├── app.module.ts + ├── auth/ + │ ├── auth.module.ts + │ ├── auth.controller.ts + │ ├── auth.service.ts + │ ├── strategies/ + │ │ └── jwt.strategy.ts + │ ├── dtos/ + │ │ └── create-user.dto.ts + │ └── entities/ + │ └── user.entity.ts + ├── users/ + │ ├── users.module.ts + │ ├── users.controller.ts + │ ├── users.service.ts + │ └── ... + ├── core/ + │ ├── filters/ + │ │ └── http-exception.filter.ts + │ ├── interceptors/ + │ │ └── logging.interceptor.ts + │ └── ... + └── main.ts + + - **File Naming Conventions:** + - Use descriptive and consistent naming conventions. Prefix files based on their role (e.g., `user.controller.ts`, `auth.service.ts`, `create-user.dto.ts`). + - Use PascalCase for classes and interfaces (e.g., `UserService`, `CreateUserDto`). + - Use camelCase for instances and variables (e.g., `userService`, `createUserDto`). + - **Module Organization:** + - Encapsulate features within modules. Each module should represent a distinct part of the application and handle related functionality. + - Modules should import necessary dependencies and export components that other modules need. + - Use the `forRoot` and `forFeature` methods for configuration and feature modules, respectively, especially when dealing with database connections or other shared resources. + - **Component Architecture:** + - Follow the SOLID principles for designing components. Each component (controller, service, etc.) should have a single responsibility. + - Use dependency injection to manage dependencies between components, making them more testable and maintainable. + - Controllers should handle request routing and validation, services should implement business logic, and entities should represent data models. + - **Code Splitting Strategies:** + - For large applications, consider splitting modules into smaller, more manageable chunks using feature modules or lazy-loaded modules. + - Use dynamic imports and lazy loading to improve initial load times and reduce bundle size. + +- **Common Patterns and Anti-patterns:** + - **Design Patterns:** + - **Dependency Injection:** Use NestJS's built-in dependency injection container to manage dependencies and promote loose coupling. + - **Repository Pattern:** Abstract data access logic into repositories to decouple services from specific database implementations. + - **Unit of Work:** Use a Unit of Work pattern for managing transactions across multiple repositories. + - **CQRS (Command Query Responsibility Segregation):** For complex applications, consider using CQRS to separate read and write operations, improving performance and scalability. + - **Recommended Approaches:** + - Use DTOs (Data Transfer Objects) for data validation and transformation between layers. + - Implement global exception filters to handle errors consistently across the application. + - Use interceptors for logging, caching, and other cross-cutting concerns. + - Utilize pipes for request validation and data transformation. + - Use asynchronous operations (`async/await`) for non-blocking I/O operations. + - **Anti-patterns:** + - **Tight Coupling:** Avoid creating tightly coupled components that are difficult to test and maintain. Use dependency injection and interfaces to promote loose coupling. + - **God Classes:** Avoid creating classes with too many responsibilities. Break down large classes into smaller, more manageable components. + - **Ignoring Errors:** Always handle errors properly using try-catch blocks, exception filters, and logging. Never ignore errors or swallow exceptions. + - **Hardcoding Configuration:** Avoid hardcoding configuration values directly in the code. Use environment variables or configuration files to manage settings. + - **State Management:** + - For simple applications, use services to manage application state. + - For more complex applications, consider using a state management library like Redux or NgRx (although this is less common on the backend). + - Avoid storing sensitive data in the client-side state. Store it securely on the server. + - **Error Handling:** + - Implement a global exception filter to catch unhandled exceptions and return appropriate error responses to the client. + - Use custom exceptions to represent specific error conditions in the application. + - Log errors with sufficient detail to facilitate debugging. + - Return consistent error responses with appropriate HTTP status codes. + +- **Performance Considerations:** + - **Optimization Techniques:** + - Use caching to reduce database load and improve response times. NestJS provides built-in support for caching using interceptors. + - Optimize database queries by using indexes, avoiding N+1 queries, and using efficient data retrieval methods. + - Use connection pooling to reduce the overhead of establishing database connections. + - Profile the application to identify performance bottlenecks and optimize accordingly. + - **Memory Management:** + - Avoid memory leaks by properly managing resources and releasing unused objects. + - Use streams for handling large files or data streams. + - Use object pooling to reuse frequently created objects. + - **Rendering Optimization (Server-Side Rendering):** + - Not directly applicable to NestJS, as it's primarily a backend framework. However, if using SSR, optimize rendering performance by caching rendered pages and using efficient templating engines. + - **Bundle Size Optimization:** + - Use tree shaking to remove unused code from the bundle. + - Minify and compress code to reduce bundle size. + - Use code splitting to load only the necessary code for each route or module. + - **Lazy Loading:** + - Use lazy loading to load modules or features on demand, improving initial load times. + - Implement code splitting to create smaller bundles that can be loaded independently. + +- **Security Best Practices:** + - **Common Vulnerabilities:** + - **SQL Injection:** Prevent SQL injection by using parameterized queries or an ORM that automatically escapes user inputs. + - **Cross-Site Scripting (XSS):** Protect against XSS by sanitizing user inputs and encoding outputs. + - **Cross-Site Request Forgery (CSRF):** Implement CSRF protection using tokens or other mechanisms. + - **Authentication and Authorization Flaws:** Secure authentication and authorization by using strong passwords, multi-factor authentication, and role-based access control. + - **Insecure Direct Object References (IDOR):** Prevent IDOR by validating user access to resources before granting access. + - **Input Validation:** + - Validate all user inputs to prevent malicious data from entering the system. Use DTOs and validation pipes to enforce input constraints. + - Sanitize user inputs to remove or escape potentially harmful characters. + - Validate file uploads to prevent malicious files from being uploaded. + - **Authentication and Authorization:** + - Use JWT (JSON Web Tokens) for authentication and authorization. + - Implement role-based access control (RBAC) to restrict access to resources based on user roles. + - Use secure password hashing algorithms (e.g., bcrypt) to store passwords securely. + - Implement rate limiting to prevent brute-force attacks. + - **Data Protection:** + - Encrypt sensitive data at rest and in transit. + - Use HTTPS to encrypt communication between the client and server. + - Store secrets securely using environment variables or a secrets management system. + - **Secure API Communication:** + - Use API keys or OAuth 2.0 for API authentication and authorization. + - Implement request validation and rate limiting to protect APIs from abuse. + - Use a secure API gateway to manage API traffic and enforce security policies. + +- **Testing Approaches:** + - **Unit Testing:** + - Write unit tests for individual components (services, controllers, etc.) to verify their functionality in isolation. + - Use mocking and stubbing to isolate components from their dependencies. + - Follow the Arrange-Act-Assert pattern for writing clear and concise unit tests. + - **Integration Testing:** + - Write integration tests to verify the interaction between multiple components or modules. + - Test the integration between the application and external systems (e.g., databases, APIs). + - **End-to-End Testing:** + - Write end-to-end tests to verify the application's functionality from a user's perspective. + - Use tools like Puppeteer or Cypress to automate end-to-end tests. + - **Test Organization:** + - Organize tests into separate directories that mirror the application's directory structure. + - Use descriptive names for test files and test cases. + - **Mocking and Stubbing:** + - Use mocking frameworks (e.g., Jest, Sinon.js) to create mock objects and stub methods. + - Use dependency injection to make components easily testable. + +- **Common Pitfalls and Gotchas:** + - **Frequent Mistakes:** + - **Not using DTOs for validation:** Always use DTOs and validation pipes to ensure data integrity. + - **Ignoring environment variables:** Use environment variables for configuration to avoid hardcoding values. + - **Not handling exceptions properly:** Implement global exception filters to catch unhandled exceptions and return appropriate error responses. + - **Overlooking security vulnerabilities:** Be aware of common security vulnerabilities and take steps to mitigate them. + - **Edge Cases:** + - **Handling large file uploads:** Use streams and appropriate buffering techniques to handle large file uploads efficiently. + - **Dealing with concurrent requests:** Use appropriate locking mechanisms or transaction management to handle concurrent requests safely. + - **Version-Specific Issues:** + - Be aware of breaking changes between NestJS versions and update code accordingly. + - Consult the NestJS documentation for migration guides and compatibility information. + - **Compatibility Concerns:** + - Ensure compatibility with different Node.js versions and operating systems. + - Test the application on different browsers and devices to ensure cross-platform compatibility. + - **Debugging Strategies:** + - Use the NestJS debugger to step through code and inspect variables. + - Use logging to track the flow of execution and identify errors. + - Use profiling tools to identify performance bottlenecks. + +- **Tooling and Environment:** + - **Recommended Development Tools:** + - **IDE:** Visual Studio Code with NestJS extensions. + - **CLI:** Nest CLI for generating and managing NestJS projects. + - **Database:** PostgreSQL, MySQL, MongoDB, or other compatible database. + - **Testing:** Jest, Supertest. + - **Build Configuration:** + - Use `tsconfig.json` to configure the TypeScript compiler. + - Use Webpack or Parcel to bundle and optimize the application. + - **Linting and Formatting:** + - Use ESLint and Prettier to enforce code style and formatting rules. + - Configure pre-commit hooks to automatically lint and format code before committing. + - **Deployment:** + - Deploy to platforms like Heroku, AWS, Google Cloud, or Azure. + - Use Docker to containerize the application and simplify deployment. + - **CI/CD Integration:** + - Integrate with CI/CD pipelines (e.g., Jenkins, Travis CI, GitHub Actions) to automate testing and deployment. + - Use environment-specific configuration files for different environments (development, staging, production). \ No newline at end of file diff --git a/.cursor/rules/netlify.mdc b/.cursor/rules/netlify.mdc new file mode 100644 index 000000000..91dc783d3 --- /dev/null +++ b/.cursor/rules/netlify.mdc @@ -0,0 +1,293 @@ +--- +description: This rule file outlines best practices for Netlify development, covering code structure, performance, security, testing, and deployment. It aims to provide a comprehensive guide for building robust and scalable applications on Netlify. +globs: *.js,*.jsx,*.ts,*.tsx,*.html,*.css,*.scss,*.md +--- +# Netlify Library Best Practices and Coding Standards + +This document outlines best practices for developing applications using Netlify, encompassing code organization, performance considerations, security measures, testing strategies, and deployment procedures. Adhering to these guidelines will ensure the creation of robust, scalable, and maintainable applications on the Netlify platform. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure + +A well-defined directory structure is crucial for maintainability and collaboration. Here's a recommended structure: + + +project-root/ +├── .netlify/ # Netlify-specific configuration (auto-generated) +├── functions/ # Serverless functions +│ └── api/ # API endpoint functions +│ └── hello.js +├── src/ # Source code +│ ├── components/ # Reusable UI components +│ │ ├── Button.jsx +│ │ └── Header.jsx +│ ├── pages/ # Pages for different routes +│ │ ├── index.jsx +│ │ └── about.jsx +│ ├── styles/ # CSS, SCSS, or other styling +│ │ ├── global.css +│ │ └── components/ +│ ├── utils/ # Utility functions +│ │ └── api.js +│ ├── App.jsx # Main application component +│ ├── index.jsx # Entry point for React +│ └── routes.js # Routing configuration +├── static/ # Static assets (images, fonts, etc.) +│ ├── img/ +│ └── fonts/ +├── public/ # Public files to be deployed +│ └── index.html +├── tests/ # Tests +│ ├── unit/ +│ └── integration/ +├── netlify.toml # Netlify configuration file +├── package.json # Node.js dependencies +└── README.md + + +### 1.2 File Naming Conventions + +* **Components:** Use PascalCase for component file names (e.g., `MyComponent.jsx`). +* **Styles:** Use kebab-case for style file names (e.g., `my-component.css`). +* **Functions:** Use camelCase or kebab-case for function file names (e.g., `helloWorld.js` or `hello-world.js`). +* **Images:** Use descriptive names (e.g., `product-image.jpg`). + +### 1.3 Module Organization + +* **Group Related Code:** Place related functions, components, and styles into separate modules/directories. +* **Single Responsibility Principle:** Each module should have a clear and single purpose. +* **Avoid Circular Dependencies:** Carefully manage dependencies between modules to prevent circular dependencies. +* **Use ES Modules:** Utilize ES modules (`import/export`) for better code organization and tree shaking. + +### 1.4 Component Architecture + +* **Component-Based Approach:** Build your UI using reusable components. +* **Presentational and Container Components:** Separate concerns by using presentational components (UI only) and container components (data fetching and logic). +* **Atomic Design:** Consider using atomic design principles (atoms, molecules, organisms, templates, pages) for a scalable component architecture. + +### 1.5 Code Splitting + +* **Dynamic Imports:** Use dynamic imports (`import()`) to load modules on demand. +* **Route-Based Splitting:** Split your application based on routes so that only the necessary code is loaded for a given route. This will improve the initial loading time for the end user. +* **Component-Based Splitting:** Use React.lazy or similar mechanisms to load components only when they are needed. +* **Webpack or Parcel Bundler Configuration:** Configure your bundler (Webpack, Parcel, etc.) for optimal code splitting. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns + +* **JAMstack Architecture:** Embrace the JAMstack architecture (JavaScript, APIs, Markup) for faster and more secure websites. +* **Serverless Functions:** Utilize Netlify Functions for backend logic (API endpoints, background tasks). Follow the function handler structure and consider using middleware for common tasks like authentication. +* **Git-Based Workflow:** Use Git for version control and continuous deployment with Netlify. +* **Environment Variables:** Store sensitive information (API keys, database credentials) in environment variables, and access them from your serverless functions. Ensure you configure these in Netlify's UI and use `process.env.VARIABLE_NAME`. + +### 2.2 Recommended Approaches + +* **Form Handling:** Use libraries like Formik or React Hook Form for form handling. +* **State Management:** Choose a state management solution appropriate for the complexity of your application (e.g., Context API, Redux, Zustand). +* **API Calls:** Use `fetch` or libraries like Axios for making API calls. Implement error handling and loading states. +* **Image Optimization:** Optimize images before deployment to improve performance. Consider using Netlify Large Media for automatic image optimization. + +### 2.3 Anti-patterns + +* **Committing Secrets:** Avoid committing API keys or other secrets to your Git repository. +* **Over-reliance on Client-Side Rendering:** Minimize client-side rendering for better SEO and initial load time. Use static site generation (SSG) or server-side rendering (SSR) where appropriate. +* **Ignoring Performance Budgets:** Establish performance budgets and monitor your application's performance regularly. Use tools such as Lighthouse, WebPageTest, and Chrome DevTools. +* **Unnecessary Dependencies:** Avoid including unnecessary dependencies in your project. +* **Direct DOM Manipulation:** Avoid direct DOM manipulation in React components; let React handle the updates. + +### 2.4 State Management + +* **Context API:** Use React's Context API for simple state management. +* **Redux:** Use Redux for more complex state management requirements. +* **Zustand:** Consider Zustand for a simple and unopinionated state management solution. +* **Avoid Global State:** Minimize the use of global state to avoid performance issues and make code easier to reason about. +* **Immutable Updates:** Use immutable updates when updating state to improve performance and prevent unexpected side effects. + +### 2.5 Error Handling + +* **Try-Catch Blocks:** Use try-catch blocks in your serverless functions to handle errors gracefully. +* **Centralized Error Handling:** Implement a centralized error handling mechanism to log errors and display user-friendly messages. +* **ErrorBoundary:** Use React's `ErrorBoundary` component to catch errors in your UI. +* **Logging:** Log errors to a service like Sentry, Rollbar, or Netlify's built-in logging for monitoring and debugging. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +* **Code Splitting:** Implement code splitting to reduce initial load time. +* **Lazy Loading:** Lazy load images and other assets to improve performance. +* **Image Optimization:** Optimize images using tools like ImageOptim, TinyPNG, or Netlify Large Media. +* **Caching:** Utilize browser caching and CDN caching to reduce server load and improve response times. Netlify automatically handles CDN caching. +* **Minification:** Minify your CSS and JavaScript files to reduce their size. +* **Compression:** Use Gzip or Brotli compression to reduce the size of your files. + +### 3.2 Memory Management + +* **Avoid Memory Leaks:** Be careful to avoid memory leaks in your JavaScript code. Properly clean up event listeners and timers. +* **Use Efficient Data Structures:** Use appropriate data structures for your data to optimize memory usage. +* **Limit Large Data Sets:** Avoid loading large datasets into memory if possible. + +### 3.3 Rendering Optimization (if applicable) + +* **Virtual DOM:** React's Virtual DOM provides significant rendering optimization. Ensure you are using React efficiently, minimizing unnecessary re-renders. +* **Memoization:** Use `React.memo` to prevent re-rendering components unless their props change. +* **Pure Components:** Use `PureComponent` or implement `shouldComponentUpdate` for class components to prevent unnecessary re-renders. + +### 3.4 Bundle Size Optimization + +* **Analyze Bundle Size:** Use tools like Webpack Bundle Analyzer or Parcel Size Analyzer to identify large dependencies. +* **Tree Shaking:** Enable tree shaking in your bundler to remove unused code. +* **Code Minimization:** Utilize code minification in your build process. +* **Remove Unused Dependencies:** Review your dependencies and remove any unused ones. + +### 3.5 Lazy Loading + +* **Intersection Observer:** Use the Intersection Observer API to lazy load images and other assets when they come into view. +* **React Lazy:** Use React.lazy for component-level lazy loading. +* **Dynamic Imports:** Use dynamic imports for modules that are not needed immediately. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities + +* **Cross-Site Scripting (XSS):** Prevent XSS attacks by sanitizing user inputs and escaping output. +* **Cross-Site Request Forgery (CSRF):** Protect against CSRF attacks by using CSRF tokens. +* **Injection Attacks:** Prevent SQL injection and other injection attacks by using parameterized queries and validating user inputs. +* **Authentication and Authorization:** Implement secure authentication and authorization mechanisms to protect sensitive data. +* **Exposed API Keys:** Avoid exposing API keys in client-side code. Store them in environment variables and access them from your serverless functions. + +### 4.2 Input Validation + +* **Client-Side Validation:** Use client-side validation to provide immediate feedback to users. +* **Server-Side Validation:** Always perform server-side validation to ensure data integrity and prevent malicious inputs. +* **Sanitization:** Sanitize user inputs to remove potentially harmful characters. +* **Escaping:** Escape output to prevent XSS attacks. + +### 4.3 Authentication and Authorization + +* **Netlify Identity:** Use Netlify Identity for simple authentication and authorization. +* **Third-Party Authentication:** Use third-party authentication providers like Auth0, Firebase Authentication, or AWS Cognito. +* **JSON Web Tokens (JWT):** Use JWTs to securely transmit user information between the client and server. +* **Role-Based Access Control (RBAC):** Implement RBAC to control access to resources based on user roles. + +### 4.4 Data Protection + +* **Encryption:** Encrypt sensitive data at rest and in transit. +* **Secure Storage:** Store sensitive data in secure storage locations. +* **Regular Backups:** Perform regular backups of your data. +* **Data Minimization:** Only collect and store the data that is necessary. +* **Compliance:** Ensure your application complies with relevant data protection regulations (e.g., GDPR, CCPA). + +### 4.5 Secure API Communication + +* **HTTPS:** Use HTTPS for all API communication. +* **API Keys:** Use API keys to authenticate requests to your API. +* **Rate Limiting:** Implement rate limiting to prevent abuse of your API. +* **CORS:** Configure CORS (Cross-Origin Resource Sharing) to allow only authorized domains to access your API. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +* **Test Individual Components:** Unit tests should focus on testing individual components in isolation. +* **Mock Dependencies:** Mock dependencies to isolate the component being tested. +* **Test All Code Paths:** Test all code paths, including error handling. +* **Use a Testing Framework:** Use a testing framework like Jest, Mocha, or Jasmine. + +### 5.2 Integration Testing + +* **Test Interactions:** Integration tests should focus on testing the interactions between different components and modules. +* **Test API Calls:** Test the integration with your API endpoints. +* **Use a Testing Framework:** Use a testing framework like Cypress, Puppeteer, or Selenium. + +### 5.3 End-to-End Testing + +* **Test User Flows:** End-to-end tests should focus on testing complete user flows. +* **Simulate User Interactions:** Simulate user interactions, such as clicking buttons and filling out forms. +* **Use a Testing Framework:** Use a testing framework like Cypress, Puppeteer, or Selenium. + +### 5.4 Test Organization + +* **Separate Test Files:** Create separate test files for each component or module. +* **Organize Tests by Type:** Organize tests by type (unit, integration, end-to-end). +* **Use Descriptive Test Names:** Use descriptive test names to make it easier to understand what each test is testing. + +### 5.5 Mocking and Stubbing + +* **Mock API Calls:** Mock API calls to avoid making real API calls during testing. +* **Stub Dependencies:** Stub dependencies to isolate the component being tested. +* **Use a Mocking Library:** Use a mocking library like Jest's `jest.mock` or Sinon.js. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +* **Not Using Environment Variables:** Failing to use environment variables for sensitive information. +* **Over-complicating Serverless Functions:** Creating serverless functions that are too large and complex. +* **Not Handling Errors:** Failing to handle errors gracefully in serverless functions. +* **Ignoring Performance:** Ignoring performance considerations and creating slow-loading applications. +* **Not Testing:** Failing to test your application thoroughly. + +### 6.2 Edge Cases + +* **Long Build Times:** Long build times can be a problem for large applications. Optimize your build process to reduce build times. +* **Function Cold Starts:** Serverless functions can experience cold starts, which can increase response times. Use techniques like keep-alive to reduce cold start times. +* **CDN Invalidation:** Ensure your CDN invalidates cache correctly when you deploy new versions of your application. + +### 6.3 Version-Specific Issues + +* **Netlify CLI Updates:** Stay up-to-date with the latest Netlify CLI to avoid compatibility issues. +* **Node.js Versions:** Ensure your Node.js version is compatible with Netlify Functions. +* **Dependency Updates:** Regularly update your dependencies to address security vulnerabilities and bug fixes. + +### 6.4 Compatibility Concerns + +* **Browser Compatibility:** Test your application in different browsers to ensure compatibility. +* **Device Compatibility:** Test your application on different devices (desktop, mobile, tablet) to ensure compatibility. +* **Accessibility:** Ensure your application is accessible to users with disabilities by following accessibility guidelines (WCAG). + +### 6.5 Debugging Strategies + +* **Netlify Logs:** Use Netlify's built-in logging to debug issues. +* **Console Logging:** Use `console.log` statements to debug your code. +* **Debugging Tools:** Use browser debugging tools like Chrome DevTools or Firefox Developer Tools. +* **Remote Debugging:** Use remote debugging tools to debug serverless functions. + +## 7. Tooling and Environment + +### 7.1 Recommended Tools + +* **VS Code:** VS Code with extensions like ESLint, Prettier, and Netlify. +* **Netlify CLI:** Netlify Command Line Interface for local development and deployment. +* **Git:** Git for version control. +* **npm or Yarn:** npm or Yarn for package management. +* **Webpack, Parcel, or Rollup:** Webpack, Parcel, or Rollup for bundling your code. + +### 7.2 Build Configuration + +* **Netlify TOML:** Use the `netlify.toml` file to configure your build process. Set environment variables, build commands, and deploy contexts. +* **Build Commands:** Define build commands to transpile your code, optimize assets, and generate static files. +* **Deploy Contexts:** Use deploy contexts to create different environments (e.g., production, staging, development). + +### 7.3 Linting and Formatting + +* **ESLint:** Use ESLint to enforce code style and prevent errors. +* **Prettier:** Use Prettier to automatically format your code. +* **Husky and lint-staged:** Use Husky and lint-staged to run linters and formatters before committing code. + +### 7.4 Deployment + +* **Git-Based Deployment:** Deploy your application by pushing your code to a Git repository connected to Netlify. +* **Netlify CLI Deployment:** Deploy your application using the Netlify CLI. +* **Atomic Deploys:** Netlify provides atomic deploys, ensuring that your site is always in a consistent state during updates. +* **Rollbacks:** Easily roll back to previous deployments if necessary. + +### 7.5 CI/CD Integration + +* **GitHub Actions:** Integrate with GitHub Actions for automated testing and deployment. +* **CircleCI:** Integrate with CircleCI for continuous integration and continuous deployment. +* **Travis CI:** Integrate with Travis CI for continuous integration and continuous deployment. + +By following these best practices, you can create robust, scalable, secure, and maintainable applications using Netlify. \ No newline at end of file diff --git a/.cursor/rules/next-js.mdc b/.cursor/rules/next-js.mdc new file mode 100644 index 000000000..fc85b6558 --- /dev/null +++ b/.cursor/rules/next-js.mdc @@ -0,0 +1,288 @@ +--- +description: This rule provides comprehensive guidance for Next.js development, covering code organization, performance, security, testing, and common pitfalls. It helps developers build robust, scalable, and maintainable Next.js applications by adhering to community-accepted best practices and coding standards. +globs: *.js,*.jsx,*.ts,*.tsx +--- +# Next.js Best Practices + +This document outlines best practices for developing Next.js applications, focusing on code organization, performance optimization, security, testing strategies, and common pitfalls to avoid. Adhering to these guidelines will help you build robust, scalable, and maintainable applications. + +## 1. Code Organization and Structure + +### Directory Structure + +* **`app/`**: (Recommended - Next.js 13+) Contains route handlers, server components, and client components. + * `page.tsx`: Represents the UI for a route. + * `layout.tsx`: Defines the layout for a route and its children. + * `loading.tsx`: Displays a loading UI while a route segment is loading. + * `error.tsx`: Handles errors within a route segment. + * `head.tsx`: Manages the `` metadata for a route. + * `route.ts`: Defines server-side route handlers (API routes). + * `[dynamic-segment]`: Dynamic route segments, using brackets. + * `@folder-name`: Route Groups to organize routes without affecting URL structure. +* **`pages/`**: (Legacy - Before Next.js 13) Contains page components. + * `api/`: Serverless functions (API routes). + * `_app.js/tsx`: Custom App component (wraps all pages). + * `_document.js/tsx`: Custom Document component (control the entire HTML document). +* **`components/`**: Reusable UI components. +* **`lib/`**: Utility functions, helper functions, and third-party integrations. +* **`hooks/`**: Custom React hooks. +* **`styles/`**: Global styles and CSS modules. +* **`public/`**: Static assets (images, fonts, etc.). +* **`types/`**: TypeScript type definitions and interfaces. +* **`utils/`**: Contains utilities and helper functions, along with any API-related logic. + +**Recommendation:** Prefer the `app/` directory structure for new projects as it aligns with the latest Next.js features and best practices. When using `pages/`, keep it simple and migrate to `app/` when feasible. + +### File Naming Conventions + +* **Components:** `ComponentName.jsx` or `ComponentName.tsx` +* **Pages:** `page.js`, `page.jsx`, `page.ts`, `page.tsx` (within the `app` or `pages` directory) +* **Layouts:** `layout.js`, `layout.jsx`, `layout.ts`, `layout.tsx` (within the `app` directory) +* **API Routes:** `route.js`, `route.ts` (within the `app/api` directory or `pages/api` directory) +* **Hooks:** `useHookName.js` or `useHookName.ts` +* **Styles:** `ComponentName.module.css` or `ComponentName.module.scss` +* **Types:** `types.ts` or `interfaces.ts` + +### Module Organization + +* **Co-location:** Keep related components, styles, and tests in the same directory. +* **Feature-based modules:** Group files by feature rather than type (e.g., `components/user-profile/`, not `components/button`, `components/form`). +* **Avoid deeply nested directories:** Keep the directory structure relatively flat to improve navigation. + +### Component Architecture + +* **Presentational vs. Container Components:** Separate components that handle data fetching and state management (container components) from those that only render UI (presentational components). +* **Atomic Design:** Organize components into atoms, molecules, organisms, templates, and pages for better reusability and maintainability. +* **Composition over inheritance:** Favor composition to create flexible and reusable components. +* **Server Components (app directory):** Use server components by default for improved performance. Only use client components when interactivity (event handlers, useState, useEffect) is required. + +### Code Splitting + +* **Dynamic imports:** Use `next/dynamic` to load components only when they are needed, improving initial load time. Example: `dynamic(() => import('../components/MyComponent'))`. +* **Route-level code splitting:** Next.js automatically splits code based on routes, so each page only loads the necessary JavaScript. +* **Granular code splitting:** Break down large components into smaller chunks that can be loaded independently. + +## 2. Common Patterns and Anti-patterns + +### Design Patterns + +* **Higher-Order Components (HOCs):** Reusable component logic. +* **Render Props:** Sharing code between React components using a prop whose value is a function. +* **Hooks:** Extracting stateful logic into reusable functions. +* **Context API:** Managing global state. +* **Compound Components:** Combining multiple components that work together implicitly. + +### Recommended Approaches + +* **Data fetching:** Use `getServerSideProps` or `getStaticProps` or server components for fetching data on the server-side. Use `SWR` or `React Query` for client-side data fetching and caching. +* **Styling:** Use CSS Modules, Styled Components, or Tailwind CSS for component-level styling. Prefer Tailwind CSS for rapid development. +* **State Management:** Use React Context, Zustand, Jotai, or Recoil for managing global state. Redux is an option, but often overkill for smaller Next.js projects. +* **Form Handling:** Use `react-hook-form` for managing forms and validation. +* **API Routes:** Use Next.js API routes for serverless functions. + +### Anti-patterns and Code Smells + +* **Over-fetching data:** Only fetch the data that is needed by the component. +* **Blocking the main thread:** Avoid long-running synchronous operations in the main thread. +* **Mutating state directly:** Always use `setState` or hooks to update state. +* **Not memoizing components:** Use `React.memo` to prevent unnecessary re-renders. +* **Using `useEffect` without a dependency array:** Ensure the dependency array is complete to prevent unexpected behavior. +* **Writing server side code in client components:** Can expose secrets or cause unexpected behavior. + +### State Management + +* **Local State:** Use `useState` for component-specific state. +* **Context API:** Use `useContext` for application-wide state that doesn't change often. +* **Third-party libraries:** Use `Zustand`, `Jotai`, or `Recoil` for more complex state management needs. These are simpler and more performant alternatives to Redux for many Next.js use cases. + +### Error Handling + +* **`try...catch`:** Use `try...catch` blocks for handling errors in asynchronous operations. +* **Error Boundary Components:** Create reusable error boundary components to catch errors in child components. Implement `getDerivedStateFromError` or `componentDidCatch` lifecycle methods. +* **Centralized error logging:** Log errors to a central service like Sentry or Bugsnag. +* **Custom Error Pages:** Use `_error.js` or `_error.tsx` to create custom error pages. +* **Route-level error handling (app directory):** Use `error.tsx` within route segments to handle errors specific to that route. + +## 3. Performance Considerations + +### Optimization Techniques + +* **Image optimization:** Use `next/image` component for automatic image optimization, including lazy loading and responsive images. +* **Font optimization:** Use `next/font` to optimize font loading and prevent layout shift. +* **Code splitting:** Use dynamic imports and route-level code splitting to reduce initial load time. +* **Caching:** Use caching strategies (e.g., `Cache-Control` headers, `SWR`, `React Query`) to reduce data fetching overhead. +* **Memoization:** Use `React.memo` to prevent unnecessary re-renders of components. +* **Prefetching:** Use the `` tag to prefetch pages that are likely to be visited. +* **SSR/SSG:** Use Static Site Generation (SSG) for content that doesn't change often and Server-Side Rendering (SSR) for dynamic content. +* **Incremental Static Regeneration (ISR):** Use ISR to update statically generated pages on a regular interval. + +### Memory Management + +* **Avoid memory leaks:** Clean up event listeners and timers in `useEffect` hooks. +* **Minimize re-renders:** Only update state when necessary to reduce the number of re-renders. +* **Use immutable data structures:** Avoid mutating data directly to prevent unexpected side effects. + +### Rendering Optimization + +* **Server Components (app directory):** Render as much as possible on the server to reduce client-side JavaScript. +* **Client Components (app directory):** Only use client components when interactivity is required. Defer rendering of non-critical client components using `React.lazy`. + +### Bundle Size Optimization + +* **Analyze bundle size:** Use tools like `webpack-bundle-analyzer` to identify large dependencies. +* **Remove unused code:** Use tree shaking to remove unused code from your bundles. +* **Use smaller dependencies:** Replace large dependencies with smaller, more lightweight alternatives. +* **Compression:** Enable Gzip or Brotli compression on your server to reduce the size of the transferred files. + +### Lazy Loading + +* **Images:** Use `next/image` for automatic lazy loading of images. +* **Components:** Use `next/dynamic` for lazy loading of components. +* **Intersection Observer:** Use the Intersection Observer API for manual lazy loading of content. + +## 4. Security Best Practices + +### Common Vulnerabilities + +* **Cross-Site Scripting (XSS):** Sanitize user input to prevent XSS attacks. Be especially careful when rendering HTML directly from user input. +* **Cross-Site Request Forgery (CSRF):** Use CSRF tokens to protect against CSRF attacks. +* **SQL Injection:** Use parameterized queries or an ORM to prevent SQL injection attacks. +* **Authentication and Authorization vulnerabilities:** Implement secure authentication and authorization mechanisms. Avoid storing secrets in client-side code. +* **Exposing sensitive data:** Protect API keys and other sensitive data by storing them in environment variables and accessing them on the server-side. + +### Input Validation + +* **Server-side validation:** Always validate user input on the server-side. +* **Client-side validation:** Use client-side validation for immediate feedback, but don't rely on it for security. +* **Sanitize input:** Sanitize user input to remove potentially malicious code. +* **Use a validation library:** Use a library like `zod` or `yup` for validating user input. + +### Authentication and Authorization + +* **Use a secure authentication provider:** Use a service like Auth0, NextAuth.js, or Firebase Authentication for secure authentication. +* **Store tokens securely:** Store tokens in HTTP-only cookies or local storage. +* **Implement role-based access control:** Use role-based access control to restrict access to sensitive resources. +* **Protect API endpoints:** Use authentication middleware to protect API endpoints. + +### Data Protection + +* **Encrypt sensitive data:** Encrypt sensitive data at rest and in transit. +* **Use HTTPS:** Use HTTPS to encrypt communication between the client and the server. +* **Regularly update dependencies:** Keep your dependencies up to date to patch security vulnerabilities. +* **Secure environment variables:** Never commit environment variables to your repository. Use a secrets management tool if necessary. + +### Secure API Communication + +* **Use HTTPS:** Use HTTPS for all API communication. +* **Authenticate API requests:** Use API keys or tokens to authenticate API requests. +* **Rate limiting:** Implement rate limiting to prevent abuse of your API. +* **Input validation:** Validate all API request parameters. +* **Output encoding:** Properly encode API responses to prevent injection attacks. + +## 5. Testing Approaches + +### Unit Testing + +* **Test individual components:** Write unit tests for individual components to ensure they are working correctly. +* **Use a testing framework:** Use a testing framework like Jest or Mocha. +* **Mock dependencies:** Mock external dependencies to isolate components during testing. +* **Test edge cases:** Test edge cases and error conditions to ensure the component is robust. +* **Use React Testing Library:** Prefer React Testing Library for component testing as it encourages testing from a user perspective, promoting better accessibility and more robust tests. + +### Integration Testing + +* **Test interactions between components:** Write integration tests to ensure that components are working together correctly. +* **Test API calls:** Test API calls to ensure that data is being fetched and saved correctly. +* **Use a testing framework:** Use a testing framework like Jest or Mocha with libraries like `msw` (Mock Service Worker) to intercept and mock API calls. + +### End-to-End Testing + +* **Test the entire application:** Write end-to-end tests to ensure that the entire application is working correctly. +* **Use a testing framework:** Use a testing framework like Cypress or Playwright. +* **Test user flows:** Test common user flows to ensure that the application is providing a good user experience. +* **Focus on critical paths:** Prioritize end-to-end tests for critical user flows to ensure application stability. + +### Test Organization + +* **Co-locate tests with components:** Keep tests in the same directory as the components they are testing. +* **Use a consistent naming convention:** Use a consistent naming convention for test files (e.g., `ComponentName.test.js`). +* **Organize tests by feature:** Organize tests by feature to improve maintainability. + +### Mocking and Stubbing + +* **Mock external dependencies:** Mock external dependencies to isolate components during testing. +* **Stub API calls:** Stub API calls to prevent network requests during testing. +* **Use a mocking library:** Use a mocking library like Jest's built-in mocking capabilities or `msw`. + +## 6. Common Pitfalls and Gotchas + +### Frequent Mistakes + +* **Not understanding server-side rendering:** Failing to utilize SSR effectively can impact SEO and initial load performance. +* **Over-complicating state management:** Using Redux for simple state management needs can add unnecessary complexity. +* **Not optimizing images:** Not using `next/image` can result in large image sizes and slow loading times. +* **Ignoring security best practices:** Neglecting security can lead to vulnerabilities. +* **Not testing the application thoroughly:** Insufficient testing can result in bugs and regressions. +* **Accidentally exposing API keys or secrets in client-side code.** + +### Edge Cases + +* **Handling errors gracefully:** Implement proper error handling to prevent the application from crashing. +* **Dealing with different screen sizes:** Ensure the application is responsive and works well on different screen sizes. +* **Supporting different browsers:** Test the application in different browsers to ensure compatibility. +* **Managing complex data structures:** Use appropriate data structures and algorithms to efficiently manage complex data. + +### Version-Specific Issues + +* **Breaking changes:** Be aware of breaking changes when upgrading Next.js versions. +* **Deprecated features:** Avoid using deprecated features. +* **Compatibility with third-party libraries:** Ensure that third-party libraries are compatible with the Next.js version being used. + +### Compatibility Concerns + +* **Browser compatibility:** Ensure that the application is compatible with the target browsers. +* **Third-party library compatibility:** Ensure that third-party libraries are compatible with Next.js. + +### Debugging Strategies + +* **Use the browser developer tools:** Use the browser developer tools to inspect the DOM, debug JavaScript, and analyze network requests. +* **Use console.log statements:** Use `console.log` statements to debug code. +* **Use a debugger:** Use a debugger to step through code and inspect variables. +* **Use error logging:** Log errors to a central service to track and analyze issues. + +## 7. Tooling and Environment + +### Recommended Development Tools + +* **VS Code:** Code editor with excellent support for JavaScript, TypeScript, and React. +* **ESLint:** Linter for identifying and fixing code style issues. +* **Prettier:** Code formatter for automatically formatting code. +* **Chrome Developer Tools:** Browser developer tools for debugging and profiling. +* **React Developer Tools:** Browser extension for inspecting React components. +* **Webpack Bundle Analyzer:** Tool for analyzing the size of the Webpack bundle. + +### Build Configuration + +* **Use environment variables:** Store configuration values in environment variables. +* **Use a build script:** Use a build script to automate the build process. +* **Optimize build settings:** Optimize build settings for production (e.g., enable minification, tree shaking). + +### Linting and Formatting + +* **Use ESLint with recommended rules:** Configure ESLint with a set of recommended rules for JavaScript and React. +* **Use Prettier for automatic formatting:** Configure Prettier to automatically format code on save. +* **Integrate linting and formatting into the build process:** Integrate linting and formatting into the build process to ensure that code is always consistent. +* **Use a shared configuration:** Ensure that all developers are using the same linting and formatting configuration. + +### Deployment + +* **Use Vercel for easy deployment:** Vercel is the recommended platform for deploying Next.js applications. +* **Use a CDN for static assets:** Use a CDN to serve static assets from a location that is geographically close to the user. +* **Configure caching:** Configure caching to improve performance and reduce server load. +* **Monitor application health:** Monitor application health to detect and resolve issues quickly. + +### CI/CD Integration + +* **Use a CI/CD pipeline:** Use a CI/CD pipeline to automate the build, test, and deployment process. +* **Run tests in the CI/CD pipeline:** Run tests in the CI/CD pipeline to ensure that code is working correctly before it is deployed. +* **Automate deployments:** Automate deployments to reduce the risk of human error. \ No newline at end of file diff --git a/.cursor/rules/nginx.mdc b/.cursor/rules/nginx.mdc new file mode 100644 index 000000000..6c786d1c3 --- /dev/null +++ b/.cursor/rules/nginx.mdc @@ -0,0 +1,406 @@ +--- +description: This rule provides a comprehensive guide to nginx configuration best practices, covering code organization, common patterns, performance, security, testing, pitfalls, and tooling. +globs: nginx.conf +--- +# nginx Configuration Best Practices + +This document outlines best practices for configuring nginx as a web server, reverse proxy, and load balancer. Adhering to these guidelines promotes maintainability, security, and performance. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure + +Maintain a well-organized directory structure to manage configuration files efficiently. + + +/etc/nginx/ +├── nginx.conf # Main nginx configuration file +├── conf.d/ # Site-specific configurations (symlinks preferred) +│ ├── default.conf # Default configuration +│ ├── example.com.conf# Site-specific configuration +│ └── ... +├── snippets/ # Reusable configuration snippets +│ ├── ssl-params.conf # SSL parameters +│ ├── proxy-params.conf # Reverse proxy parameters +│ └── ... +└── modules-enabled/ # Enabled modules (usually managed by package manager) + + +**Explanation:** + +* `/etc/nginx/nginx.conf`: The main configuration file that includes other configurations. +* `/etc/nginx/conf.d/`: Contains site-specific configurations. Symlinking from `/etc/nginx/sites-available/` to `/etc/nginx/sites-enabled/` is a common, more manageable pattern. +* `/etc/nginx/snippets/`: Stores reusable configuration snippets, promoting modularity and reducing duplication. + +### 1.2 File Naming Conventions + +Adopt consistent file naming conventions for clarity. + +* Site-specific configurations: `domain.com.conf` or `application-name.conf`. +* Snippets: `description.conf` (e.g., `ssl-params.conf`, `proxy-params.conf`). + +### 1.3 Modular Configuration + +Break down complex configurations into smaller, reusable modules (snippets). + +**Example: `snippets/ssl-params.conf`** + +nginx +ssl_protocols TLSv1.2 TLSv1.3; +ssl_prefer_server_ciphers on; +ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384; +ssl_ecdh_curve prime256v1; +ssl_session_timeout 10m; +ssl_session_cache shared:SSL:10m; +ssl_session_tickets off; + +add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always; +add_header X-Frame-Options SAMEORIGIN; +add_header X-Content-Type-Options nosniff; +add_header X-XSS-Protection "1; mode=block"; + +ssl_stapling on; +ssl_stapling_verify on; +resolver 8.8.8.8 8.8.4.4 valid=300s; +resolver_timeout 5s; + + +**Usage in site configuration:** + +nginx +server + listen 443 ssl; + server_name example.com; + + include snippets/ssl-params.conf; + + ssl_certificate /etc/nginx/ssl/example.com.crt; + ssl_certificate_key /etc/nginx/ssl/example.com.key; + + location / { + # ... + +} + + +### 1.4 Component Architecture + +Design your nginx configurations with a clear separation of concerns: + +* **Global Settings**: Core directives in `nginx.conf` affecting all virtual hosts. +* **Virtual Host Configuration**: Separate files for each site/application in `conf.d/`. +* **Reusable Blocks**: Snippets for common settings like SSL, reverse proxy configurations, or access control. + +### 1.5 Code Splitting + +For large and complex configurations, split files by functionality (e.g., routing rules, security policies, caching configurations). Use `include` directives to assemble the complete configuration. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns + +* **Reverse Proxy**: Forward requests to backend servers. Useful for load balancing, caching, and security. +* **Load Balancer**: Distribute traffic across multiple backend servers for high availability and scalability. Use `upstream` blocks. +* **Static Content Server**: Efficiently serve static files like images, CSS, and JavaScript. +* **Cache**: Reduce latency and server load by caching frequently accessed content. + +### 2.2 Recommended Approaches for Common Tasks + +* **Serving Static Content:** + + nginx + location /static/ { + alias /var/www/example.com/static/; + expires 30d; + access_log off; + } + + +* **Reverse Proxy to a Backend Server:** + + nginx + location / { + proxy_pass http://backend_server; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; + } + + upstream backend_server { + server backend1.example.com; + server backend2.example.com; + } + + +* **Implementing Basic Authentication:** + + nginx + location /protected/ { + auth_basic "Restricted Area"; + auth_basic_user_file /etc/nginx/.htpasswd; + } + + +### 2.3 Anti-patterns and Code Smells + +* **Large, Monolithic Configuration Files**: Difficult to manage and debug. Break them into smaller, modular files using `include`. +* **Duplicated Configuration Blocks**: Create snippets to reuse common configurations. +* **Using `if` Statements Excessively**: `if` statements can have performance implications. Prefer using `try_files` or other more efficient directives. +* **Insecure Configuration**: Exposing sensitive information, using weak ciphers, or not implementing proper access controls. +* **Ignoring Error Logs**: Neglecting to monitor and analyze error logs can lead to undetected issues. + +### 2.4 State Management + +Nginx itself is generally stateless. State management is typically handled by backend applications. However, Nginx Plus offers features like session persistence (sticky sessions) for load balancing. + +* **Session Persistence (Nginx Plus):** + + nginx + upstream backend { + server backend1.example.com; + server backend2.example.com; + sticky cookie srv_id expires=1h; + } + + +### 2.5 Error Handling + +* **Custom Error Pages**: Configure custom error pages for a better user experience. + + nginx + error_page 404 /404.html; + location = /404.html { + root /var/www/example.com/html; + internal; + } + + +* **Logging**: Use comprehensive logging to capture errors and warnings for troubleshooting. +* **Retry Logic**: Implement retry logic in your application or use Nginx's `proxy_next_upstream` directive to retry failed requests on different backend servers. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +* **Enable Gzip Compression**: Compress text-based responses to reduce bandwidth usage. + + nginx +gzip on; +gzip_disable "msie6"; +gzip_vary on; +gzip_proxied any; +gzip_comp_level 6; +gzip_buffers 16 8k; +gzip_http_version 1.1; +gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript; + + +* **Enable Caching**: Cache static content and dynamic responses to reduce server load and improve response times. Use `proxy_cache_path` and `proxy_cache` directives. + + nginx +proxy_cache_path /tmp/nginx_cache levels=1:2 keys_zone=my_cache:10m max_size=10g inactive=60m use_temp_path=off; + +server { + location / { + proxy_pass http://backend_server; + proxy_cache my_cache; + proxy_cache_valid 200 302 1h; + proxy_cache_valid 404 10m; + proxy_cache_use_stale error timeout updating invalid_header http_500 http_502 http_503 http_504; + add_header X-Cache-Status $upstream_cache_status; + } +} + + +* **Tune Worker Processes and Connections**: Adjust `worker_processes` and `worker_connections` based on your server's resources. + + nginx +worker_processes auto; +events { + worker_connections 1024; +} + + +* **HTTP/2 or HTTP/3**: Enable HTTP/2 or HTTP/3 for improved performance. +* **Keep-Alive Connections**: Enable keep-alive connections to reduce latency by reusing existing connections. + + nginx +keepalive_timeout 65; + + +### 3.2 Memory Management + +* **Optimize Buffer Sizes**: Tune buffer sizes (e.g., `proxy_buffer_size`, `proxy_buffers`) to match your application's needs and avoid memory fragmentation. +* **Monitor Memory Usage**: Use tools like `top` or `htop` to monitor nginx's memory usage and identify potential memory leaks. + +### 3.3 Rendering Optimization (If applicable) + +Nginx primarily serves content; rendering optimization is typically handled by the backend application. + +* **Content Delivery Network (CDN)**: Offload static content delivery to a CDN. + +### 3.4 Bundle Size Optimization (If applicable) + +This primarily applies to serving static assets. + +* **Minify and Compress**: Minify and compress CSS and JavaScript files before serving them. +* **Use Brotli**: Consider using Brotli compression, which offers better compression ratios than Gzip. + +### 3.5 Lazy Loading (If applicable) + +This is typically implemented at the application level. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and Prevention + +* **SQL Injection**: Prevent SQL injection by validating and sanitizing user inputs at the application level. +* **Cross-Site Scripting (XSS)**: Implement proper output encoding and use HTTP headers like `X-XSS-Protection` and `Content-Security-Policy`. + + nginx +add_header X-XSS-Protection "1; mode=block"; +add_header Content-Security-Policy "default-src 'self'"; + + +* **Clickjacking**: Prevent clickjacking by setting the `X-Frame-Options` header. + + nginx +add_header X-Frame-Options "SAMEORIGIN"; + + +* **Buffer Overflow**: Keep Nginx up to date to patch vulnerabilities related to buffer overflows. +* **Denial of Service (DoS)**: Implement rate limiting to prevent DoS attacks. + + nginx +limit_req_zone $binary_remote_addr zone=mylimit:10m rate=1r/s; + +location / { + limit_req zone=mylimit burst=5 nodelay; + # ... +} + + +### 4.2 Input Validation + +* **Validate User Inputs**: Validate all user inputs at the application level to prevent malicious data from reaching backend servers. +* **Limit Request Size**: Limit the size of incoming requests to prevent buffer overflows and DoS attacks. + + nginx +client_max_body_size 2m; + + +### 4.3 Authentication and Authorization + +* **Basic Authentication**: Use basic authentication for simple access control. However, it's not secure over HTTP; always use HTTPS. +* **JWT (JSON Web Tokens)**: Implement JWT-based authentication for APIs. +* **OAuth 2.0**: Use OAuth 2.0 for delegated authorization. +* **Access Control Lists (ACLs)**: Use ACLs to control access to specific resources based on IP addresses or other criteria. + + nginx +allow 192.168.1.0/24; +deny all; + + +### 4.4 Data Protection + +* **HTTPS**: Use HTTPS to encrypt all traffic between clients and the server. Obtain and configure SSL/TLS certificates. +* **Secure Storage**: Store sensitive data (e.g., passwords, API keys) securely using encryption and access controls. + +### 4.5 Secure API Communication + +* **API Keys**: Require API keys for all API requests. +* **Rate Limiting**: Implement rate limiting to prevent abuse and DoS attacks. +* **Input Validation**: Validate all API request parameters. +* **Output Encoding**: Encode API responses to prevent XSS attacks. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +Nginx configuration files are difficult to unit test directly. Unit testing is more relevant for custom Nginx modules written in C. + +### 5.2 Integration Testing + +* **Test Configuration Syntax**: Use `nginx -t` to test the syntax of your configuration files. +* **Test with Docker**: Use Docker containers to create isolated test environments. +* **Test with Real Traffic**: Use tools like `ab` (Apache Bench) or `wrk` to simulate real traffic and measure performance. + +### 5.3 End-to-End Testing + +Use tools like Selenium or Cypress to automate end-to-end tests of your application. + +### 5.4 Test Organization + +* **Separate Test Environment**: Create a separate test environment that mirrors your production environment. +* **Automated Testing**: Automate your testing process using CI/CD pipelines. + +### 5.5 Mocking and Stubbing + +* **Mock Backend Services**: Mock backend services to isolate your Nginx configuration and test different scenarios. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +* **Incorrect Syntax**: Nginx configuration syntax can be tricky. Always test your configuration files using `nginx -t`. +* **Missing Semicolons**: Forgetting semicolons at the end of directives. +* **Incorrect File Permissions**: Ensure that Nginx has the necessary permissions to read configuration files and access static content. +* **Not Reloading Configuration**: Forgetting to reload Nginx after making changes to the configuration files (e.g., `nginx -s reload`). +* **Using `if` Statements Inappropriately**: Overusing `if` can lead to unexpected behavior and performance issues. Prefer `try_files` or other alternatives. + +### 6.2 Edge Cases + +* **Handling Large Files**: Tune buffer sizes and timeouts when handling large file uploads or downloads. +* **WebSockets**: Configure Nginx to properly proxy WebSocket connections. +* **Long URIs**: Configure `client_header_buffer_size` and `large_client_header_buffers` to handle long URIs. + +### 6.3 Version-Specific Issues + +* **Check Release Notes**: Review the release notes for each Nginx version to identify any breaking changes or known issues. +* **Test Upgrades**: Test Nginx upgrades in a staging environment before deploying to production. + +### 6.4 Compatibility Concerns + +* **Backend Application Compatibility**: Ensure that your Nginx configuration is compatible with the backend applications you are using. +* **TLS Versions and Ciphers**: Choose TLS versions and ciphers that are compatible with your clients and backend servers. + +### 6.5 Debugging Strategies + +* **Error Logs**: Check Nginx error logs (`/var/log/nginx/error.log`) for errors and warnings. +* **Access Logs**: Analyze Nginx access logs (`/var/log/nginx/access.log`) to understand traffic patterns and identify potential issues. +* **`nginx -t`**: Use `nginx -t` to test the syntax of your configuration files. +* **Verbose Logging**: Enable verbose logging to capture more detailed information about requests and responses. +* **tcpdump/Wireshark**: Use `tcpdump` or Wireshark to capture and analyze network traffic. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +* **Text Editor**: Use a text editor with syntax highlighting and autocompletion for Nginx configuration files (e.g., VS Code with Nginx configuration extension). +* **Command-Line Tools**: Use command-line tools like `nginx`, `curl`, `ab`, and `tcpdump` for testing and debugging. +* **Docker**: Use Docker for creating isolated test environments. + +### 7.2 Build Configuration + +* **Version Control**: Store your Nginx configuration files in a version control system (e.g., Git). +* **Configuration Management**: Use configuration management tools like Ansible or Chef to automate the deployment and management of your Nginx configuration. + +### 7.3 Linting and Formatting + +* **Nginx Linter**: Use an Nginx linter to check your configuration files for syntax errors and best practices violations (e.g., `nginx -t`). +* **Code Formatting**: Use a code formatter to ensure consistent formatting of your configuration files. + +### 7.4 Deployment Best Practices + +* **Blue-Green Deployment**: Use blue-green deployment to minimize downtime during deployments. +* **Canary Deployment**: Use canary deployment to test new Nginx configurations with a small subset of users before rolling them out to the entire user base. +* **Rollback Strategy**: Have a rollback strategy in place in case of deployment failures. + +### 7.5 CI/CD Integration + +* **Automated Testing**: Integrate automated testing into your CI/CD pipeline. +* **Configuration Validation**: Validate Nginx configuration files as part of your CI/CD pipeline. +* **Automated Deployment**: Automate the deployment of Nginx configuration files using CI/CD tools like Jenkins, GitLab CI, or CircleCI. + +By following these best practices, you can create a robust, secure, and performant Nginx configuration that meets the needs of your web applications. \ No newline at end of file diff --git a/.cursor/rules/nltk.mdc b/.cursor/rules/nltk.mdc new file mode 100644 index 000000000..f0be4aa7f --- /dev/null +++ b/.cursor/rules/nltk.mdc @@ -0,0 +1,304 @@ +--- +description: Provides comprehensive guidance on best practices for coding standards, performance, security, and testing in NLTK projects. This rule helps developers write clean, maintainable, and efficient NLP code using NLTK. +globs: *.py +--- +- Follow these guidelines for consistency and maintainability of your code. + +# NLTK Coding Standards and Best Practices + +This document outlines coding standards and best practices for developing applications using the Natural Language Toolkit (NLTK) in Python. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure Best Practices + +Adopt a clear and logical directory structure to organize your NLTK project. A typical structure might look like this: + + +my_nltk_project/ +├── data/ # Raw data files +├── processed_data/ # Processed data files (e.g., tokenized, stemmed) +├── models/ # Trained models (e.g., sentiment analysis models) +├── scripts/ # Python scripts for data processing, training, etc. +│ ├── __init__.py # Make the scripts directory a Python package +│ ├── data_processing.py +│ ├── model_training.py +│ └── utils.py +├── tests/ # Unit and integration tests +│ ├── __init__.py +│ ├── test_data_processing.py +│ └── test_model_training.py +├── notebooks/ # Jupyter notebooks for experimentation +├── requirements.txt # Project dependencies +├── README.md +└── .gitignore + + +* `data/`: Stores raw data used for training and analysis. Keep this separate and under version control if the dataset is small. For larger datasets, use DVC (Data Version Control) or similar. +* `processed_data/`: Save intermediate and final processed data here. This avoids recomputing the same steps repeatedly. Consider using Parquet or Feather format for efficient storage and retrieval of processed dataframes. +* `models/`: Stores trained NLTK models. Use a consistent naming convention (e.g., `sentiment_model_v1.pkl`) and consider using a model registry (e.g., MLflow, Weights & Biases) for managing model versions and metadata. +* `scripts/`: Contains reusable Python modules for data processing, model training, and utility functions. Structure this directory into submodules if the project becomes complex. +* `tests/`: Holds unit tests and integration tests to ensure code correctness. Aim for high test coverage. +* `notebooks/`: Jupyter notebooks are useful for exploring data and prototyping code. Keep notebooks clean and well-documented, and consider refactoring useful code into the `scripts/` directory. +* `requirements.txt`: Lists all project dependencies, allowing for easy installation via `pip install -r requirements.txt`. Use `pip freeze > requirements.txt` to generate this file. + +### 1.2 File Naming Conventions + +Follow consistent file naming conventions to improve readability and maintainability: + +* Python scripts: Use snake_case (e.g., `data_processing.py`). +* Data files: Use descriptive names that indicate the contents (e.g., `raw_text_data.csv`, `processed_data.parquet`). +* Model files: Include versioning and relevant information in the name (e.g., `sentiment_model_v1.pkl`). +* Test files: Prefix with `test_` and mirror the naming of the modules they test (e.g., `test_data_processing.py`). + +### 1.3 Module Organization Best Practices + +Organize your code into logical modules within the `scripts/` directory. Each module should focus on a specific task or set of related functionalities. Here's an example: + +python +# scripts/data_processing.py + +import nltk + +def tokenize_text(text): + # Tokenize text using NLTK + tokens = nltk.word_tokenize(text) + return tokens + +def remove_stopwords(tokens): + # Remove stopwords using NLTK + stopwords = nltk.corpus.stopwords.words('english') + filtered_tokens = [token for token in tokens if token.lower() not in stopwords] + return filtered_tokens + +# scripts/model_training.py + +import nltk +import pickle + +def train_sentiment_model(data): + # Train a sentiment analysis model using NLTK + # ... + model = ... + return model + +def save_model(model, filepath): + # Save the trained model to a file + with open(filepath, 'wb') as f: + pickle.dump(model, f) + + +### 1.4 Component Architecture Recommendations + +Design your application with a clear component architecture. Consider using a layered architecture: + +* **Data Layer:** Handles data loading, cleaning, and preprocessing. Responsible for interacting with data sources (e.g., files, databases, APIs). +* **Processing Layer:** Implements NLP tasks such as tokenization, stemming, lemmatization, POS tagging, and parsing. This layer relies heavily on NLTK functionalities. +* **Model Layer:** Trains and evaluates NLP models. Includes code for feature extraction, model selection, and hyperparameter tuning. +* **Application Layer:** Provides the user interface or API for interacting with the NLP application. Handles user input and presents results. + +### 1.5 Code Splitting Strategies + +Break down large code files into smaller, more manageable chunks. Use functions, classes, and modules to improve code organization and reusability. + +* **Functions:** Encapsulate specific tasks into well-defined functions with clear inputs and outputs. Use docstrings to document function purpose, arguments, and return values. +* **Classes:** Group related data and functions into classes. Use inheritance and polymorphism to create reusable and extensible code. For example, you could create a base `TextProcessor` class and subclasses for different text processing tasks (e.g., `SentimentAnalyzer`, `TopicModeler`). +* **Modules:** Organize code into modules based on functionality. Use relative imports to access modules within the same package. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns + +* **Strategy Pattern:** Use this pattern to implement different text processing algorithms (e.g., stemming algorithms). Define a common interface for all algorithms and allow the client to choose the desired algorithm at runtime. +* **Factory Pattern:** Use this pattern to create different types of NLTK objects (e.g., different types of tokenizers or stemmers). This promotes loose coupling and makes it easier to switch between different implementations. +* **Observer Pattern:** Use this pattern to notify interested components when data changes (e.g., when a new document is added to a corpus). This can be useful for real-time NLP applications. + +### 2.2 Recommended Approaches for Common Tasks + +* **Text Preprocessing:** Create a reusable pipeline for text preprocessing. This pipeline should include tokenization, lowercasing, punctuation removal, stopword removal, stemming/lemmatization, and any other necessary steps. +* **Feature Extraction:** Use NLTK's feature extraction capabilities (e.g., bag-of-words, TF-IDF) to convert text data into numerical features suitable for machine learning models. +* **Model Training:** Use scikit-learn or other machine learning libraries in conjunction with NLTK to train and evaluate NLP models. Use cross-validation to estimate model performance and hyperparameter tuning to optimize model parameters. +* **Sentiment Analysis:** Use pre-trained sentiment analysis models (e.g., VADER) or train your own model using NLTK and scikit-learn. +* **Topic Modeling:** Use NLTK and Gensim to perform topic modeling on text data. + +### 2.3 Anti-patterns and Code Smells + +* **Global Variables:** Avoid using global variables, as they can lead to unexpected side effects and make code difficult to debug. Use dependency injection or other techniques to pass data between components. +* **Hardcoded Values:** Avoid hardcoding values in your code. Use configuration files or environment variables to store configurable parameters. +* **Long Functions:** Avoid writing long, complex functions. Break down large functions into smaller, more manageable chunks. +* **Code Duplication:** Avoid duplicating code. Create reusable functions or classes to encapsulate common functionality. +* **Ignoring Errors:** Don't ignore errors. Implement proper error handling to prevent unexpected crashes and provide informative error messages. +* **Over-commenting/Under-commenting:** Strike a balance. Comments should explain *why* you're doing something, not *what* the code is already clearly showing. Conversely, don't under-comment complex logic. + +### 2.4 State Management Best Practices + +* **Stateless Functions:** Prefer stateless functions whenever possible. Stateless functions are easier to test and reason about. +* **Immutable Data Structures:** Use immutable data structures whenever possible. Immutable data structures prevent accidental modification of data and make code more robust. +* **Configuration Management:** Use a configuration management library (e.g., `configparser`) to store application settings and parameters. This makes it easier to change settings without modifying code. + +### 2.5 Error Handling Patterns + +* **Try-Except Blocks:** Use `try-except` blocks to handle potential exceptions. Catch specific exceptions rather than using a generic `except` block. Log exceptions for debugging purposes. +* **Logging:** Use the `logging` module to log events, errors, and warnings. Configure logging levels to control the amount of information that is logged. Use a consistent logging format. +* **Custom Exceptions:** Define custom exceptions for specific error conditions in your application. This makes it easier to handle errors in a consistent and informative way. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +* **Profiling:** Use profiling tools (e.g., `cProfile`) to identify performance bottlenecks in your code. +* **Vectorization:** Use NumPy's vectorized operations to perform calculations on arrays of data. Vectorization is significantly faster than looping over individual elements. +* **Caching:** Use caching to store frequently accessed data and avoid redundant calculations. Use the `functools.lru_cache` decorator for simple caching. +* **Parallelization:** Use multiprocessing or threading to parallelize computationally intensive tasks. +* **Efficient Data Structures:** Choose appropriate data structures for your needs. Use sets for fast membership testing and dictionaries for fast key-value lookups. +* **Lazy Loading:** Defer the loading of large datasets or models until they are actually needed. This can significantly reduce startup time. + +### 3.2 Memory Management + +* **Generators:** Use generators to process large datasets in chunks. Generators produce data on demand, which reduces memory consumption. +* **Data Type Optimization:** Use the most appropriate data types for your data. For example, use `int8` instead of `int64` if your integers are within the range of `int8`. +* **Garbage Collection:** Be aware of Python's garbage collection mechanism. Avoid creating circular references that can prevent objects from being garbage collected. +* **Del Statements:** Use `del` statements to explicitly release memory when objects are no longer needed. + +### 3.3 Rendering Optimization (If Applicable) + +NLTK itself doesn't typically involve direct rendering like a UI framework, but visualization of NLTK outputs (e.g., parse trees) might. In such cases, libraries like Matplotlib or Graphviz might be used. Follow their respective optimization guides. + +### 3.4 Bundle Size Optimization + +This is more relevant for web applications using NLTK in the backend. For such cases: + +* **Dependency Management:** Only include the NLTK modules that are actually needed by your application. Avoid importing entire modules if you only need a few functions. +* **Code Minification:** Minify your Python code to reduce its size. This can be done using tools like `pyminify`. + +### 3.5 Lazy Loading Strategies + +* **Delayed Imports:** Import NLTK modules only when they are needed. This can reduce startup time if your application doesn't use all of NLTK's functionalities. +* **Conditional Loading:** Load different NLTK modules based on the application's configuration. This allows you to only load the modules that are relevant to the current task. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and Prevention + +* **Arbitrary Code Execution:** Be careful when loading data from untrusted sources. Avoid using `eval()` or `pickle.load()` on untrusted data, as these can be exploited to execute arbitrary code. Use safer alternatives such as JSON or CSV. +* **Denial of Service (DoS):** Protect your application from DoS attacks by limiting the size of input data and using appropriate timeouts. Avoid unbounded loops or recursive functions that can consume excessive resources. +* **Regular Expression Denial of Service (ReDoS):** Be careful when using regular expressions, as complex regular expressions can be exploited to cause ReDoS attacks. Use simple and efficient regular expressions, and limit the backtracking depth. + +### 4.2 Input Validation + +* **Data Type Validation:** Validate the data type of input values to prevent type errors and unexpected behavior. +* **Range Validation:** Validate that input values are within acceptable ranges. +* **Format Validation:** Validate that input data is in the expected format (e.g., using regular expressions). +* **Sanitization:** Sanitize input data to remove potentially harmful characters or code. Use appropriate escaping techniques to prevent injection attacks. + +### 4.3 Authentication and Authorization + +* **Authentication:** Implement authentication to verify the identity of users. Use strong passwords and multi-factor authentication. +* **Authorization:** Implement authorization to control access to resources. Use role-based access control (RBAC) to assign permissions to users based on their roles. + +### 4.4 Data Protection + +* **Encryption:** Encrypt sensitive data at rest and in transit. Use strong encryption algorithms and manage keys securely. +* **Data Masking:** Mask sensitive data when displaying it to users. This can prevent unauthorized access to sensitive information. +* **Access Control:** Implement strict access control policies to limit access to sensitive data. Only grant access to users who need it. + +### 4.5 Secure API Communication + +* **HTTPS:** Use HTTPS to encrypt communication between your application and the API. This prevents eavesdropping and man-in-the-middle attacks. +* **API Keys:** Use API keys to authenticate your application with the API. Protect your API keys and do not embed them in your code. +* **Rate Limiting:** Implement rate limiting to prevent abuse of your API. This can protect your API from DoS attacks. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +* **Test-Driven Development (TDD):** Write unit tests before writing code. This helps you to design your code in a testable way and ensures that your code meets the requirements. +* **Test Coverage:** Aim for high test coverage. Use a test coverage tool to measure the percentage of code that is covered by tests. +* **Assertions:** Use assertions to verify that your code is behaving as expected. Use informative error messages to help you debug your code. + +### 5.2 Integration Testing + +* **Test Dependencies:** Test the integration of your code with external dependencies (e.g., databases, APIs). Use mock objects or test doubles to isolate your code from external dependencies during unit testing. +* **Test Scenarios:** Test different scenarios to ensure that your code handles different inputs and edge cases correctly. + +### 5.3 End-to-End Testing + +* **User Interface Testing:** Test the user interface of your application to ensure that it is working as expected. Use automated testing tools to automate UI testing. +* **System Testing:** Test the entire system to ensure that all components are working together correctly. + +### 5.4 Test Organization + +* **Test Directory:** Create a separate `tests/` directory to store your tests. +* **Test Modules:** Create separate test modules for each module in your application. Use the same naming conventions as your application modules. +* **Test Classes:** Group related tests into test classes. Use descriptive names for your test classes. +* **Test Functions:** Create separate test functions for each test case. Use descriptive names for your test functions. + +### 5.5 Mocking and Stubbing + +* **Mock Objects:** Use mock objects to replace external dependencies during unit testing. This allows you to isolate your code from external dependencies and test it in a controlled environment. +* **Stubbing:** Use stubbing to replace complex or time-consuming operations with simple or fast operations during testing. This can improve the speed and reliability of your tests. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +* **Incorrect Data Preparation:** Failing to properly clean and prepare data can lead to inaccurate results. +* **Overfitting:** Training a model that is too complex for the data can lead to overfitting. Use regularization techniques and cross-validation to prevent overfitting. +* **Ignoring Rare Words:** Ignoring rare words can lead to a loss of information. Use techniques such as stemming or lemmatization to group related words together. +* **Using Default Parameters:** Relying on default parameters without understanding their implications can lead to unexpected behavior. +* **Not Documenting Code:** Failing to document code makes it difficult to understand and maintain. + +### 6.2 Edge Cases + +* **Empty Strings:** Handle empty strings gracefully. +* **Special Characters:** Handle special characters correctly. +* **Unicode:** Handle Unicode characters correctly. Use UTF-8 encoding for all text files. +* **Missing Data:** Handle missing data appropriately. + +### 6.3 Version-Specific Issues + +* **API Changes:** Be aware of API changes between different versions of NLTK. Check the NLTK documentation for migration guides. +* **Dependency Conflicts:** Be aware of potential dependency conflicts between NLTK and other libraries. Use a virtual environment to isolate your project's dependencies. + +### 6.4 Compatibility Concerns + +* **Python Version:** Ensure that your code is compatible with the desired version of Python. +* **Operating System:** Be aware of potential compatibility issues between different operating systems. + +### 6.5 Debugging Strategies + +* **Print Statements:** Use `print` statements to debug your code. Print the values of variables and the results of calculations. +* **Debuggers:** Use a debugger to step through your code and inspect the values of variables. +* **Logging:** Use the `logging` module to log events, errors, and warnings. Review the logs to identify the cause of problems. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +* **IDE:** Use an IDE such as Visual Studio Code, PyCharm, or Spyder. These IDEs provide features such as code completion, debugging, and refactoring. +* **Virtual Environment Manager:** Use a virtual environment manager such as `venv` or `conda` to isolate your project's dependencies. +* **Package Manager:** Use a package manager such as `pip` or `conda` to install and manage your project's dependencies. +* **Version Control System:** Use a version control system such as Git to track changes to your code. +* **Data Version Control (DVC):** For larger datasets that are under version control use DVC. + +### 7.2 Build Configuration + +* **Makefile:** Use a `Makefile` to automate common tasks such as building, testing, and deploying your application. +* **Setup Script:** Use a `setup.py` script to package your application for distribution. + +### 7.3 Linting and Formatting + +* **Linting:** Use a linter such as `pylint` or `flake8` to check your code for style errors and potential problems. +* **Formatting:** Use a code formatter such as `black` or `autopep8` to automatically format your code according to a consistent style guide. + +### 7.4 Deployment + +* **Containerization:** Use containerization technologies such as Docker to package your application and its dependencies into a self-contained unit. This makes it easier to deploy your application to different environments. +* **Cloud Platform:** Deploy your application to a cloud platform such as AWS, Google Cloud, or Azure. + +### 7.5 CI/CD Integration + +* **Continuous Integration:** Use a continuous integration (CI) system such as Jenkins, Travis CI, or CircleCI to automatically build and test your code whenever changes are pushed to your version control system. +* **Continuous Deployment:** Use a continuous deployment (CD) system to automatically deploy your application to production after it has passed all tests. + +By adhering to these coding standards and best practices, you can develop NLTK applications that are clean, maintainable, efficient, and secure. \ No newline at end of file diff --git a/.cursor/rules/nose2.mdc b/.cursor/rules/nose2.mdc new file mode 100644 index 000000000..3999344d1 --- /dev/null +++ b/.cursor/rules/nose2.mdc @@ -0,0 +1,316 @@ +--- +description: Comprehensive best practices and coding standards for Python projects using the nose2 testing framework. Covers code organization, common patterns, performance, security, testing, and tooling. +globs: *test*.py +--- +# nose2 Best Practices and Coding Standards + +This document provides a comprehensive guide to best practices and coding standards for Python projects using the nose2 testing framework. Following these guidelines will help you write clean, maintainable, and efficient testing code. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure + +- **Dedicated Test Directory:** Maintain a dedicated directory (e.g., `tests/`) at the project root to house all test files. This keeps test code separate from the application code. +- **Mirror Application Structure:** Mirror the structure of your application code within the test directory. For example, if your application has modules in `myapp/module1/` and `myapp/module2/`, create corresponding directories `tests/module1/` and `tests/module2/`. +- **Test Modules:** Each application module should have a corresponding test module. For `myapp/module.py`, the test module should be named `tests/test_module.py` or `tests/module_test.py` + +Example: + + +myproject/ +├── myapp/ +│ ├── __init__.py +│ ├── module1.py +│ ├── module2.py +│ └── ... +├── tests/ +│ ├── __init__.py +│ ├── test_module1.py +│ ├── test_module2.py +│ └── ... +├── ... + + +### 1.2. File Naming Conventions + +- **Test File Prefix:** Name test files with a prefix like `test_` or a suffix `_test` (e.g., `test_module.py` or `module_test.py`). nose2 automatically discovers files that follow this convention. +- **Descriptive Names:** Use descriptive file names that clearly indicate the module or component being tested. For example, `test_user_authentication.py` for tests related to user authentication. + +### 1.3. Module Organization + +- **Single Responsibility:** Each test module should focus on testing a single application module or a closely related set of functionalities. +- **Test Classes:** Organize tests within a module into test classes. Each class should test a specific aspect or component of the module. Inherit from `unittest.TestCase`. +- **Test Functions/Methods:** Each test case should be a separate function (starting with `test_`) within a test class. These methods should be small, focused, and test a single condition or scenario. + +Example: + +python +# tests/test_user.py +import unittest +from myapp import user + +class TestUser(unittest.TestCase): + + def setUp(self): + # Setup code to run before each test + self.user = user.User("testuser", "password") + + def tearDown(self): + # Teardown code to run after each test + pass + + def test_user_creation(self): + self.assertEqual(self.user.username, "testuser") + self.assertTrue(self.user.check_password("password")) + + def test_invalid_password(self): + self.assertFalse(self.user.check_password("wrong_password")) + + + +### 1.4. Component Architecture + +- **Isolate Components:** Design your application to isolate components, making them easier to test independently. Use dependency injection or other techniques to decouple modules. +- **Testable Interfaces:** Define clear and testable interfaces between components. This allows you to mock or stub dependencies during testing. + +### 1.5. Code Splitting Strategies + +- **Functional Decomposition:** Split large test functions into smaller, more manageable functions that focus on specific aspects of the test. +- **Parameterization:** Use parameterized tests (e.g., with `nose2.tools.params`) to test the same code with different inputs and expected outputs. This avoids code duplication and improves test coverage. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +- **Arrange-Act-Assert (AAA):** Follow the AAA pattern in each test: _Arrange_ the test environment (set up data, mock dependencies), _Act_ (execute the code under test), and _Assert_ (verify the expected outcome). +- **Test Fixtures:** Use `setUp` and `tearDown` methods in `unittest.TestCase` to create and clean up test fixtures (e.g., database connections, temporary files) before and after each test. +- **Mocking:** Use mocking libraries like `unittest.mock` or `mock` to replace real dependencies with controlled substitutes during testing. This allows you to isolate the code under test and simulate various scenarios. + +### 2.2. Recommended Approaches + +- **Test-Driven Development (TDD):** Write tests before writing the application code. This helps you define clear requirements and ensure that the code is testable. +- **Behavior-Driven Development (BDD):** Use BDD frameworks like `behave` to write tests in a human-readable format that describes the expected behavior of the system. +- **Focus on Edge Cases:** Don't only test the happy path. Dedicate tests to verify edge cases, boundary conditions, and error handling. + +### 2.3. Anti-patterns and Code Smells + +- **Over-testing:** Avoid testing implementation details that are likely to change. Focus on testing the behavior and expected outputs of the code. +- **Fragile Tests:** Tests that break frequently due to minor code changes are fragile. Strive for tests that are resilient to refactoring and implementation changes. +- **Slow Tests:** Slow tests can significantly slow down the development process. Optimize your tests to run quickly. Use techniques like mocking and in-memory databases to speed up testing. +- **Ignoring Test Failures:** Always investigate and fix failing tests. Don't ignore them or disable them without a proper reason. +- **Redundant Assertions:** Avoid having more than one assertion per test case unless absolutely necessary. Multiple assertions make it hard to pinpoint what failed. + +### 2.4. State Management + +- **Isolate Test State:** Each test should be independent and not rely on the state left by previous tests. Use `setUp` and `tearDown` to ensure a clean state for each test. +- **Avoid Global State:** Minimize the use of global variables or shared state in your application, as they can make tests harder to write and maintain. + +### 2.5. Error Handling + +- **Test Exception Handling:** Write tests to verify that your code raises the correct exceptions in error scenarios. +- **Use `assertRaises`:** Use `self.assertRaises(ExceptionType, callable, *args, **kwargs)` to assert that a specific exception is raised when calling a function or method. +- **Context Managers for Asserting Exceptions:** Use context managers when checking for exceptions in more complex scenarios. + +Example: + +python +import unittest + +class MyClass: + def divide(self, a, b): + if b == 0: + raise ValueError("Cannot divide by zero") + return a / b + +class TestMyClass(unittest.TestCase): + + def setUp(self): + self.my_class = MyClass() + + def test_divide_valid(self): + self.assertEqual(self.my_class.divide(10, 2), 5) + + def test_divide_by_zero(self): + with self.assertRaises(ValueError) as context: + self.my_class.divide(10, 0) + self.assertEqual(str(context.exception), "Cannot divide by zero") + + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +- **Profiling:** Use profiling tools to identify performance bottlenecks in your tests. Optimize the slowest tests first. +- **Mocking Expensive Operations:** Mock out slow operations (e.g., database queries, network requests) during testing to reduce test execution time. +- **Caching Test Data:** Cache frequently used test data to avoid redundant calculations or data retrieval. + +### 3.2. Memory Management + +- **Clean Up Resources:** Ensure that tests properly clean up any resources they allocate (e.g., files, database connections) to avoid memory leaks. +- **Limit Test Data Size:** Use realistic but minimal test data to reduce memory consumption during testing. + +### 3.3. Rendering Optimization (If Applicable) + +- This is generally not applicable to `nose2` testing itself, but if testing code involves UI rendering, strategies like reducing render frequency and complexity should be considered. + +### 3.4. Bundle Size Optimization (If Applicable) + +- This is generally not applicable to `nose2` testing itself. + +### 3.5. Lazy Loading + +- **Lazy Initialization of Test Data:** Defer the initialization of test data until it is actually needed by a test case. This can reduce test setup time and memory consumption. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities + +- **Insecure Dependencies:** Use dependency scanning tools to identify and address vulnerabilities in third-party libraries used by your application or tests. +- **Hardcoded Secrets:** Avoid hardcoding secrets (e.g., passwords, API keys) in your tests. Use environment variables or configuration files to store sensitive information. +- **Insufficient Input Validation:** Test that your application properly validates user inputs to prevent injection attacks and other security vulnerabilities. + +### 4.2. Input Validation + +- **Test Boundary Conditions:** Test that your application handles invalid or unexpected inputs gracefully, including boundary conditions, edge cases, and malicious inputs. + +### 4.3. Authentication and Authorization + +- **Test Authentication Flows:** Write tests to verify that your application correctly authenticates users and grants access to authorized resources. +- **Role-Based Access Control (RBAC):** If your application uses RBAC, test that users with different roles have the appropriate permissions. + +### 4.4. Data Protection + +- **Test Data Encryption:** If your application encrypts sensitive data, test that the encryption and decryption processes are working correctly. +- **Secure Storage:** Test that sensitive data is stored securely (e.g., using hashed passwords, encrypted databases). + +### 4.5. Secure API Communication + +- **Test HTTPS:** If your application communicates with external APIs, test that it uses HTTPS to encrypt the communication. +- **API Key Security:** Protect API keys and other credentials used for API communication. Don't store them directly in the code, and use appropriate access controls. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +- **Focus on Individual Units:** Unit tests should focus on testing individual functions, methods, or classes in isolation. +- **Mock Dependencies:** Use mocking to isolate the unit under test from its dependencies. +- **Test All Code Paths:** Aim for high code coverage by testing all possible code paths and branches. + +### 5.2. Integration Testing + +- **Test Interactions Between Components:** Integration tests verify that different components of the application work together correctly. +- **Use Real Dependencies (Where Possible):** Unlike unit tests, integration tests should use real dependencies (e.g., databases, external APIs) where feasible, or integration-specific lightweight replacements. + +### 5.3. End-to-End (E2E) Testing + +- **Test the Entire System:** E2E tests simulate real user scenarios and test the entire application from end to end. +- **Use Automated Testing Tools:** Use automated testing tools like Selenium or Cypress to automate E2E tests. + +### 5.4. Test Organization + +- **Test Suites:** Use test suites to group related tests together. This makes it easier to run specific sets of tests. +- **Test Runners:** Use test runners (e.g., `nose2`, `pytest`) to discover and execute tests automatically. + +### 5.5. Mocking and Stubbing + +- **`unittest.mock`:** Use the `unittest.mock` library (or the older `mock` library for Python 2) to create mock objects and stub out dependencies. +- **`patch` Decorator:** Use the `patch` decorator to replace objects or attributes with mock objects during testing. +- **`side_effect`:** Use the `side_effect` attribute of mock objects to simulate different behaviors or raise exceptions when the mock object is called. + +Example: + +python +import unittest +from unittest.mock import patch + +class MyClass: + def get_data(self): + # This method might make an external API call + return "Real data" + + def process_data(self): + data = self.get_data() + return data.upper() + +class TestMyClass(unittest.TestCase): + + @patch('__main__.MyClass.get_data') # Patch the get_data method + def test_process_data(self, mock_get_data): + # Configure the mock to return a specific value + mock_get_data.return_value = "Mock data" + + my_class = MyClass() + result = my_class.process_data() + + self.assertEqual(result, "MOCK DATA") + mock_get_data.assert_called_once() # Verify that the mock was called + + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +- **Not Using `setUp` and `tearDown` Correctly:** Failing to properly set up test conditions or clean up resources after each test can lead to inconsistent test results. +- **Ignoring Dependencies:** Forgetting to mock out dependencies can make tests slow, unreliable, and prone to external factors. +- **Asserting Too Much in One Test:** Trying to test too many things in a single test case makes it difficult to identify the source of failures and maintain the tests. +- **Writing Tests That Are Too Specific:** Writing tests that are tightly coupled to the implementation details can make them brittle and difficult to maintain. + +### 6.2. Edge Cases + +- **Null Values:** Test how your application handles null or missing values. +- **Empty Collections:** Test how your application handles empty lists, dictionaries, or sets. +- **Large Data Sets:** Test how your application performs with large amounts of data. +- **Special Characters:** Test how your application handles special characters in user inputs. +- **Date and Time Zones:** Test how your application handles different date and time zones. + +### 6.3. Version-Specific Issues + +- **Python Version Compatibility:** nose2 requires Python 3. Ensure compatibility across different minor versions of Python 3. +- **Dependency Versions:** Be aware of compatibility issues between nose2 and its dependencies. Specify version constraints in your project's requirements file. + +### 6.4. Compatibility Concerns + +- **`unittest` Compatibility:** nose2 extends `unittest`, so understanding `unittest` is fundamental. Be aware of any differences in behavior between standard `unittest` and nose2. + +### 6.5. Debugging Strategies + +- **Use a Debugger:** Use a debugger (e.g., `pdb`, `ipdb`) to step through your tests and examine the state of the application. +- **Print Statements:** Use `print` statements to output debugging information to the console. +- **Logging:** Use the `logging` module to record debugging information to a file. This is useful for debugging tests that run in a CI/CD environment. +- **Isolate the Problem:** When a test fails, try to isolate the problem by running the test in isolation or by commenting out parts of the test. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +- **IDE:** PyCharm, VS Code with Python extension, or other Python-friendly IDE. +- **Text Editor:** Sublime Text, Atom, or other text editor. +- **Debugger:** `pdb` (Python Debugger) or `ipdb` (IPython Debugger). +- **Mocking Library:** `unittest.mock` (built-in) or `mock` (for older Python versions). + +### 7.2. Build Configuration + +- **`setup.py` or `pyproject.toml`:** Use `setup.py` or `pyproject.toml` to define your project's dependencies and build configuration. +- **`requirements.txt`:** Use `requirements.txt` to list the project's dependencies. Generate this file using `pip freeze > requirements.txt`. +- **Virtual Environments:** Use virtual environments (e.g., `venv`, `virtualenv`) to isolate the project's dependencies from the system's Python environment. + +### 7.3. Linting and Formatting + +- **Pylint:** Use Pylint to enforce coding standards and detect potential errors in your code. Configure Pylint with a `.pylintrc` file. +- **pycodestyle (formerly pep8):** Use pycodestyle to check your code for compliance with PEP 8. +- **Flake8:** Use Flake8 as a wrapper around pycodestyle, pyflakes, and McCabe complexity checker. +- **Black:** Use Black to automatically format your code according to a consistent style. +- **isort:** Use isort to automatically sort your imports. + +### 7.4. Deployment Best Practices + +- This is generally out of scope of `nose2` itself, but automated testing as part of the deployment pipeline ensures high quality code. + +### 7.5. CI/CD Integration + +- **GitHub Actions, GitLab CI, Jenkins:** Integrate nose2 into your CI/CD pipeline to automatically run tests whenever code is pushed or merged. +- **Test Reporting:** Configure your CI/CD system to generate test reports and display test results. +- **Code Coverage Reporting:** Integrate code coverage tools (e.g., `coverage.py`) into your CI/CD pipeline to track code coverage and identify areas that need more testing. +- **Fail Fast:** Configure your CI/CD pipeline to fail immediately if any tests fail. This prevents broken code from being deployed to production. + +By following these best practices and coding standards, you can create robust and reliable Python testing code using nose2. \ No newline at end of file diff --git a/.cursor/rules/notion-api.mdc b/.cursor/rules/notion-api.mdc new file mode 100644 index 000000000..f89f5e394 --- /dev/null +++ b/.cursor/rules/notion-api.mdc @@ -0,0 +1,263 @@ +--- +description: This rule provides comprehensive best practices for developing applications using the notion-api library, covering code organization, performance, security, testing, and common pitfalls. +globs: *.js,*.ts,*.jsx,*.tsx +--- +# Notion API Library Best Practices + +This document outlines the recommended best practices for developing applications using the `notion-api` library. Following these guidelines will help you create maintainable, performant, secure, and testable Notion integrations. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure Best Practices + +Adopt a clear and consistent directory structure to improve code maintainability and collaboration. Here's a suggested structure for projects using `notion-api`: + + +project-root/ + ├── src/ + │ ├── components/ # Reusable UI components (if applicable) + │ ├── pages/ # Page-specific components/logic (if applicable) + │ ├── services/ # Notion API interaction logic + │ │ ├── notion-client.js # or notion-client.ts: Initializes the Notion client + │ │ ├── database-service.js # or database-service.ts: Handles database interactions + │ │ ├── page-service.js # or page-service.ts: Handles page interactions + │ │ └── utils.js # or utils.ts: Utility functions for API calls, data formatting + │ ├── models/ # Data models for Notion objects + │ ├── utils/ # General utility functions + │ ├── config/ # Configuration files + │ └── app.js # or app.ts: Main application entry point + ├── tests/ # Unit and integration tests + ├── .env # Environment variables + ├── package.json # Project dependencies + └── README.md # Project documentation + + +### 1.2. File Naming Conventions + +Use descriptive and consistent file names: + +* **Components:** `ComponentName.jsx` or `ComponentName.tsx` +* **Services:** `service-name.js` or `service-name.ts` +* **Models:** `model-name.js` or `model-name.ts` +* **Utility functions:** `utility-name.js` or `utility-name.ts` +* **Configuration files:** `config-name.js` or `config-name.ts` + +### 1.3. Module Organization + +Organize your code into logical modules based on functionality. For `notion-api`, this commonly involves separating: + +* **Notion Client Initialization:** A module responsible for initializing the Notion client with your API key. This should handle authentication. +* **Data Access Layer (Services):** Modules that encapsulate all interactions with the Notion API. These modules should provide functions for common operations like creating pages, querying databases, and updating content. Use separate modules for database and page-related operations. +* **Data Models:** Define TypeScript interfaces or JavaScript classes to represent Notion objects (e.g., Page, Database, Block). This improves type safety and code readability. +* **Utility Functions:** Create utility functions for tasks such as formatting data for the Notion API, handling pagination, and error handling. + +### 1.4. Component Architecture (If Applicable) + +If you're building a UI that interacts with the Notion API, consider using a component-based architecture (e.g., React, Vue, Angular). Separate concerns by creating reusable components for displaying and interacting with Notion data. + +### 1.5. Code Splitting + +For larger applications, use code splitting to improve initial load times. Lazy-load components and modules that are not immediately needed. This is particularly useful for components that display large amounts of Notion data or perform complex API operations. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +* **Repository Pattern:** Use the repository pattern to abstract data access logic. This makes it easier to switch data sources in the future (e.g., using a mock API for testing). +* **Adapter Pattern:** Adapt the data returned by the Notion API to your application's specific needs. This helps decouple your application from the API's data structure. +* **Singleton Pattern (for Notion Client):** Use the singleton pattern to ensure only one instance of the Notion client is created. This can improve performance and avoid rate limiting issues. + +### 2.2. Recommended Approaches for Common Tasks + +* **Fetching Data:** Use asynchronous functions (`async/await`) to fetch data from the Notion API. Implement proper error handling (see below). +* **Creating Pages:** Use the `pages.create` method to create new pages. Ensure you provide the required properties for the parent (database or page). +* **Querying Databases:** Use the `databases.query` method to query databases. Learn how to use filters, sorts, and pagination to retrieve the data you need efficiently. +* **Updating Content:** Use the `blocks.update` and `pages.update` method to update content. Understand how to update different types of blocks and page properties. +* **Handling Pagination:** Implement proper pagination to handle large datasets. Use the `has_more` and `next_cursor` properties returned by the API to iterate through all pages of data. + +### 2.3. Anti-patterns and Code Smells + +* **Directly calling the Notion API in UI components:** This violates separation of concerns and makes testing difficult. Use a service layer to handle API interactions. +* **Hardcoding API keys:** Store API keys in environment variables and never commit them to your repository. +* **Ignoring errors:** Always handle errors and log them appropriately. Do not swallow exceptions. +* **Over-fetching data:** Only request the data you need. Use filters and projections to reduce the amount of data transferred. +* **Creating too many Notion client instances:** Re-use existing client instances to optimize performance and avoid rate limiting. +* **Lack of Pagination:** Neglecting to handle pagination when fetching large datasets from Notion databases. + +### 2.4. State Management (If Applicable) + +If you're building a UI, choose a state management solution that fits your application's complexity (e.g., React Context, Redux, Zustand). Store Notion data in a normalized format to improve performance and avoid unnecessary re-renders. Use selectors to efficiently retrieve data from the state. + +### 2.5. Error Handling Patterns + +Implement robust error handling to gracefully handle API errors and unexpected situations. + +* **Try-Catch Blocks:** Use `try-catch` blocks to wrap API calls and handle potential errors. +* **Error Logging:** Log errors with sufficient detail to facilitate debugging. +* **User Feedback:** Provide informative error messages to the user. +* **Retry Mechanism:** Implement a retry mechanism for transient errors (e.g., network errors, rate limiting). +* **Centralized Error Handling:** Create a centralized error handling function or middleware to handle errors consistently across your application. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +* **Caching:** Implement caching to reduce the number of API calls. Cache frequently accessed data in memory or in a persistent store. +* **Batching:** Batch multiple API requests into a single request to reduce network overhead (where supported by the API. Currently, Notion's API offers limited batching capabilities. +* **Minimize Data Transfer:** Only request the data you need. Use filters and projections to reduce the amount of data transferred. +* **Compression:** Enable compression (e.g., gzip) to reduce the size of API responses. +* **Optimize Database Queries:** Carefully design your database queries to retrieve data efficiently. Use indexes to improve query performance. + +### 3.2. Memory Management + +* **Avoid Memory Leaks:** Be mindful of memory leaks, especially in long-running applications. Release resources when they are no longer needed. +* **Use Efficient Data Structures:** Use efficient data structures to store Notion data. Avoid creating unnecessary copies of data. + +### 3.3. Rendering Optimization (If Applicable) + +* **Virtualization:** Use virtualization to efficiently render large lists of Notion blocks. +* **Memoization:** Use memoization to avoid re-rendering components that haven't changed. +* **Code Splitting:** As described above. + +### 3.4. Bundle Size Optimization + +* **Tree Shaking:** Use tree shaking to remove unused code from your bundles. +* **Minification:** Minify your code to reduce bundle size. +* **Code Splitting:** As described above. +* **Use a CDN:** Serve static assets from a CDN to improve loading times. + +### 3.5. Lazy Loading + +* **Lazy Load Images:** Lazy load images to improve initial load times. +* **Lazy Load Components:** As described above. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities and Prevention + +* **API Key Exposure:** Never commit API keys to your repository. Store them in environment variables and access them securely. +* **Injection Attacks:** Sanitize user input to prevent injection attacks (e.g., SQL injection, XSS). +* **Cross-Site Scripting (XSS):** If your application renders user-generated content from Notion, sanitize it to prevent XSS attacks. +* **Cross-Site Request Forgery (CSRF):** Implement CSRF protection to prevent malicious websites from making requests on behalf of authenticated users. +* **Man-in-the-Middle Attacks:** Ensure that all communication with the Notion API is encrypted using HTTPS. + +### 4.2. Input Validation + +* **Validate All User Input:** Validate all user input before sending it to the Notion API. This includes checking data types, formats, and ranges. +* **Sanitize User Input:** Sanitize user input to remove potentially harmful characters. + +### 4.3. Authentication and Authorization + +* **Use OAuth 2.0:** Use OAuth 2.0 for authentication and authorization. This allows users to grant your application access to their Notion data without sharing their passwords. +* **Store Tokens Securely:** Store access tokens securely. Use encryption or a secure storage mechanism (e.g., Keychain). +* **Follow the Principle of Least Privilege:** Only request the permissions you need. + +### 4.4. Data Protection + +* **Encrypt Sensitive Data:** Encrypt sensitive data at rest and in transit. +* **Follow Data Privacy Regulations:** Comply with all applicable data privacy regulations (e.g., GDPR, CCPA). +* **Regularly Audit Security Practices:** Regularly audit your security practices to identify and address potential vulnerabilities. + +### 4.5. Secure API Communication + +* **Use HTTPS:** Ensure that all communication with the Notion API is encrypted using HTTPS. +* **Validate SSL Certificates:** Validate SSL certificates to prevent man-in-the-middle attacks. +* **Use a Secure API Client:** Use a secure API client that supports encryption and certificate validation. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +* **Test Individual Components:** Unit test individual components and modules to ensure they function correctly in isolation. +* **Mock External Dependencies:** Mock external dependencies (e.g., the Notion API) to isolate your code and make tests more predictable. +* **Use a Testing Framework:** Use a testing framework like Jest or Mocha to write and run your unit tests. + +### 5.2. Integration Testing + +* **Test Interactions Between Components:** Integration test the interactions between different components and modules to ensure they work together correctly. +* **Use a Test Database:** Use a test database to avoid affecting your production data. +* **Seed the Test Database:** Seed the test database with sample data to make tests more realistic. + +### 5.3. End-to-End Testing + +* **Test the Entire Application Flow:** End-to-end test the entire application flow to ensure that everything works correctly from start to finish. +* **Use a Testing Framework:** Use a testing framework like Cypress or Puppeteer to write and run your end-to-end tests. +* **Run Tests in a CI/CD Pipeline:** Run your tests in a CI/CD pipeline to ensure that changes don't break existing functionality. + +### 5.4. Test Organization + +* **Create a Dedicated Test Directory:** Create a dedicated test directory to store your tests. +* **Follow a Consistent Naming Convention:** Follow a consistent naming convention for your test files. +* **Organize Tests by Feature:** Organize tests by feature to make them easier to find and maintain. + +### 5.5. Mocking and Stubbing + +* **Use Mocking Libraries:** Use mocking libraries like Jest or Sinon to create mocks and stubs. +* **Mock the Notion API:** Mock the Notion API to isolate your code and make tests more predictable. +* **Stub API Responses:** Stub API responses to control the data returned by the API. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +* **Not Understanding the Notion API Data Model:** Failing to understand the Notion API data model can lead to incorrect API calls and unexpected results. Study the documentation thoroughly. +* **Misunderstanding Rate Limits:** Exceeding rate limits can lead to temporary blocking. Implement proper rate limiting handling and retry mechanisms. +* **Improperly Handling Errors:** Ignoring errors can lead to silent failures and difficult debugging. Always handle errors and log them appropriately. +* **Not Using Pagination:** Failing to use pagination can lead to incomplete data retrieval. Implement proper pagination to handle large datasets. + +### 6.2. Edge Cases + +* **Empty Databases:** Handle the case where a database is empty or doesn't contain the expected properties. +* **Deleted Pages:** Handle the case where a page has been deleted or moved. +* **Permission Errors:** Handle the case where the application doesn't have the required permissions to access a page or database. +* **API Downtime:** Handle the case where the Notion API is temporarily unavailable. + +### 6.3. Version-Specific Issues + +* **Breaking Changes:** Be aware of breaking changes in new versions of the Notion API. Test your application thoroughly after upgrading. +* **Deprecated Features:** Avoid using deprecated features, as they may be removed in future versions. + +### 6.4. Compatibility Concerns + +* **Library Conflicts:** Be aware of potential library conflicts between `notion-api` and other libraries in your project. +* **Browser Compatibility:** Ensure that your application is compatible with the target browsers. + +### 6.5. Debugging Strategies + +* **Use Debugging Tools:** Use debugging tools to inspect your code and identify errors. +* **Log API Requests and Responses:** Log API requests and responses to help diagnose issues. +* **Use a Network Monitor:** Use a network monitor to inspect network traffic and identify performance bottlenecks. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +* **IDE:** Visual Studio Code, IntelliJ IDEA, or WebStorm +* **Node.js:** A JavaScript runtime environment +* **npm or Yarn:** Package managers +* **Git:** Version control system +* **Postman or Insomnia:** API testing tools + +### 7.2. Build Configuration + +* **Use a Build Tool:** Use a build tool like Webpack or Parcel to bundle your code. +* **Configure Environment Variables:** Configure environment variables to store API keys and other sensitive information. +* **Use a .gitignore File:** Use a `.gitignore` file to exclude sensitive files from your repository. + +### 7.3. Linting and Formatting + +* **Use a Linter:** Use a linter like ESLint or JSHint to enforce coding standards. +* **Use a Formatter:** Use a formatter like Prettier to automatically format your code. +* **Configure CI/CD:** Configure CI/CD to automatically lint and format your code. + +### 7.4. Deployment + +* **Choose a Deployment Platform:** Choose a deployment platform that fits your application's needs (e.g., Vercel, Netlify, AWS). +* **Configure Environment Variables:** Configure environment variables on your deployment platform to store API keys and other sensitive information. +* **Monitor Your Application:** Monitor your application to identify and address performance issues and errors. + +### 7.5. CI/CD Integration + +* **Use a CI/CD Platform:** Use a CI/CD platform like GitHub Actions or GitLab CI/CD to automate your build, test, and deployment processes. +* **Run Tests in CI/CD:** Run your tests in your CI/CD pipeline to ensure that changes don't break existing functionality. +* **Automate Deployment:** Automate deployment to reduce the risk of human error. \ No newline at end of file diff --git a/.cursor/rules/numba.mdc b/.cursor/rules/numba.mdc new file mode 100644 index 000000000..e373d4726 --- /dev/null +++ b/.cursor/rules/numba.mdc @@ -0,0 +1,299 @@ +--- +description: This rule provides comprehensive best practices and coding standards for the Numba library, covering code organization, performance optimization, security, testing, and common pitfalls. It aims to help developers write efficient, maintainable, and secure Numba code. +globs: *.py +--- +# Numba Best Practices and Coding Standards + +This document outlines best practices and coding standards for the Numba library to help developers write efficient, maintainable, and secure code. + +## Library Information: + +- Name: numba +- Tags: ai, ml, gpu-computing, python, jit-compiler + +## 1. Code Organization and Structure + +### 1.1. Directory Structure + +- **`src/`:** Contains the main source code. Subdivide into modules or packages based on functionality (e.g., `src/core`, `src/cuda`, `src/npyimpl`). +- **`tests/`:** Contains unit, integration, and end-to-end tests. Mirror the `src/` structure to easily locate corresponding tests. +- **`examples/`:** Provides example scripts and notebooks demonstrating how to use numba features. +- **`docs/`:** Stores documentation in reStructuredText or Markdown format. +- **`benchmarks/`:** Contains benchmark scripts for performance evaluation. +- **`scripts/`:** Utilities and helper scripts for development, testing, or deployment. +- **`.cursor/rules/`:** Store custom rules for AI-assisted code generation (e.g., this file). + +Example: + + +my_project/ +├── src/ +│ ├── __init__.py +│ ├── core/ +│ │ ├── __init__.py +│ │ ├── functions.py +│ │ ├── types.py +│ ├── cuda/ +│ │ ├── __init__.py +│ │ ├── kernels.py +│ │ ├── device.py +│ ├── npyimpl/ +│ │ ├── __init__.py +│ │ ├── array_methods.py +│ │ ├── reductions.py +├── tests/ +│ ├── __init__.py +│ ├── core/ +│ │ ├── test_functions.py +│ │ ├── test_types.py +│ ├── cuda/ +│ │ ├── test_kernels.py +│ │ ├── test_device.py +│ ├── npyimpl/ +│ │ ├── test_array_methods.py +│ │ ├── test_reductions.py +├── examples/ +│ ├── mandelbrot.py +│ ├── gaussian_blur.ipynb +├── docs/ +│ ├── source/ +│ │ ├── conf.py +│ │ ├── index.rst +│ ├── Makefile +├── benchmarks/ +│ ├── bench_matmul.py +├── scripts/ +│ ├── build_extensions.py +├── .cursor/ +│ ├── rules/ +│ │ ├── numba_rules.mdc +├── setup.py +├── README.md +├── LICENSE + + +### 1.2. File Naming Conventions + +- Python files: Use descriptive lowercase names with underscores (e.g., `array_methods.py`, `type_inference.py`). +- Test files: Prefix with `test_` and mirror the module/file being tested (e.g., `test_array_methods.py`). +- Class names: Use PascalCase (e.g., `NumbaTypeManager`, `CUDADeviceContext`). +- Function and variable names: Use lowercase with underscores (e.g., `jit_compile`, `input_array`). +- Constants: Use uppercase with underscores (e.g., `DEFAULT_NUM_THREADS`, `MAX_ARRAY_SIZE`). + +### 1.3. Module Organization + +- Group related functionality into modules. For example, put all type-related code in a `types` module. +- Use `__init__.py` files to define packages and control namespace imports. +- Keep modules focused and avoid large, monolithic modules. +- Use relative imports within packages (e.g., `from . import functions`). + +### 1.4. Component Architecture + +- Design with clear separation of concerns (e.g., type inference, code generation, runtime execution). +- Employ interfaces or abstract base classes for loose coupling between components. +- Use dependency injection to manage component dependencies. +- Favor composition over inheritance to promote code reuse and flexibility. + +### 1.5. Code Splitting + +- Break down large functions into smaller, more manageable functions. +- Group related functions into classes or modules. +- Use decorators to add functionality without modifying the original code (e.g., `@jit`, `@vectorize`). +- Extract common code into reusable utility functions. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +- **Decorator Pattern:** Used extensively in Numba via the `@jit`, `@njit`, `@vectorize`, and `@cuda.jit` decorators for applying transformations and optimizations to functions. +- **Factory Pattern:** Can be used to create different types of compiled functions based on runtime conditions or configurations. +- **Strategy Pattern:** Employ different compilation strategies based on target architecture or optimization level. + +### 2.2. Recommended Approaches + +- **Use `@njit` whenever possible:** This enforces no-python mode, maximizing performance. Only fall back to `@jit` if no-python mode is not feasible. +- **Explicitly specify types:** When possible, define input and output types using Numba's type system for increased performance and type safety. +- **Utilize `prange` for parallel loops:** When using `parallel=True`, use `numba.prange` instead of `range` for loops that can be executed in parallel. +- **Minimize data transfer between CPU and GPU:** When using CUDA, keep data on the GPU as much as possible and pre-allocate output arrays. +- **Use structured array when possible:** When working with complex data, leverage the power of structured arrays to improve data access times and readability. + +### 2.3. Anti-patterns + +- **Relying on object mode:** Object mode is significantly slower than no-python mode. Avoid it by ensuring your code is compatible with no-python mode. +- **Excessive use of global variables:** Global variables can hinder type inference and optimization. Pass data as function arguments instead. +- **Unnecessary data copying:** Avoid copying data between CPU and GPU or within memory unless absolutely necessary. +- **Ignoring profiling results:** Don't guess where performance bottlenecks are. Use profiling tools to identify and address them. +- **Using Python data structures like dictionaries and lists directly inside `@njit` decorated code:** These data structures usually trigger object mode and hinder performance. Opt for `typed-list` and `typed-dict` when possible. + +### 2.4. State Management + +- Avoid mutable global state within Numba-compiled functions. Mutating global state can lead to unpredictable behavior and make debugging difficult. +- If state needs to be maintained, consider using a class with methods decorated with `@njit` to encapsulate the state and operations. +- When working with CUDA, manage device memory explicitly to avoid memory leaks and ensure efficient resource utilization. + +### 2.5. Error Handling + +- Use `try...except` blocks to handle potential exceptions within Numba-compiled functions. +- Propagate errors gracefully and provide informative error messages. +- Consider using Numba's error reporting mechanisms for custom error handling. +- Handle CUDA errors properly when working with GPU code. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +- **Just-In-Time (JIT) Compilation:** Leverage `@jit` and `@njit` decorators for automatic code optimization. +- **Vectorization:** Utilize `@vectorize` decorator for element-wise operations on arrays. +- **Parallelization:** Employ `parallel=True` and `prange` for parallel loop execution on multi-core CPUs. +- **CUDA Acceleration:** Use `@cuda.jit` decorator to compile and execute code on NVIDIA GPUs. +- **`fastmath`:** Use `fastmath=True` when strict IEEE 754 compliance is not required to enable aggressive optimizations. +- **Loop Unrolling:** Manually unroll loops to reduce loop overhead when appropriate. +- **Memory Alignment:** Ensure data is properly aligned in memory for efficient access. +- **Kernel Fusion:** Combine multiple operations into a single kernel to reduce memory access and improve performance. +- **Use Intel SVML:** If possible, install and use the Intel Short Vector Math Library (SVML) for optimized transcendental functions. + +### 3.2. Memory Management + +- **Minimize data transfer between CPU and GPU:** Keep data on the GPU as much as possible. +- **Pre-allocate output arrays:** Use `cuda.device_array` or `np.empty` to pre-allocate memory. +- **Use CUDA Device Arrays:** Operate directly on device arrays to avoid implicit data transfers. +- **Use `del` keyword to free unused arrays:** Use the `del` keyword to release large arrays or variables and free up memory after they're no longer needed. + +### 3.3. Bundle Size Optimization + +- Numba itself doesn't directly impact front-end bundle sizes (as it operates on the backend Python code). However, minimizing dependencies and using efficient numerical algorithms can indirectly reduce the overall application size. + +### 3.4. Lazy Loading + +- Numba JIT compilation performs lazy loading by default. Functions are compiled only when they are first called. +- This can be beneficial for large applications where not all functions need to be compiled immediately. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities + +- **Code Injection:** Avoid using `eval()` or `exec()` with user-supplied input, as this can lead to arbitrary code execution. +- **Integer Overflow:** Be mindful of potential integer overflows when performing arithmetic operations. +- **Buffer Overflow:** Prevent buffer overflows by validating array dimensions and sizes before performing operations. +- **Denial of Service (DoS):** Limit resource usage (e.g., memory, CPU time) to prevent DoS attacks. + +### 4.2. Input Validation + +- Validate all user-supplied input before passing it to Numba-compiled functions. +- Check data types, ranges, and sizes to prevent unexpected behavior or errors. +- Sanitize input to remove potentially malicious characters or code. + +### 4.3. Data Protection + +- Avoid storing sensitive data in plain text. Encrypt sensitive data at rest and in transit. +- Implement access controls to restrict access to sensitive data to authorized users only. + +### 4.4. Secure API Communication + +- Use HTTPS for all API communication to encrypt data in transit. +- Implement authentication and authorization mechanisms to verify the identity of clients and restrict access to resources. +- Protect against Cross-Site Scripting (XSS) and Cross-Site Request Forgery (CSRF) attacks. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +- Write unit tests for individual functions and classes to verify their correctness. +- Use the `unittest` or `pytest` frameworks for writing and running tests. +- Mock external dependencies to isolate the code being tested. +- Test different input values and edge cases to ensure robustness. + +### 5.2. Integration Testing + +- Write integration tests to verify the interaction between different components or modules. +- Test the integration between Numba-compiled functions and other parts of the application. +- Use realistic test data to simulate real-world scenarios. + +### 5.3. End-to-End Testing + +- Write end-to-end tests to verify the overall functionality of the application. +- Test the entire workflow from user input to output to ensure that all components are working correctly. +- Use automated testing tools to run tests regularly. + +### 5.4. Test Organization + +- Organize tests in a directory structure that mirrors the source code. +- Use descriptive test names to indicate the functionality being tested. +- Write clear and concise test cases that are easy to understand and maintain. + +### 5.5. Mocking and Stubbing + +- Use mocking and stubbing to replace external dependencies with controlled substitutes during testing. +- This allows you to isolate the code being tested and simulate different scenarios. +- Use libraries like `unittest.mock` or `pytest-mock` for mocking and stubbing. +- Make sure mocks respect type constraints used within the tested code. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +- **Incorrect data types:** Numba relies on type inference, so providing incorrect data types can lead to unexpected results or errors. Explicitly specify types when possible. +- **Unsupported Python features:** Numba does not support all Python features. Be aware of the limitations and use supported alternatives. +- **Not using no-python mode:** Forgetting to use `@njit` or `nopython=True` can result in significantly slower performance due to object mode compilation. +- **Ignoring performance profiling:** Failing to profile your code can lead to inefficient optimizations or missed opportunities for performance improvements. +- **Over-optimizing too early:** Focus on writing correct and readable code first, then optimize only when necessary. +- **Incompatible NumPy versions:** Make sure you are using a compatible NumPy version that Numba supports. + +### 6.2. Edge Cases + +- **Division by zero:** Handle potential division by zero errors gracefully. +- **NaN and Inf values:** Be aware of how NaN and Inf values can propagate through calculations. +- **Large arrays:** Be mindful of memory usage when working with large arrays, especially on GPUs. +- **Multithreading issues:** When using `parallel=True`, be aware of potential race conditions and data synchronization issues. + +### 6.3. Version-Specific Issues + +- Consult the Numba documentation and release notes for any version-specific issues or compatibility concerns. +- Pay attention to deprecation warnings and update your code accordingly. + +### 6.4. Compatibility Concerns + +- Ensure compatibility between Numba and other libraries you are using, such as NumPy, SciPy, and CUDA. +- Be aware of potential conflicts between different versions of these libraries. + +### 6.5. Debugging Strategies + +- Use the `print()` function or a debugger to inspect variables and execution flow. +- Check the Numba documentation and community forums for troubleshooting tips. +- Use the `numba.errors.DEBUG` environment variable to enable debug output from Numba. +- Use `numba --sysinfo` to get details about your system for inclusion in bug reports or when seeking help. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +- **IDE:** VS Code with the Python extension, PyCharm, or Jupyter Notebook. +- **Profiler:** `cProfile` or `line_profiler` for identifying performance bottlenecks. +- **Debugger:** `pdb` or an IDE-integrated debugger for stepping through code and inspecting variables. +- **Linter:** `flake8` or `pylint` for enforcing code style and identifying potential errors. +- **Formatter:** `black` or `autopep8` for automatically formatting code according to PEP 8. + +### 7.2. Build Configuration + +- Use `setup.py` or `pyproject.toml` for managing project dependencies and build configuration. +- Specify Numba as a dependency in your project's configuration file. +- Consider using a virtual environment (e.g., `venv` or `conda`) to isolate project dependencies. + +### 7.3. Linting and Formatting + +- Use `flake8` or `pylint` to enforce code style and identify potential errors. +- Configure the linter to check for Numba-specific best practices, such as using `@njit` whenever possible. +- Use `black` or `autopep8` to automatically format code according to PEP 8. + +### 7.4. Deployment Best Practices + +- Package your application and its dependencies into a self-contained executable or container. +- Use a deployment platform that supports Numba and its dependencies (e.g., AWS Lambda, Google Cloud Functions, Docker). +- Optimize your application for the target deployment environment. + +### 7.5. CI/CD Integration + +- Integrate Numba into your Continuous Integration/Continuous Delivery (CI/CD) pipeline. +- Run unit tests, integration tests, and end-to-end tests automatically on every commit. +- Use a CI/CD tool like GitHub Actions, GitLab CI, or Jenkins to automate the build, test, and deployment process. +- Perform performance testing as part of the CI/CD pipeline to detect regressions. \ No newline at end of file diff --git a/.cursor/rules/numpy.mdc b/.cursor/rules/numpy.mdc new file mode 100644 index 000000000..ff50a2a84 --- /dev/null +++ b/.cursor/rules/numpy.mdc @@ -0,0 +1,271 @@ +--- +description: This rule provides best practices for using NumPy in Python, covering coding standards, performance optimization, security, and testing strategies to enhance code quality and maintainability. +globs: *.py +--- +# NumPy Library Best Practices and Coding Standards + +This document outlines best practices for using the NumPy library in Python for scientific computing, data science, machine learning, and AI development. Adhering to these guidelines will improve code readability, maintainability, performance, security, and overall project success. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure + +* **`root_directory/`**: The project's root directory. + * **`data/`**: Stores datasets (e.g., CSV files, NumPy arrays in `.npy` format). Consider using subdirectories within `data/` to categorize datasets (e.g., `raw/`, `processed/`). Use version control (e.g., DVC) for larger datasets. + * **`notebooks/`**: Jupyter notebooks for exploration, prototyping, and EDA. Number notebooks sequentially (e.g., `01_data_loading.ipynb`, `02_feature_engineering.ipynb`). Keep notebooks clean and well-documented; move reusable code to modules. + * **`src/`** (or `package_name/`): Contains the Python source code. + * **`package_name/`**: The main package directory. Use your project's name as the package name. + * **`__init__.py`**: Marks the directory as a Python package. Can be empty or contain package-level initialization code. + * **`utils/`**: Utility functions and helper classes. Split into submodules if necessary (e.g., `utils/data_handling.py`, `utils/math_functions.py`). + * **`models/`**: Model definitions and training scripts (if applicable). Subdirectories can organize models further (e.g., `models/regression/`, `models/classification/`). + * **`preprocessing/`**: Data preprocessing functions and classes. + * **`visualization/`**: Functions for creating plots and visualizations. + * **`config.py`**: Configuration settings (e.g., file paths, hyperparameters). Use a library like `python-box` or `dynaconf` to manage configurations. + * **`main.py`**: The main entry point for the application (if applicable). Use `if __name__ == "__main__":` to control execution. + * **`tests/`**: Unit and integration tests. Mirror the `src/` structure. Use `pytest` or `unittest`. + * **`tests/utils/`**: Tests for the `utils/` module. + * **`tests/models/`**: Tests for the `models/` module. + * **`conftest.py`**: Configuration file for `pytest`. Can contain fixtures and hooks. + * **`docs/`**: Documentation for the project (e.g., using Sphinx). + * **`scripts/`**: Scripts for data downloading, processing, or model deployment. + * **`requirements.txt`**: Lists Python dependencies. Use `pip freeze > requirements.txt` to generate. + * **`.gitignore`**: Specifies files and directories to ignore in Git (e.g., `data/`, `notebooks/`, `__pycache__/`). + * **`README.md`**: Provides a high-level overview of the project. + * **`LICENSE`**: Specifies the license for the project. + * **`setup.py`** or **`pyproject.toml`**: Used for packaging and distribution. + +### 1.2. File Naming Conventions + +* **Python files**: Use lowercase with underscores (snake_case): `data_loader.py`, `feature_engineering.py`. +* **Class names**: Use CamelCase: `DataLoader`, `FeatureEngineer`. +* **Function and variable names**: Use snake_case: `load_data()`, `feature_names`. +* **Constants**: Use uppercase with underscores: `MAX_ITERATIONS`, `DEFAULT_LEARNING_RATE`. + +### 1.3. Module Organization + +* **Separate concerns**: Each module should have a single, well-defined purpose. +* **Avoid circular dependencies**: Design modules to minimize dependencies on each other. +* **Use relative imports**: Within a package, use relative imports to refer to other modules: `from . import utils`, `from ..models import BaseModel`. +* **Expose a clear API**: Each module should have a well-defined API (Application Programming Interface) of functions and classes that are intended for external use. Hide internal implementation details using leading underscores (e.g., `_helper_function()`). + +### 1.4. Component Architecture + +* **Layered architecture**: Divide the application into layers (e.g., data access, business logic, presentation) to promote separation of concerns. +* **Microservices**: For larger applications, consider breaking them down into smaller, independent microservices. +* **Design patterns**: Implement relevant design patterns to enhance flexibility and maintainability. + +### 1.5. Code Splitting + +* **Functions**: Break down large functions into smaller, more manageable functions. +* **Classes**: Use classes to encapsulate related data and behavior. +* **Modules**: Split code into multiple modules based on functionality. +* **Packages**: Organize modules into packages to create a hierarchical structure. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +* **Strategy**: Use a Strategy pattern to encapsulate different algorithms or approaches for a specific task. +* **Factory**: Employ a Factory pattern to create objects without specifying their concrete classes. +* **Observer**: Use an Observer pattern to notify dependent objects when the state of an object changes. + +### 2.2. Recommended Approaches + +* **Vectorization**: Leverage NumPy's vectorized operations whenever possible to avoid explicit loops. Vectorization significantly improves performance. +* **Broadcasting**: Understand and utilize NumPy's broadcasting rules to perform operations on arrays with different shapes. +* **Ufuncs (Universal Functions)**: Use NumPy's built-in ufuncs (e.g., `np.sin`, `np.exp`, `np.add`) for element-wise operations. Ufuncs are highly optimized. +* **Masking**: Use boolean masks to select and modify specific elements in arrays. +* **Views vs. Copies**: Be aware of the difference between views and copies when slicing and manipulating arrays. Modifying a view affects the original array. + +### 2.3. Anti-patterns and Code Smells + +* **Explicit loops**: Avoid explicit loops for element-wise operations. Use vectorized operations instead. +* **Unnecessary copies**: Avoid creating unnecessary copies of arrays. Use views when possible. +* **Modifying arrays in place**: Be careful when modifying arrays in place, as it can lead to unexpected side effects. +* **Ignoring broadcasting rules**: Not understanding broadcasting rules can lead to incorrect results or errors. +* **Hardcoding array shapes**: Avoid hardcoding array shapes. Use `array.shape` to get the shape dynamically. +* **Relying on mutable default arguments**: Avoid using mutable default arguments (e.g., lists, dictionaries) in function definitions. + +### 2.4. State Management + +* **Immutable data structures**: When appropriate, use immutable data structures to prevent accidental modification of data. Consider libraries like `namedtuple` or `dataclasses` (with `frozen=True`). +* **Encapsulation**: Encapsulate state within classes to control access and modification. +* **Dependency injection**: Use dependency injection to decouple components and make them more testable. + +### 2.5. Error Handling + +* **Exceptions**: Use exceptions to handle errors and unexpected conditions. Raise specific exceptions (e.g., `ValueError`, `TypeError`) when appropriate. +* **Assertions**: Use assertions to check for conditions that should always be true. Assertions are helpful for debugging and validating data. +* **Logging**: Use logging to record errors, warnings, and informational messages. Configure logging to write to a file or stream. +* **`np.errstate`**: Use `np.errstate` context manager to handle floating-point exceptions (e.g., division by zero, overflow). You can configure how NumPy handles these exceptions (e.g., ignore, warn, raise). + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +* **Vectorization**: As mentioned earlier, prioritize vectorized operations over explicit loops. +* **Memory Alignment**: NumPy arrays are typically aligned in memory, which can improve performance. Ensure that your data is aligned correctly. +* **Data Types**: Choose the smallest possible data type that can represent your data. Using smaller data types reduces memory usage and improves performance. For example, use `np.int8` instead of `np.int64` if the values are within the range of `np.int8`. +* **`np.einsum`**: Use `np.einsum` (Einstein summation) for complex array operations. `np.einsum` can often be more efficient than explicit loops or other NumPy functions. +* **Numba**: Use Numba to JIT (Just-In-Time) compile NumPy code. Numba can significantly improve the performance of computationally intensive code. +* **Cython**: Use Cython to write NumPy code in C. Cython allows you to take advantage of C's performance while still using Python's syntax. +* **BLAS/LAPACK**: NumPy relies on BLAS (Basic Linear Algebra Subprograms) and LAPACK (Linear Algebra Package) libraries for linear algebra operations. Ensure that you are using an optimized BLAS/LAPACK implementation (e.g., OpenBLAS, MKL). +* **`np.fft`**: For FFT (Fast Fourier Transform) operations, use the functions provided in NumPy's `np.fft` module. They're usually highly optimized. + +### 3.2. Memory Management + +* **Avoid creating large temporary arrays**: Minimize the creation of large temporary arrays, especially within loops. Use in-place operations when possible. +* **`np.empty`**: Use `np.empty` to create uninitialized arrays when you don't need to initialize the values immediately. This can be faster than `np.zeros` or `np.ones`. +* **`np.memmap`**: Use `np.memmap` to create memory-mapped arrays. Memory-mapped arrays allow you to work with large datasets that don't fit in memory. +* **Garbage collection**: Be mindful of Python's garbage collection. Explicitly delete large arrays when they are no longer needed to free up memory: `del my_array`. + +### 3.3. Rendering Optimization (If Applicable) + +* This is mainly relevant when NumPy arrays are used for image processing or other visualization tasks. Libraries like Matplotlib, Seaborn, or specialized rendering engines may offer specific optimizations for handling NumPy arrays. + +### 3.4. Bundle Size Optimization (If Applicable) + +* If you are deploying a web application or other application that includes NumPy, consider using tree shaking to remove unused code. Tools like Webpack or Parcel can help with tree shaking. + +### 3.5. Lazy Loading + +* If you are working with very large datasets, use lazy loading to load data only when it is needed. Libraries like Dask or Apache Arrow can help with lazy loading. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities + +* **Arbitrary code execution**: Avoid using `np.fromstring` or `np.frombuffer` with untrusted input, as this can lead to arbitrary code execution. +* **Denial of service**: Be careful when using NumPy functions with user-supplied input, as this can lead to denial-of-service attacks. Validate input to prevent excessively large array allocations or computationally intensive operations. +* **Integer overflow**: Be aware of integer overflow issues when performing arithmetic operations on large arrays. Use appropriate data types to prevent overflow. + +### 4.2. Input Validation + +* **Data type validation**: Ensure that the input data has the expected data type. +* **Shape validation**: Ensure that the input data has the expected shape. +* **Range validation**: Ensure that the input data falls within the expected range. +* **Sanitize input**: Sanitize input data to prevent injection attacks. +* **Use `np.asarray`**: When receiving data from external sources, use `np.asarray` to convert it to a NumPy array. This ensures that you are working with a NumPy array and not some other type of object that might have unexpected behavior. + +### 4.3. Authentication and Authorization (If Applicable) + +* NumPy itself doesn't handle authentication or authorization. If your application requires these features, use appropriate authentication and authorization mechanisms (e.g., OAuth, JWT). + +### 4.4. Data Protection + +* **Encryption**: Encrypt sensitive data at rest and in transit. +* **Access control**: Restrict access to data based on user roles and permissions. +* **Data masking**: Mask sensitive data to protect it from unauthorized access. +* **Regular Security Audits**: Conduct regular security audits to identify and address potential vulnerabilities. + +### 4.5. Secure API Communication (If Applicable) + +* Use HTTPS for all API communication. +* Validate all API requests. +* Use rate limiting to prevent denial-of-service attacks. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +* **Test individual functions and classes**: Unit tests should focus on testing individual functions and classes in isolation. +* **Use `pytest` or `unittest`**: Use a testing framework like `pytest` or `unittest` to write and run unit tests. +* **Test edge cases**: Test edge cases and boundary conditions to ensure that your code handles them correctly. +* **Test for exceptions**: Test that your code raises the correct exceptions when errors occur. +* **Use parametrization**: Use parametrization to run the same test with multiple sets of inputs. +* **Assert almost equal**: Use `np.testing.assert_allclose` instead of `assert a == b` when comparing floating-point numbers. This accounts for potential floating-point precision errors. + +### 5.2. Integration Testing + +* **Test interactions between components**: Integration tests should focus on testing the interactions between different components of your application. +* **Use mock objects**: Use mock objects to isolate components during integration testing. + +### 5.3. End-to-End Testing (If Applicable) + +* **Test the entire application flow**: End-to-end tests should focus on testing the entire application flow from start to finish. + +### 5.4. Test Organization + +* **Mirror the source code structure**: Organize your tests in a directory structure that mirrors the source code structure. +* **Use descriptive test names**: Use descriptive test names that clearly indicate what the test is testing. +* **Keep tests small and focused**: Keep tests small and focused to make them easier to understand and maintain. + +### 5.5. Mocking and Stubbing + +* **Use mock objects to isolate components**: Use mock objects to isolate components during testing. +* **Use `unittest.mock` or `pytest-mock`**: Use a mocking library like `unittest.mock` or `pytest-mock` to create mock objects. +* **Mock external dependencies**: Mock external dependencies (e.g., databases, APIs) to avoid relying on them during testing. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +* **Incorrect array indexing**: NumPy uses 0-based indexing, which can be confusing for developers who are used to other languages. +* **Incorrect array slicing**: Be careful when slicing arrays, as this can create views or copies depending on the slicing operation. +* **Incorrect broadcasting**: Not understanding broadcasting rules can lead to incorrect results or errors. +* **Modifying views**: Modifying a view affects the original array, which can lead to unexpected side effects. +* **Ignoring data types**: Not specifying the correct data type can lead to integer overflow or other issues. + +### 6.2. Edge Cases + +* **Empty arrays**: Be careful when working with empty arrays, as they can have unexpected behavior. +* **Arrays with NaN or Inf values**: Be aware that arrays can contain NaN (Not a Number) or Inf (Infinity) values, which can affect calculations. +* **Arrays with mixed data types**: NumPy arrays should typically have a single data type. Be cautious when working with arrays that have mixed data types (e.g., object arrays). + +### 6.3. Version-Specific Issues + +* Consult the NumPy release notes for information on version-specific issues and changes. + +### 6.4. Compatibility Concerns + +* **Python version**: Ensure that your code is compatible with the Python version you are using. +* **Other libraries**: Be aware of compatibility issues between NumPy and other libraries. +* **Operating system**: Be aware of potential differences in behavior across different operating systems. + +### 6.5. Debugging Strategies + +* **Print statements**: Use print statements to inspect the values of variables and arrays. +* **Debuggers**: Use a debugger (e.g., pdb, ipdb) to step through your code and inspect the state of the program. +* **Logging**: Use logging to record errors, warnings, and informational messages. +* **`np.seterr`**: Use `np.seterr` to configure how NumPy handles floating-point exceptions. +* **`np.info`**: Use `np.info` to get information about NumPy objects. +* **Visual Inspection**: Use visualization tools (Matplotlib, Seaborn, etc.) to visually inspect data and identify potential problems. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +* **IDE**: Use an IDE (Integrated Development Environment) like VS Code, PyCharm, or Spyder. +* **Jupyter Notebook**: Use Jupyter Notebook for exploration, prototyping, and EDA. +* **IPython**: Use IPython for interactive computing. +* **Debugging Tools**: Utilize debuggers like pdb or IDE-integrated debuggers for stepping through code and inspecting variables. + +### 7.2. Build Configuration + +* **`setup.py` or `pyproject.toml`**: Use `setup.py` or `pyproject.toml` to configure the build process. +* **`requirements.txt`**: Use `requirements.txt` to specify dependencies. +* **Virtual environments**: Use virtual environments to isolate dependencies. +* **Conda**: Consider using Conda for environment and package management, particularly for scientific computing workflows. + +### 7.3. Linting and Formatting + +* **PEP 8**: Follow PEP 8 style guidelines for Python code. +* **Linters**: Use a linter (e.g., pylint, flake8) to check for style violations and potential errors. +* **Formatters**: Use a formatter (e.g., black, autopep8) to automatically format your code. +* **Pre-commit hooks**: Use pre-commit hooks to run linters and formatters before committing code. + +### 7.4. Deployment + +* **Containerization (Docker)**: Use Docker to create a containerized environment for your application. This helps ensure consistency across different environments. +* **Cloud platforms**: Deploy your application to a cloud platform (e.g., AWS, Azure, GCP). +* **Serverless functions**: Consider using serverless functions (e.g., AWS Lambda, Azure Functions, Google Cloud Functions) for simple applications. +* **Model serving frameworks**: If deploying machine learning models, use a model serving framework like TensorFlow Serving or TorchServe. + +### 7.5. CI/CD Integration + +* **Continuous Integration (CI)**: Use a CI system (e.g., Jenkins, Travis CI, CircleCI, GitHub Actions) to automatically build and test your code when changes are made. +* **Continuous Delivery (CD)**: Use a CD system to automatically deploy your code to production after it has been tested. +* **Automated Testing**: Integrate automated tests into your CI/CD pipeline to ensure code quality and prevent regressions. +* **Infrastructure as Code (IaC)**: Define your infrastructure using code (e.g., Terraform, CloudFormation) to automate the provisioning and management of your environment. + +By following these best practices, you can write high-quality, maintainable, and performant NumPy code. \ No newline at end of file diff --git a/.cursor/rules/nuxt.mdc b/.cursor/rules/nuxt.mdc new file mode 100644 index 000000000..317a90306 --- /dev/null +++ b/.cursor/rules/nuxt.mdc @@ -0,0 +1,201 @@ +--- +description: This rule provides comprehensive best practices and coding standards for Nuxt.js projects, covering code organization, performance, security, testing, and common pitfalls. It aims to ensure maintainable, scalable, and secure Nuxt.js applications. +globs: *.vue,*.js,*.ts,*.mjs,*.mts,*.jsx,*.tsx,*.config.js,*.config.ts +--- +- **Enable ESLint support:** Use the `@nuxt/eslint` module for project-aware ESLint configuration. This ensures code quality and consistency. + - `npx nuxi module add eslint` to add the module. + - Customize the generated `eslint.config.mjs` file as needed. +- **Adopt Nuxt.js Modules:** Leverage Nuxt.js modules to encapsulate functionalities and maintain a clean codebase. Explore existing modules before implementing custom solutions (e.g., `@nuxt/auth` for server-side authentication). +- **Convention over Configuration:** Adhere to Nuxt.js conventions to simplify development and collaboration. Avoid deviating from conventions unless absolutely necessary. +- **Efficiently Utilize Nuxt Layouts:** Create reusable layouts for components shared across multiple pages to ensure consistency and save development time. Layouts are located in the `layouts/` directory. +- **Manage State with Pinia:** Use Pinia for state management. Organize store modules based on features or functionalities for better maintainability. +- **Divide Pages into Components:** Break down pages into small, reusable components to enhance maintainability, testability, and reusability. Each component should have a single responsibility. +- **Leverage Nuxt Plugins Wisely:** Use Nuxt plugins to run code before Vue.js initializes or add global functionality. Be mindful of plugin performance impact. Plugins are located in the `plugins/` directory. +- **Optimize for SEO and Performance:** Utilize Nuxt.js's server-side rendering (SSR) for SEO. Implement lazy loading for images and optimize assets to minimize initial load time. Use tools like Lighthouse to identify performance bottlenecks. +- **Implement Error Handling and Validation:** Implement robust error handling and validation mechanisms to provide a seamless user experience. Use Nuxt.js middleware to intercept requests and responses for error handling and data validation. +- **Document Your Code:** Provide clear and concise documentation for components, modules, and custom functions using tools like JSDoc. +- **Embrace Testing:** Write unit tests, integration tests, and end-to-end tests using tools like Jest, Vue Test Utils and Vitest. + +## 1. Code Organization and Structure: + +- **Directory Structure:** + - `components/`: Reusable Vue components. + - `composables/`: Reusable composable functions. + - `layouts/`: Application layouts. + - `middleware/`: Route middleware. + - `pages/`: Application pages (route definitions). + - `plugins/`: Nuxt.js plugins. + - `server/`: API routes and server-side logic. + - `static/`: Static assets (e.g., images, fonts). + - `store/`: Pinia stores (optional, but recommended). + - `utils/`: Utility functions. +- **File Naming Conventions:** + - Components: `PascalCase.vue` (e.g., `MyComponent.vue`) + - Composables: `usePascalCase.js` or `usePascalCase.ts` (e.g., `useCounter.js`) + - Layouts: `kebab-case.vue` (e.g., `default.vue` or `custom-layout.vue`) + - Pages: `kebab-case.vue` (e.g., `index.vue`, `about.vue`, `product-details.vue`) + - Plugins: `kebab-case.js` or `kebab-case.ts` (e.g., `analytics.js`) + - Stores: `kebab-case.js` or `kebab-case.ts` (e.g., `user-store.js`) + - Utility functions: `camelCase.js` or `camelCase.ts` (e.g., `formatDate.js`) +- **Module Organization:** + - Group related functionalities into separate modules. + - Use the `@nuxt/modules` array in `nuxt.config.js` or `nuxt.config.ts` to register modules. + - Create custom modules to encapsulate complex logic. +- **Component Architecture:** + - Favor composition over inheritance. + - Use functional components for simple UI elements. + - Design components for reusability and testability. + - Consider using slots for flexible component composition. +- **Code Splitting:** + - Utilize dynamic imports for route-based code splitting. + - Split large components into smaller chunks using `import()`. + - Analyze bundle size with tools like Webpack Bundle Analyzer. + +## 2. Common Patterns and Anti-patterns: + +- **Design Patterns:** + - **Composition API:** Use the Composition API for organizing component logic. + - **Store Pattern (Pinia):** Implement a centralized state management system with Pinia. + - **Middleware Pattern:** Use middleware for authentication, authorization, and data validation. + - **Plugin Pattern:** Create plugins for global functionality and third-party library integrations. +- **Recommended Approaches:** + - **API Communication:** Use the `useFetch` or `useAsyncData` composables for API calls within components. + - **Form Handling:** Utilize Vue's built-in form handling features with `v-model` and validation libraries like VeeValidate. + - **Authentication:** Implement a secure authentication flow using a library like `@nuxt/auth` or a custom solution. + - **Authorization:** Implement role-based access control (RBAC) using middleware and Pinia stores. +- **Anti-patterns:** + - **Mutating props directly:** Avoid modifying parent component data directly from child components. Use `emit` instead. + - **Overusing global state:** Limit the use of global state to essential application data. Consider component-level state for local data. + - **Ignoring error handling:** Always handle potential errors in API calls and other asynchronous operations. + - **Writing overly complex components:** Break down large components into smaller, more manageable pieces. +- **State Management Best Practices:** + - **Single Source of Truth:** Maintain a single, consistent source of truth for application state in Pinia stores. + - **Immutability:** Treat state as immutable. Use functions to update the store rather than directly manipulating data. + - **Clear Naming Conventions:** Use descriptive names for store modules, actions, and mutations. + - **Modularization:** Divide stores into modules based on features or functionalities. +- **Error Handling Patterns:** + - **Centralized Error Handling:** Implement a global error handler to catch unhandled exceptions. + - **Error Boundaries:** Use error boundaries to isolate component failures and prevent cascading errors. + - **User-Friendly Error Messages:** Provide clear and helpful error messages to users. + +## 3. Performance Considerations: + +- **Optimization Techniques:** + - **Lazy Loading:** Implement lazy loading for images, components, and routes. + - **Code Splitting:** Split the application into smaller chunks for faster initial load times. + - **Tree Shaking:** Remove unused code during the build process. + - **Caching:** Cache API responses and static assets to reduce server load. + - **Image Optimization:** Optimize images using tools like `nuxt/image`. Use appropriate image formats (WebP). Resize the image to appropriate size. Consider using a CDN for image delivery. +- **Memory Management:** + - **Avoid Memory Leaks:** Clean up event listeners and timers when components are unmounted. + - **Use Weak References:** Use weak references for DOM elements when possible. + - **Minimize Object Creation:** Avoid creating unnecessary objects and arrays. +- **Rendering Optimization:** + - **Virtualization:** Use virtualization for large lists to improve rendering performance. + - **Memoization:** Memoize expensive calculations to avoid redundant computations. Use `computed` properties effectively to avoid unnecessary re-renders. + - **Debouncing and Throttling:** Use debouncing and throttling for event handlers to reduce the number of function calls. +- **Bundle Size Optimization:** + - **Analyze Bundle Size:** Use Webpack Bundle Analyzer to identify large dependencies. + - **Remove Unused Dependencies:** Remove unused dependencies to reduce bundle size. + - **Use Smaller Alternatives:** Consider using smaller alternatives to large libraries. + - **Optimize Dependencies:** Review dependencies and ensure you're using the most efficient ones. +- **Lazy Loading Strategies:** + - **Route-based Lazy Loading:** Load components only when the corresponding route is accessed. + - **Component-based Lazy Loading:** Load components only when they are visible in the viewport. + +## 4. Security Best Practices: + +- **Common Vulnerabilities:** + - **Cross-Site Scripting (XSS):** Prevent XSS attacks by properly sanitizing user input and using Vue's built-in HTML escaping. + - **Cross-Site Request Forgery (CSRF):** Protect against CSRF attacks by implementing CSRF tokens. + - **SQL Injection:** Avoid raw SQL queries. Use an ORM (Object-Relational Mapper) to prevent SQL injection. + - **Authentication and Authorization Flaws:** Implement secure authentication and authorization mechanisms. + - **Insecure Direct Object References (IDOR):** Implement proper access control to prevent unauthorized access to resources. +- **Input Validation:** + - **Server-Side Validation:** Always validate user input on the server-side. + - **Client-Side Validation:** Provide client-side validation for a better user experience (but don't rely on it as the sole source of validation). + - **Sanitize Input:** Sanitize user input to remove potentially harmful characters. +- **Authentication and Authorization Patterns:** + - **JWT (JSON Web Tokens):** Use JWT for authentication and authorization. + - **OAuth 2.0:** Implement OAuth 2.0 for third-party authentication. + - **Role-Based Access Control (RBAC):** Implement RBAC to control access to resources based on user roles. +- **Data Protection Strategies:** + - **Encryption:** Encrypt sensitive data at rest and in transit. + - **Hashing:** Hash passwords and other sensitive data using strong hashing algorithms. + - **Data Masking:** Mask sensitive data in logs and other non-production environments. +- **Secure API Communication:** + - **HTTPS:** Always use HTTPS for API communication. + - **API Rate Limiting:** Implement API rate limiting to prevent abuse. + - **Authentication and Authorization:** Require authentication and authorization for all API endpoints. + +## 5. Testing Approaches: + +- **Unit Testing:** + - **Test Individual Components:** Test individual components in isolation. + - **Mock Dependencies:** Mock external dependencies to isolate components during testing. + - **Verify Component Behavior:** Verify that components render correctly and behave as expected. +- **Integration Testing:** + - **Test Component Interactions:** Test the interactions between components. + - **Test Data Flow:** Test the data flow between components and stores. + - **Test API Integrations:** Test the integration with external APIs. +- **End-to-End Testing:** + - **Simulate User Interactions:** Simulate user interactions to test the application's functionality. + - **Test the Entire Application Flow:** Test the entire application flow from start to finish. + - **Use a Browser Automation Tool:** Use a browser automation tool like Cypress or Playwright. +- **Test Organization:** + - **Organize Tests by Feature:** Organize tests by feature or functionality. + - **Use Descriptive Test Names:** Use descriptive test names to clearly indicate what each test is testing. + - **Keep Tests Isolated:** Keep tests isolated from each other to avoid interference. +- **Mocking and Stubbing:** + - **Use Mock Objects:** Use mock objects to replace external dependencies during testing. + - **Use Stubs:** Use stubs to replace complex functions with simplified versions. + - **Avoid Over-Mocking:** Avoid mocking too much code, as this can make tests less effective. + +## 6. Common Pitfalls and Gotchas: + +- **Frequent Mistakes:** + - **Incorrect `this` Context:** Be mindful of the `this` context in Vue components and use arrow functions or `bind` to maintain the correct context. + - **Asynchronous Data Handling:** Properly handle asynchronous data loading with `async/await` or Promises. + - **Forgetting to Unsubscribe:** Unsubscribe from event listeners and timers when components are unmounted to prevent memory leaks. + - **Overusing `forceUpdate`:** Avoid using `forceUpdate` unless absolutely necessary, as it can negatively impact performance. +- **Edge Cases:** + - **Server-Side Rendering (SSR):** Be aware of the differences between client-side and server-side rendering. + - **Browser Compatibility:** Test the application in different browsers to ensure compatibility. + - **Accessibility:** Consider accessibility when designing and developing the application. +- **Version-Specific Issues:** + - **Nuxt 2 vs Nuxt 3:** Be aware of the differences between Nuxt 2 and Nuxt 3. + - **Vue 2 vs Vue 3:** Be aware of the differences between Vue 2 and Vue 3. + - **Dependency Updates:** Carefully review dependency updates for potential breaking changes. +- **Compatibility Concerns:** + - **Browser Support:** Ensure compatibility with the target browsers. + - **Device Compatibility:** Test the application on different devices. + - **Operating System Compatibility:** Ensure compatibility with the target operating systems. +- **Debugging Strategies:** + - **Use Browser Developer Tools:** Use browser developer tools to inspect the application's state and network activity. + - **Use Vue Devtools:** Use Vue Devtools to inspect Vue components and data. + - **Use Logging:** Use logging to track the application's behavior. + +## 7. Tooling and Environment: + +- **Recommended Development Tools:** + - **VS Code:** Visual Studio Code is a popular code editor with excellent Vue.js support. + - **Vue Devtools:** Vue Devtools is a browser extension that provides debugging tools for Vue.js applications. + - **ESLint:** ESLint is a linter that enforces coding standards. + - **Prettier:** Prettier is a code formatter that automatically formats code. +- **Build Configuration:** + - **`nuxt.config.js` or `nuxt.config.ts`:** Configure the application's build settings in `nuxt.config.js` or `nuxt.config.ts`. + - **Webpack:** Nuxt uses Webpack to bundle the application. + - **Vite:** Nuxt 3 uses Vite to bundle the application by default, providing faster build and development times. +- **Linting and Formatting:** + - **ESLint:** Use ESLint to enforce coding standards. + - **Prettier:** Use Prettier to automatically format code. + - **Husky:** Use Husky to run linters and formatters before commits. +- **Deployment Best Practices:** + - **Server-Side Rendering (SSR):** Deploy the application to a server that supports SSR. + - **Static Site Generation (SSG):** Generate a static site for content-heavy applications. + - **CDN:** Use a CDN to deliver static assets. +- **CI/CD Integration:** + - **Continuous Integration (CI):** Use a CI tool like Jenkins, GitLab CI, or GitHub Actions to automate the build and testing process. + - **Continuous Deployment (CD):** Use a CD tool to automate the deployment process. + +By following these best practices, you can build robust, maintainable, and scalable Nuxt.js applications. \ No newline at end of file diff --git a/.cursor/rules/openai.mdc b/.cursor/rules/openai.mdc new file mode 100644 index 000000000..4818b10ad --- /dev/null +++ b/.cursor/rules/openai.mdc @@ -0,0 +1,246 @@ +--- +description: Comprehensive best practices and coding standards for projects using the openai library, covering code structure, performance, security, and common pitfalls. +globs: *.py +--- +# openai Library Best Practices and Coding Standards + +This document outlines best practices and coding standards for developing applications using the `openai` library. Following these guidelines will lead to more maintainable, performant, and secure code. + +## Library Information: +- Name: openai +- Tags: ai, ml, llm, api + +## 1. Code Organization and Structure + +### 1.1 Directory Structure Best Practices + +Adopt a clear and consistent directory structure to improve code organization and maintainability. Here's a recommended structure for projects using the openai library: + + +project_root/ +├── src/ # Source code directory +│ ├── models/ # Definitions for your models (e.g., data classes, schemas) +│ ├── services/ # Service layer for interacting with the OpenAI API +│ │ ├── openai_service.py # Encapsulates OpenAI API calls +│ ├── utils/ # Utility functions +│ ├── main.py # Entry point of your application +├── tests/ # Tests directory +│ ├── unit/ # Unit tests +│ ├── integration/ # Integration tests +│ ├── conftest.py # Pytest configuration file +├── data/ # Data storage (e.g., prompts, training data) +├── docs/ # Documentation +├── .env # Environment variables +├── requirements.txt # Dependencies +├── README.md # Project README + + +### 1.2 File Naming Conventions + +- Use descriptive and consistent file names. +- Python files should use snake_case (e.g., `openai_service.py`). +- Class names should use CamelCase (e.g., `OpenAIService`). +- Variable names should use snake_case (e.g., `api_key`). + +### 1.3 Module Organization + +- Group related functionalities into modules. +- Avoid circular dependencies between modules. +- Use clear and concise module names. +- Use `__init__.py` files to define packages and control namespace. + +### 1.4 Component Architecture + +Consider using a layered architecture to separate concerns: + +- **Presentation Layer:** Handles user interface or external API interactions. +- **Service Layer:** Encapsulates business logic and interacts with the OpenAI API. +- **Data Access Layer:** Handles data persistence and retrieval. + +This separation makes testing and maintenance easier. + +### 1.5 Code Splitting Strategies + +- Split large files into smaller, more manageable modules based on functionality. +- Use abstract base classes and interfaces to define contracts between components. +- Apply the Single Responsibility Principle (SRP) to classes and functions. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns + +- **Factory Pattern:** Use a factory to create OpenAI API client instances with different configurations. +- **Strategy Pattern:** Implement different prompt strategies based on the task. +- **Decorator Pattern:** Add logging, caching, or rate limiting to OpenAI API calls. + +### 2.2 Recommended Approaches + +- **Prompt Engineering:** Follow best practices for prompt design. Place clear instructions at the beginning of prompts, be specific, and use examples. +- **Configuration:** Store API keys and other sensitive information in environment variables using a library like `python-dotenv`. +- **Asynchronous Calls:** Use `asyncio` and `aiohttp` for non-blocking API calls to improve performance. +- **Retries and Exponential Backoff:** Implement retry mechanisms with exponential backoff to handle transient API errors. + +### 2.3 Anti-patterns + +- **Hardcoding API Keys:** Never hardcode API keys directly into your code. Always use environment variables. +- **Ignoring Rate Limits:** Implement rate limiting to avoid exceeding OpenAI API limits. +- **Lack of Error Handling:** Always handle API errors gracefully and provide informative error messages. +- **Overly Complex Prompts:** Keep prompts simple and focused. Break down complex tasks into smaller steps. +- **Mixing Concerns:** Avoid mixing presentation, business logic, and data access in the same component. + +### 2.4 State Management + +- Use appropriate data structures to manage the state of your OpenAI interactions. +- Consider using a state management library if your application has complex state requirements. +- Avoid storing sensitive information in application state. + +### 2.5 Error Handling + +- Use `try-except` blocks to catch potential exceptions. +- Log errors with sufficient context for debugging. +- Implement custom exception classes for specific error conditions. +- Handle rate limit errors and implement retry logic. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +- **Caching:** Cache API responses to reduce the number of API calls. +- **Batching:** Batch multiple API requests into a single request when possible. +- **Asynchronous Operations:** Use asynchronous programming to avoid blocking the main thread. +- **Token Optimization:** Reduce the number of tokens in your prompts to lower costs and improve response times. + +### 3.2 Memory Management + +- Be mindful of the size of your prompts and responses, especially when working with large language models. +- Use generators to process large datasets in chunks. +- Clean up resources (e.g., file handles, network connections) promptly. + +### 3.3 Rendering Optimization (If Applicable) + +- If your application involves rendering OpenAI-generated content, optimize the rendering process to minimize latency. + +### 3.4 Bundle Size Optimization (If Applicable) + +- For web applications, minimize bundle size by using tree shaking and code splitting. + +### 3.5 Lazy Loading + +- Use lazy loading to load modules or data only when they are needed. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities + +- **API Key Exposure:** Protect your OpenAI API key. Never commit it to version control or share it publicly. +- **Prompt Injection:** Validate and sanitize user inputs to prevent prompt injection attacks. +- **Data Leakage:** Avoid exposing sensitive data in prompts or API responses. + +### 4.2 Input Validation + +- Validate all user inputs to prevent malicious or unexpected data from being sent to the OpenAI API. +- Sanitize inputs to remove potentially harmful characters or code. + +### 4.3 Authentication and Authorization + +- Implement authentication and authorization mechanisms to protect your application and data. +- Use secure storage for API keys and other sensitive information. + +### 4.4 Data Protection + +- Encrypt sensitive data at rest and in transit. +- Follow data privacy regulations (e.g., GDPR, CCPA). + +### 4.5 Secure API Communication + +- Use HTTPS to encrypt communication with the OpenAI API. +- Verify the authenticity of the OpenAI API server using SSL certificates. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +- Write unit tests for individual components to ensure they function correctly in isolation. +- Use mocking and stubbing to isolate components from external dependencies (e.g., the OpenAI API). + +### 5.2 Integration Testing + +- Write integration tests to verify that different components work together correctly. +- Test the interaction between your application and the OpenAI API. + +### 5.3 End-to-End Testing + +- Write end-to-end tests to simulate user interactions and verify that the entire application works as expected. + +### 5.4 Test Organization + +- Organize your tests into a clear and consistent directory structure. +- Use descriptive test names. +- Follow a consistent testing style. + +### 5.5 Mocking and Stubbing + +- Use mocking libraries like `unittest.mock` or `pytest-mock` to mock the OpenAI API. +- Create stubs for API responses to control the behavior of the API during testing. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +- **Not handling API errors:** Implement proper error handling for OpenAI API calls. +- **Exceeding rate limits:** Implement rate limiting and exponential backoff to avoid exceeding API limits. +- **Incorrect prompt formatting:** Follow OpenAI's prompt engineering guidelines to optimize model performance. +- **Not validating inputs:** Validate user inputs to prevent prompt injection attacks and unexpected behavior. + +### 6.2 Edge Cases + +- **Handling very long prompts:** Be aware of token limits and consider splitting long prompts into smaller chunks. +- **Dealing with ambiguous or unclear instructions:** Craft prompts carefully to provide clear and specific instructions. +- **Handling unexpected API responses:** Implement robust error handling to deal with unexpected API responses. + +### 6.3 Version-Specific Issues + +- Be aware of changes between different versions of the `openai` library. +- Consult the release notes and migration guides when upgrading to a new version. + +### 6.4 Compatibility Concerns + +- Ensure compatibility between the `openai` library and other libraries used in your project. +- Test your application thoroughly after upgrading any dependencies. + +### 6.5 Debugging Strategies + +- Use logging to track the flow of your application and identify potential issues. +- Use a debugger to step through your code and inspect variables. +- Use unit tests to isolate and debug individual components. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +- **IDE:** VS Code, PyCharm +- **Virtual Environment Manager:** venv, conda, pipenv +- **Package Manager:** pip, poetry +- **Testing Framework:** pytest, unittest +- **Linting and Formatting:** pylint, flake8, black + +### 7.2 Build Configuration + +- Use a `requirements.txt` or `pyproject.toml` file to manage dependencies. +- Use a build system like `setuptools` or `poetry` to package your application. + +### 7.3 Linting and Formatting + +- Use a linter like `pylint` or `flake8` to enforce coding style guidelines. +- Use a formatter like `black` to automatically format your code. + +### 7.4 Deployment Best Practices + +- Use a containerization technology like Docker to package your application and its dependencies. +- Deploy your application to a cloud platform like AWS, Azure, or Google Cloud. +- Use a process manager like `systemd` or `supervisor` to manage your application. + +### 7.5 CI/CD Integration + +- Use a CI/CD pipeline to automate the build, test, and deployment process. +- Integrate your tests into the CI/CD pipeline to ensure that all changes are thoroughly tested before being deployed. \ No newline at end of file diff --git a/.cursor/rules/opencv-python.mdc b/.cursor/rules/opencv-python.mdc new file mode 100644 index 000000000..6c0cff574 --- /dev/null +++ b/.cursor/rules/opencv-python.mdc @@ -0,0 +1,199 @@ +--- +description: This rule outlines best practices for developing with the opencv-python library, focusing on code organization, performance, security, testing, and common pitfalls. It provides comprehensive guidelines for efficient and maintainable opencv-python projects. +globs: *.py +--- +- Always use UV or Pipenv when installing dependencies for consistent environments. +- Always specify Python 3.10 or higher, as older versions may lack feature parity and security updates. +- Use type hints extensively for improved code readability and maintainability. + +# Library Information: +- Name: opencv-python +- Tags: python, image-processing, computer-vision, opencv + +## 1. Code Organization and Structure + +- **Directory Structure:** + - + project_root/ + ├── data/ # Store image, video, and other data files. + ├── src/ # Source code directory. + │ ├── modules/ # Reusable modules and utility functions. + │ │ ├── image_processing/ # Specific image processing functions. + │ │ │ ├── __init__.py + │ │ │ ├── filters.py # Implement filters as functions or classes + │ │ │ ├── transforms.py # Implement image transforms as functions or classes + │ │ ├── utils.py # Utility functions for data loading, etc. + │ ├── main.py # Entry point of your application. + ├── tests/ # Test suite. + │ ├── unit/ # Unit tests for individual components. + │ ├── integration/ # Integration tests for combined components. + ├── models/ # Saved model files + ├── notebooks/ # Jupyter notebooks for experimentation. + ├── requirements.txt # Project dependencies. + ├── pyproject.toml # Project metadata and build configuration. + └── README.md # Project documentation. + + +- **File Naming Conventions:** + - Use descriptive and consistent naming. + - `module_name.py` (e.g., `image_utils.py`, `feature_extraction.py`). + - Class names: `PascalCase` (e.g., `ImageProcessor`, `FeatureDetector`). + - Function names: `snake_case` (e.g., `load_image`, `apply_filter`). + +- **Module Organization:** + - Group related functions and classes into modules within the `src/modules/` directory. + - Use `__init__.py` files to make directories packages. + - Employ clear and concise module names. + +- **Component Architecture:** + - Follow a layered architecture: + - **Data Access Layer:** Handles image loading, saving, and data source interactions. + - **Processing Layer:** Implements image processing algorithms and feature extraction. + - **Application Layer:** Orchestrates the processing and presents the results. + +- **Code Splitting:** + - Decompose complex functions into smaller, well-defined units. + - Create separate modules for distinct functionalities (e.g., filtering, feature detection, object tracking). + - Utilize classes to encapsulate related state and behavior. + +## 2. Common Patterns and Anti-patterns + +- **Design Patterns:** + - **Strategy Pattern:** To swap image processing algorithms at runtime. + - **Factory Pattern:** To create different image processing objects based on configuration. + - **Observer Pattern:** To notify subscribers of image processing events. + +- **Recommended Approaches:** + - **Image Loading:** Use `cv2.imread()` for loading images. Handle potential file errors gracefully. + - **Image Display:** Use `cv2.imshow()` for displaying images, and remember to call `cv2.waitKey()` to prevent freezing. + - **Video Processing:** Use `cv2.VideoCapture()` to capture video from a file or camera. + - **Iterating Pixels:** Access pixel values directly using NumPy array indexing for performance. Avoid slow Python loops when possible. + +- **Anti-patterns and Code Smells:** + - **Deeply Nested Loops:** Indicate inefficient pixel-level operations. Consider vectorization using NumPy. + - **Global Variables:** Lead to unpredictable state. Encapsulate state within classes. + - **Hardcoded Paths:** Make code inflexible. Use relative paths and configurable settings. + - **Ignoring Errors:** Can lead to unexpected crashes. Implement robust error handling. + +- **State Management:** + - Encapsulate state within classes. + - Minimize mutable state; favor immutable data structures where appropriate. + - Use dependency injection to manage external dependencies. + +- **Error Handling:** + - Use `try...except` blocks to handle potential exceptions (e.g., file not found, invalid image format). + - Log errors for debugging and monitoring. + - Raise exceptions to propagate errors up the call stack. + +## 3. Performance Considerations + +- **Optimization Techniques:** + - **Vectorization:** Use NumPy's vectorized operations instead of loops for faster processing. + - **Pre-allocation:** Pre-allocate NumPy arrays instead of repeatedly resizing them. + - **Caching:** Cache frequently accessed data (e.g., lookup tables) to avoid recalculation. + - **Multiprocessing/Multithreading:** Use `multiprocessing` or `threading` to parallelize independent image processing tasks. + - **Cython:** Implement performance-critical sections in Cython for C-like speeds. + - **GPU Acceleration:** Leverage OpenCV's CUDA module for GPU-accelerated processing (if applicable). + +- **Memory Management:** + - Release unused memory by explicitly deleting large arrays or objects. + - Be mindful of memory copies. Use `view()` or `ascontiguousarray()` to avoid unnecessary copies. + - Use generators to process large images in chunks. + +- **Rendering Optimization:** + - Minimize the number of draw calls. + - Use efficient drawing functions (e.g., `cv2.polylines()` instead of individual `cv2.line()` calls). + +- **Bundle Size Optimization:** + - Not directly applicable to opencv-python. For applications that use opencv-python, dependencies should be managed effectively. + +- **Lazy Loading:** + - Load images only when needed. + - Use lazy initialization for computationally expensive objects. + +## 4. Security Best Practices + +- **Common Vulnerabilities:** + - **Denial of Service (DoS):** Prevent processing extremely large or malformed images that can consume excessive resources. + - **Code Injection:** Avoid executing arbitrary code based on user input (e.g., constructing file paths dynamically without proper sanitization). + - **Buffer Overflows:** Check image dimensions and data types to prevent buffer overflows when accessing pixel data directly. + +- **Input Validation:** + - Validate image file extensions and formats. + - Check image dimensions and data types. + - Sanitize filenames to prevent path traversal attacks. + +- **Authentication and Authorization:** + - Not typically applicable to core opencv-python functionality. These are relevant for applications leveraging opencv-python for tasks like facial recognition or video surveillance, requiring access control mechanisms. + +- **Data Protection:** + - Encrypt sensitive image data at rest and in transit (if applicable). + - Implement access control mechanisms to restrict access to sensitive image data. + - Anonymize or redact personally identifiable information (PII) from images. + +- **Secure API Communication:** + - Enforce HTTPS for all API communication. + - Use secure authentication mechanisms (e.g., API keys, OAuth). + - Validate all API requests and responses. + +## 5. Testing Approaches + +- **Unit Testing:** + - Test individual functions and classes in isolation. + - Use mocking to isolate components from external dependencies (e.g., file system, camera). + - Test edge cases and boundary conditions. + - Use libraries like `pytest` and `unittest`. + +- **Integration Testing:** + - Test the interaction between different components. + - Verify that data flows correctly between modules. + - Use realistic test data. + +- **End-to-End Testing:** + - Test the entire application workflow. + - Simulate user interactions. + - Verify that the application meets the requirements. + +- **Test Organization:** + - Structure tests to mirror the source code organization. + - Use descriptive test names. + - Keep tests concise and focused. + +- **Mocking and Stubbing:** + - Use `unittest.mock` or `pytest-mock` to create mock objects for external dependencies. + - Stub out functions that perform I/O operations or interact with hardware. + +## 6. Common Pitfalls and Gotchas + +- **BGR vs. RGB:** Be aware that OpenCV uses BGR color format by default, while other libraries (e.g., Matplotlib, PIL) use RGB. Convert between formats using `cv2.cvtColor()`. +- **Image Data Types:** Understand the different image data types (e.g., `uint8`, `float32`) and their implications for processing. +- **Memory Layout:** Be aware of the memory layout of NumPy arrays and use `ascontiguousarray()` when necessary. +- **`cv2.waitKey()`:** Remember to call `cv2.waitKey()` after `cv2.imshow()` to display the image and allow the window to respond to events. A value of 0 will wait indefinitely for a key press. +- **Incorrect File Paths:** Double-check file paths when loading images or videos. +- **Version Compatibility:** Be aware of compatibility issues between different versions of opencv-python. +- **Global Interpreter Lock (GIL):** Understand the implications of the GIL for multithreading in Python. Use multiprocessing for CPU-bound tasks. + +## 7. Tooling and Environment + +- **Recommended Development Tools:** + - **IDE:** VS Code, PyCharm, Jupyter Notebook. + - **Virtual Environment:** `venv`, `conda`. + - **Package Manager:** `pip`, `uv`. + +- **Build Configuration:** + - Use `pyproject.toml` for project metadata and build configuration. + - Specify dependencies in `requirements.txt` or `pyproject.toml`. + - Use a build system like `setuptools` or `poetry`. + +- **Linting and Formatting:** + - Use `flake8` for linting. + - Use `black` for formatting. + - Configure your IDE to automatically format code on save. + +- **Deployment:** + - Containerize your application using Docker. + - Use a deployment platform like AWS, Azure, or Google Cloud. + +- **CI/CD:** + - Integrate with a CI/CD system like Jenkins, GitLab CI, or GitHub Actions. + - Automate testing, linting, and deployment. \ No newline at end of file diff --git a/.cursor/rules/pandas.mdc b/.cursor/rules/pandas.mdc new file mode 100644 index 000000000..96c92be06 --- /dev/null +++ b/.cursor/rules/pandas.mdc @@ -0,0 +1,175 @@ +--- +description: This rule outlines best practices for using the pandas library in Python, covering code style, performance, data handling, and testing. It aims to promote maintainable, efficient, and robust data analysis workflows. +globs: *.py +--- +# Pandas Best Practices + +This document provides guidelines for writing high-quality pandas code, covering various aspects from code style to performance optimization. + +## 1. Code Organization and Structure + +- **Directory Structure:** + - Organize your project with a clear directory structure. + - Use separate directories for data, scripts, modules, tests, and documentation. + - Example: + + project_root/ + ├── data/ + │ ├── raw/ + │ └── processed/ + ├── src/ + │ ├── modules/ + │ │ ├── data_cleaning.py + │ │ ├── feature_engineering.py + │ │ └── ... + │ ├── main.py + ├── tests/ + │ ├── unit/ + │ ├── integration/ + │ └── ... + ├── docs/ + ├── notebooks/ + ├── requirements.txt + └── ... + + +- **File Naming Conventions:** + - Use descriptive and consistent file names. + - Use snake_case for Python files and variables. + - Example: `data_processing.py`, `load_data.py` + +- **Module Organization:** + - Break down your code into reusable modules. + - Each module should focus on a specific task or functionality. + - Use clear and concise function and class names within modules. + - Follow the Single Responsibility Principle (SRP). + +- **Component Architecture:** + - Design your pandas-based applications with a clear component architecture. + - Separate data loading, preprocessing, analysis, and visualization into distinct components. + - Use classes to encapsulate related functionality and data. + +- **Code Splitting Strategies:** + - Split large pandas operations into smaller, more manageable steps. + - Use functions or methods to encapsulate these steps. + - This improves readability and makes debugging easier. + +## 2. Common Patterns and Anti-patterns + +- **Design Patterns Specific to Pandas:** + - **Chain of Responsibility:** Use method chaining to perform a sequence of operations on a DataFrame. + - **Strategy Pattern:** Implement different data processing strategies based on input parameters. + - **Factory Pattern:** Use factory functions to create DataFrames or Series from various sources. + +- **Recommended Approaches for Common Tasks:** + - **Data Loading:** Use `pd.read_csv()`, `pd.read_excel()`, `pd.read_sql()` to load data from different sources. + - **Data Cleaning:** Use `dropna()`, `fillna()`, `replace()` to clean missing or incorrect data. + - **Data Transformation:** Use `apply()`, `map()`, `groupby()`, `pivot_table()` to transform data. + - **Data Aggregation:** Use `groupby()`, `agg()`, `transform()` to aggregate data. + - **Data Visualization:** Use `matplotlib`, `seaborn`, or `plotly` to visualize data. + +- **Anti-patterns and Code Smells to Avoid:** + - **Iterating over DataFrames with `iterrows()` or `itertuples()`:** These are slow; prefer vectorized operations. + - **Chained Indexing:** Avoid using chained indexing (e.g., `df['A']['B']`) as it can lead to unexpected behavior. Use `.loc` or `.iloc` instead. + - **Ignoring Data Types:** Always be aware of your data types and convert them appropriately using `astype()`. + - **Writing loops to filter/process the data:** Vectorize these operations to avoid performance issues. + - **Modifying DataFrame inplace:** Creating copies is a safer approach. + +- **State Management Best Practices:** + - Avoid modifying DataFrames in place unless absolutely necessary. + - Make copies of DataFrames using `.copy()` to avoid unintended side effects. + - Use functional programming principles where possible to minimize state changes. + +- **Error Handling Patterns:** + - Use `try-except` blocks to handle potential errors during data loading, processing, or analysis. + - Log errors and warnings using the `logging` module. + - Raise exceptions when necessary to signal errors to the calling code. + +## 3. Performance Considerations + +- **Optimization Techniques:** + - **Vectorization:** Use vectorized operations instead of loops whenever possible. Pandas is optimized for vectorized operations. + - **Cython:** Consider using Cython to optimize performance-critical parts of your code. + - **Numba:** Use Numba to JIT-compile NumPy and pandas code for improved performance. + - **Dask:** Use Dask for parallel processing of large datasets that don't fit in memory. + - **Parquet or Feather:** Use Parquet or Feather file formats for efficient data storage and retrieval. + - **Categorical Data:** Use categorical data types for columns with a limited number of unique values. + +- **Memory Management:** + - **Data Types:** Choose appropriate data types to minimize memory usage (e.g., `int8`, `float32` instead of `int64`, `float64`). + - **Chunking:** Load large datasets in chunks to avoid memory errors. + - **Garbage Collection:** Use `gc.collect()` to explicitly release memory. + - **Sparse Data:** Use sparse data structures for data with many missing values. + +## 4. Security Best Practices + +- **Common Vulnerabilities and How to Prevent Them:** + - **SQL Injection:** When reading data from SQL databases, use parameterized queries or ORM frameworks to prevent SQL injection attacks. + - **CSV Injection:** Be cautious when loading CSV files from untrusted sources, as they may contain malicious formulas that can execute arbitrary code. + - **Arbitrary Code Execution:** Avoid using `eval()` or `exec()` on data loaded from untrusted sources, as this can lead to arbitrary code execution. + +- **Input Validation:** + - Validate user inputs to prevent malicious data from entering your pandas workflows. + - Use regular expressions or custom validation functions to check the format and content of inputs. + +- **Data Protection Strategies:** + - Encrypt sensitive data at rest and in transit. + - Use access control mechanisms to restrict access to sensitive data. + - Anonymize or pseudonymize data when possible to protect privacy. + +## 5. Testing Approaches + +- **Unit Testing Strategies:** + - Write unit tests for individual functions and classes. + - Use `pytest` or `unittest` for writing and running tests. + - Test edge cases and boundary conditions. + - Use assert statements to verify the correctness of your code. + +- **Integration Testing:** + - Write integration tests to verify the interaction between different modules or components. + - Test the end-to-end functionality of your pandas-based applications. + +- **Test Organization:** + - Organize your tests in a separate `tests` directory. + - Use a clear naming convention for your test files and functions. + - Example: `test_data_cleaning.py`, `test_load_data()` + +- **Mocking and Stubbing:** + - Use mocking and stubbing to isolate units of code during testing. + - Use the `unittest.mock` module or third-party libraries like `pytest-mock`. + +## 6. Common Pitfalls and Gotchas + +- **Frequent Mistakes Developers Make:** + - **Forgetting to set the index properly:** This can lead to performance issues when joining or merging DataFrames. + - **Incorrectly handling missing data:** Be aware of how missing data is represented in your DataFrames and handle it appropriately. + - **Not understanding the difference between `.loc` and `.iloc`:** These methods are used for different types of indexing and can lead to unexpected results if used incorrectly. + +- **Edge Cases to Be Aware Of:** + - **Empty DataFrames:** Handle the case where a DataFrame is empty. + - **DataFrames with duplicate indices:** Be aware of how pandas handles DataFrames with duplicate indices. + +- **Debugging Strategies:** + - Use the `print()` function or the `logging` module to debug your code. + - Use a debugger to step through your code and inspect variables. + - Use the `pdb` module for interactive debugging. + +## 7. Tooling and Environment + +- **Recommended Development Tools:** + - **Jupyter Notebook/Lab:** For interactive data exploration and analysis. + - **VS Code with Python extension:** For code editing, debugging, and testing. + - **PyCharm:** A full-featured IDE for Python development. + +- **Linting and Formatting:** + - Use `flake8` to lint your code and identify potential issues. + - Use `black` to automatically format your code according to PEP 8. + - Use `isort` to automatically sort your imports. + - Integrate these tools into your pre-commit hooks to ensure consistent code style. + +- **CI/CD Integration:** + - Use CI/CD pipelines to automate the testing and deployment of your pandas-based applications. + - Integrate your CI/CD pipelines with your version control system (e.g., GitHub, GitLab, Bitbucket). + - Use Docker to containerize your applications for consistent deployment. + +This comprehensive guide covers the key aspects of pandas best practices and coding standards. By following these guidelines, you can write more maintainable, efficient, and robust pandas code. \ No newline at end of file diff --git a/.cursor/rules/pdoc.mdc b/.cursor/rules/pdoc.mdc new file mode 100644 index 000000000..ee29fcaff --- /dev/null +++ b/.cursor/rules/pdoc.mdc @@ -0,0 +1,243 @@ +--- +description: Comprehensive best practices for using pdoc to generate and maintain Python project documentation. Covers code structure, performance, security, testing, and tooling to ensure high-quality documentation and maintainable projects. +globs: *.py +--- +# pdoc Best Practices: A Comprehensive Guide + +This document outlines best practices for using `pdoc` to generate API documentation for Python projects. It covers code organization, performance considerations, security, testing, and common pitfalls, all tailored for effective documentation generation and maintainability. + +## Library Information: + +- Name: pdoc +- Tags: development, documentation, python, auto-generation + +## 1. Code Organization and Structure + +### 1.1 Directory Structure Best Practices + +- **Clear Module Grouping:** Organize your code into logical modules and sub-packages. This makes it easier for `pdoc` to generate a navigable API. +- **`__init__.py` Files:** Ensure that each package directory contains an `__init__.py` file. This file can be empty, or it can contain initialization code and, importantly, the package-level docstring which `pdoc` will use. +- **Separate Concerns:** Isolate different functionalities into separate modules. For example, separate data models, business logic, and utility functions into distinct files. + +Example: + + +my_project/ +├── my_package/ +│ ├── __init__.py # Package-level docstring here +│ ├── module_a.py # Contains functions and classes +│ ├── module_b.py # Contains more functions and classes +│ └── subpackage/ +│ ├── __init__.py # Subpackage-level docstring here +│ └── module_c.py # Contains functions and classes +└── setup.py # Installation script + + +### 1.2 File Naming Conventions + +- **Descriptive Names:** Use descriptive names for modules and packages. For example, `database_utils.py` is much better than `db.py`. +- **Lowercase with Underscores:** Follow the standard Python convention of using lowercase letters with underscores for module names (e.g., `my_module.py`). + +### 1.3 Module Organization + +- **Top-Level Docstrings:** Every module should have a top-level docstring that describes its purpose and lists the main classes, functions, and exceptions it defines. +- **`__all__` Variable:** Use the `__all__` variable to explicitly define the public API of a module. This tells `pdoc` (and other tools) which names should be included in the generated documentation. + +Example: + +python +# my_module.py +"""This module provides utility functions for data processing. + +Classes: + DataProcessor: Processes data. + +Functions: + clean_data: Cleans the data. +""" + +__all__ = ['DataProcessor', 'clean_data'] + +class DataProcessor: + """Processes data from various sources.""" + ... + +def clean_data(data): + """Cleans the input data.""" + ... + + +### 1.4 Component Architecture + +- **Loose Coupling:** Design your components to be loosely coupled. This means that components should have minimal dependencies on each other. +- **Clear Interfaces:** Define clear interfaces for your components. This makes it easier to test and document your code. +- **Abstraction:** Use abstraction to hide the implementation details of your components. This makes your code more flexible and easier to maintain. + +### 1.5 Code Splitting + +- **Divide and Conquer:** Split large modules into smaller, more manageable files. This makes your code easier to understand and maintain. +- **By Functionality:** Split code based on functionality. For example, if you have a module that contains both database access code and business logic, split it into two separate modules. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns + +- **Factory Pattern:** Use the factory pattern to create objects. This makes it easier to switch between different implementations of a class. +- **Strategy Pattern:** Use the strategy pattern to encapsulate different algorithms. This makes it easier to switch between different algorithms at runtime. +- **Decorator Pattern:** Use the decorator pattern to add functionality to objects dynamically. + +### 2.2 Recommended Approaches + +- **Use Docstrings:** All modules, classes, functions, and methods must include docstrings. Docstrings are used by `pdoc` to create the API documentation. +- **Follow PEP 257:** Adhere to PEP 257 for docstring conventions. This ensures consistency and readability. +- **Choose a Style:** Consistently use a docstring style (Google, NumPy, reStructuredText) throughout your project. +- **Include Examples:** Provide simple, executable examples in your docstrings. This helps users understand how to use your code. These examples can be embedded directly within docstrings using the `doctest` module. + +### 2.3 Anti-patterns + +- **Missing Docstrings:** Failing to document modules, classes, functions, or methods. +- **Incomplete Docstrings:** Providing docstrings that are too brief or lack essential information. +- **Inconsistent Style:** Mixing different docstring styles within the same project. +- **Redundant Documentation:** Repeating information that is already clear from the code itself. Focus on the *why* rather than the *how*. +- **Ignoring `__all__`:** Not defining `__all__` or defining it incorrectly can lead to undocumented or wrongly documented API surfaces. + +### 2.4 State Management + +- **Minimize Global State:** Avoid using global variables as much as possible. If you must use global variables, document them clearly. +- **Use Classes to Encapsulate State:** Use classes to encapsulate state. This makes it easier to manage and reason about your code. + +### 2.5 Error Handling + +- **Raise Exceptions:** Raise exceptions when errors occur. This makes it easier to handle errors gracefully. +- **Use Try-Except Blocks:** Use try-except blocks to catch exceptions. This prevents your program from crashing when errors occur. +- **Document Exceptions:** Document the exceptions that your functions and methods can raise. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +- **Profiling:** Use a profiler to identify performance bottlenecks in your code. +- **Caching:** Use caching to store frequently accessed data. Consider using `functools.lru_cache` for memoization. +- **Vectorization:** Use vectorized operations when working with numerical data. This is often faster than using loops. + +### 3.2 Memory Management + +- **Avoid Memory Leaks:** Be careful to avoid memory leaks. Free up memory when it is no longer needed. +- **Use Generators:** Use generators to process large amounts of data. This can reduce memory usage. +- **Use Data Structures Efficiently:** Use the right data structures for the job. For example, use sets for membership testing and dictionaries for lookups. + +### 3.3 Rendering Optimization + +- Not generally applicable to `pdoc` itself, as it mainly generates static HTML. However, the design of your code *can* affect how quickly `pdoc` can process it. Well-structured, easily-introspected code will be processed faster. + +### 3.4 Bundle Size + +- Not applicable to `pdoc` itself, as it is a documentation generator and not a web application framework. + +### 3.5 Lazy Loading + +- Not directly applicable to `pdoc`, but consider lazy loading modules within your own code to improve startup time if you have a large project that `pdoc` needs to process. This is done using `importlib.import_module`. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities + +- **Not Directly Applicable:** `pdoc` generates static documentation and does not execute arbitrary user-supplied code. Therefore, it is less vulnerable than many other types of applications. However, be mindful of the dependencies you use and keep them up to date. + +### 4.2 Input Validation + +- **Not Directly Applicable:** `pdoc` does not take direct user input. + +### 4.3 Authentication and Authorization + +- **Not Applicable:** `pdoc` is a documentation generation tool and does not require authentication or authorization. + +### 4.4 Data Protection + +- **Not Applicable:** `pdoc` does not handle sensitive data. + +### 4.5 Secure API Communication + +- **Not Applicable:** `pdoc` does not communicate with external APIs. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +- **Test Docstrings:** Use `doctest` to test examples embedded in your docstrings. This ensures that your examples are up-to-date and correct. `pdoc` renders these examples, making them executable directly from the documentation. +- **Focus on Public API:** Write unit tests for the public API of your modules, classes, and functions. + +### 5.2 Integration Testing + +- **Test Interactions:** Test the interactions between different components of your system. + +### 5.3 End-to-End Testing + +- **Test Complete Workflows:** Test complete workflows to ensure that your system is working correctly. + +### 5.4 Test Organization + +- **Separate Test Files:** Keep your tests in separate files from your code. This makes it easier to find and run your tests. +- **Organize by Module:** Organize your tests by module. This makes it easier to find the tests for a specific module. + +### 5.5 Mocking and Stubbing + +- **Use Mock Objects:** Use mock objects to isolate the code that you are testing from its dependencies. This makes it easier to test your code in isolation. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +- **Incorrect Docstring Formatting:** Using the wrong docstring format or inconsistent formatting. +- **Omitting Parameters:** Failing to document all parameters of a function or method. +- **Not Updating Documentation:** Failing to update the documentation when the code changes. +- **Circular Imports:** Circular imports can confuse `pdoc`. Refactor your code to avoid them. + +### 6.2 Edge Cases + +- **Dynamic Code Generation:** If your code heavily uses dynamic code generation (`eval`, `exec`), `pdoc` may have difficulty introspecting it. +- **Metaclasses:** Complex metaclasses can also make it harder for `pdoc` to generate accurate documentation. + +### 6.3 Version-Specific Issues + +- **Check Compatibility:** Always check the compatibility of `pdoc` with your Python version. Refer to the official `pdoc` documentation for version-specific information. + +### 6.4 Compatibility Concerns + +- **Third-Party Libraries:** Ensure that `pdoc` is compatible with any third-party libraries that you are using. + +### 6.5 Debugging Strategies + +- **Verbose Mode:** Run `pdoc` in verbose mode to see more detailed output. This can help you identify errors. +- **Check for Errors:** Check the output of `pdoc` for any errors or warnings. +- **Inspect Generated HTML:** Inspect the generated HTML to see if the documentation looks correct. + +## 7. Tooling and Environment + +### 7.1 Recommended Tools + +- **IDE:** Use a good IDE (e.g., VS Code, PyCharm) with Python support. +- **Linting:** Use a linter (e.g., pylint, flake8) to catch errors and enforce coding standards. +- **Formatting:** Use a code formatter (e.g., black, autopep8) to format your code. + +### 7.2 Build Configuration + +- **`setup.py` or `pyproject.toml`:** Use a `setup.py` file or `pyproject.toml` to define your project's dependencies and metadata. +- **Include Documentation Files:** Make sure to include documentation files in your build configuration. + +### 7.3 Linting and Formatting + +- **Consistent Style:** Enforce a consistent coding style using a linter and code formatter. +- **Follow PEP 8:** Follow the PEP 8 style guide for Python code. +- **Use Docstring Checks:** Configure your linter to check for missing or incomplete docstrings. + +### 7.4 Deployment Best Practices + +- **Generate Documentation Automatically:** Automate the generation of documentation as part of your build process. +- **Host Documentation:** Host the generated documentation on a web server or documentation hosting service (e.g., Read the Docs). + +### 7.5 CI/CD Integration + +- **Integrate with CI/CD:** Integrate `pdoc` with your CI/CD pipeline to automatically generate and deploy documentation on every commit. + +By following these best practices, you can ensure that your Python projects have high-quality, well-maintained documentation that is easy for users to understand and use. \ No newline at end of file diff --git a/.cursor/rules/peewee.mdc b/.cursor/rules/peewee.mdc new file mode 100644 index 000000000..0f132cf47 --- /dev/null +++ b/.cursor/rules/peewee.mdc @@ -0,0 +1,365 @@ +--- +description: Comprehensive guide for Peewee ORM best practices, covering code organization, performance, security, testing, and common pitfalls. Provides actionable guidance for developers to write maintainable and efficient database-driven applications using Peewee. +globs: *.py +--- +# Peewee ORM Best Practices and Coding Standards + +This document outlines best practices and coding standards for using Peewee ORM in Python development. Following these guidelines will lead to more maintainable, efficient, and secure database-driven applications. + +## Library Information: + +- Name: peewee +- Tags: database, orm, python, sql + +## 1. Code Organization and Structure + +### 1.1. Directory Structure + + +project_root/ +├── models/ +│ ├── __init__.py +│ ├── user.py +│ ├── product.py +│ └── ... +├── migrations/ +│ ├── 0001_initial.py +│ ├── 0002_add_index.py +│ └── ... +├── utils/ +│ ├── db.py # Database connection and initialization +│ └── ... +├── tests/ +│ ├── __init__.py +│ ├── test_user.py +│ ├── test_product.py +│ └── ... +├── main.py # Application entry point +└── requirements.txt + + +- **models/:** Contains model definitions, each in a separate file. +- **migrations/:** Stores database migration scripts. +- **utils/:** Utility functions, including database connection management. +- **tests/:** Unit and integration tests. +- **main.py:** The main application file. + +### 1.2. File Naming Conventions + +- Model files: Use lowercase, singular nouns (e.g., `user.py`, `product.py`). +- Migration files: Use a sequential numbering scheme (e.g., `0001_initial.py`, `0002_add_index.py`). +- Test files: Prefix with `test_` (e.g., `test_user.py`). +- Utility files: Use descriptive names (e.g., `db.py`). + +### 1.3. Module Organization + +- Group related models into the same module. For example, models related to user management would reside in `models/user.py`. +- Use `__init__.py` files to make directories importable as packages and to provide a convenient way to import all models from the `models` directory. Example: + +python +# models/__init__.py +from .user import User +from .product import Product + + +### 1.4. Component Architecture + +- Separate database interactions from business logic. Create service layers or repositories that handle Peewee queries and data manipulation, keeping models thin and focused on schema definition. +- Favor composition over inheritance where appropriate, using mixins for reusable model functionality. + +### 1.5. Code Splitting + +- Use separate files for different model definitions to improve readability and maintainability. +- Decompose complex queries into smaller, reusable functions. +- Consider using submodules within `models/` for logical groupings of models when projects grow large. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +- **Repository Pattern:** Create a layer between the application and the database to abstract data access logic. This makes it easier to switch databases or change ORM implementations later. +- **Unit of Work:** Manage database transactions within a single unit of work, ensuring consistency and atomicity. Peewee's `atomic()` context manager facilitates this pattern. +- **Data Mapper:** Maps data between domain objects and the database. Peewee models inherently implement this pattern. + +### 2.2. Recommended Approaches + +- **Explicit Database Connection Management:** Always open and close database connections explicitly, especially in web applications or concurrent environments. Use `with db:` or `@db.connection_context()` to ensure connections are properly handled. Disable `autoconnect` when initializing the database: + +python +db = SqliteDatabase('my_app.db', autoconnect=False) + + +- **Base Model Class:** Define a base model class that all other models inherit from. This centralizes database configuration and ensures consistency: + +python +from peewee import * + +db = SqliteDatabase('my_app.db') + +class BaseModel(Model): + class Meta: + database = db + + +- **Relationships and Joins:** Utilize Peewee's built-in support for relationships and joins to simplify complex queries: + +python +class User(BaseModel): + username = CharField() + +class Tweet(BaseModel): + user = ForeignKeyField(User, backref='tweets') + content = TextField() + +query = Tweet.select().join(User).where(User.username == 'john_doe') + + +- **Migrations:** Use a migration library (like `peewee-migrate`) to manage schema changes effectively. This ensures that database schemas can evolve smoothly over time. + +### 2.3. Anti-patterns and Code Smells + +- **Implicit Database Connections:** Relying on Peewee's `autoconnect` feature can lead to unpredictable behavior and connection leaks. Manage connections explicitly. +- **Global Database Instance:** Avoid creating a single global database instance without proper connection management, especially in multi-threaded applications. Use connection pools or context managers to manage connections per request or thread. +- **String Interpolation:** Never use string interpolation to construct SQL queries, as this can lead to SQL injection vulnerabilities. Always use parameterized queries. + +python +# Anti-pattern (SQL injection vulnerability!) +username = input("Enter username:") +query = User.select().where(SQL("username = '%s'" % username)) + +# Correct approach (parameterized query) +query = User.select().where(User.username == username) + + +- **Over-fetching Data:** Avoid selecting all columns when only a subset is needed. Specify the required fields in the `select()` method. + +### 2.4. State Management + +- In web applications, manage database connections on a per-request basis. Open a connection at the beginning of the request and close it at the end. Framework integration examples are provided in the base documentation. +- In asynchronous applications, utilize connection pools and context managers to manage connections concurrently. + +### 2.5. Error Handling + +- Use try-except blocks to handle database errors, such as `IntegrityError` (for unique constraint violations) and `DoesNotExist` (when a record is not found). +- Implement logging to track database errors and diagnose issues. +- Provide informative error messages to the user, without exposing sensitive database details. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +- **Indexing:** Add indexes to frequently queried columns to improve query performance. Use Peewee's indexing features when defining your models or use migrations to create them: + +python +class User(BaseModel): + username = CharField(index=True) # Simple index + + class Meta: + indexes = ( + (('email', 'is_active'), True), # Composite index (unique=True) + ) + + +- **Caching:** Implement caching for frequently accessed data to reduce database load. Consider using a caching library like Redis or Memcached. +- **Connection Pooling:** Use connection pooling to reduce the overhead of establishing new database connections for each request. Peewee's `playhouse.pool` module provides pooled database classes: + +python +from playhouse.pool import PooledSqliteDatabase + +db = PooledSqliteDatabase('my_app.db', max_connections=16) + + +- **Batch Inserts/Updates:** Use `insert_many()` and `update()` methods for bulk operations instead of iterating and executing individual queries. + +python +data = [ + {'username': 'user1', 'email': 'user1@example.com'}, + {'username': 'user2', 'email': 'user2@example.com'}, +] + +with db.atomic(): # Transaction for atomicity + User.insert_many(data).execute() + + +- **Query Optimization:** Use `.prefetch()` to reduce N+1 query problems. Use `.defer()` to avoid fetching unnecessary columns. Profile your queries and analyze execution plans to identify bottlenecks. + +### 3.2. Memory Management + +- Limit the number of records returned by queries to avoid loading large amounts of data into memory. Use pagination or filtering to retrieve data in smaller chunks. +- Close database connections promptly after use to release resources. +- Be mindful of large data types (e.g., BLOBs) and handle them efficiently to avoid memory exhaustion. + +### 3.3. Bundle Size Optimization + +- Peewee is a lightweight library, so its direct impact on bundle size is minimal. However, be mindful of dependencies introduced by database drivers or extensions. +- Use a minifier to reduce the size of your Python code before deployment. + +### 3.4. Lazy Loading + +- Use `ForeignKeyField` with caution, understanding that accessing the related object will trigger a new database query if the related object is not prefetched. Utilize `.prefetch()` for efficient loading of related data. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities + +- **SQL Injection:** Prevent SQL injection by using parameterized queries and avoiding string concatenation when constructing SQL statements. +- **Cross-Site Scripting (XSS):** If displaying data retrieved from the database in a web application, sanitize the data to prevent XSS attacks. (This is primarily a front-end concern, but relevant when handling user-generated content stored in the database). +- **Sensitive Data Exposure:** Avoid storing sensitive data (e.g., passwords, API keys) in plain text. Use proper encryption or hashing techniques. Peewee itself does not provide encryption; use appropriate Python libraries for this (like `bcrypt` for passwords). + +### 4.2. Input Validation + +- Validate all user inputs before using them in database queries or model fields. This prevents malicious data from being stored in the database or used to exploit vulnerabilities. +- Use Peewee's field validation capabilities to enforce data types, lengths, and other constraints at the model level: + +python +class User(BaseModel): + username = CharField(max_length=50) + email = CharField(unique=True) + age = IntegerField(null=True, constraints=[Check('age > 0')]) #CHECK constraint + + + +### 4.3. Authentication and Authorization + +- Implement robust authentication and authorization mechanisms to protect sensitive data and restrict access to authorized users only. Peewee can integrate with authentication frameworks like Flask-Login or Django's authentication system. +- Do not rely solely on client-side validation for security. Always perform server-side validation. + +### 4.4. Data Protection + +- Implement data encryption at rest and in transit to protect sensitive data from unauthorized access. Consider using database-level encryption or encrypting sensitive fields at the application level. +- Use HTTPS to secure communication between the client and the server. +- Comply with relevant data privacy regulations (e.g., GDPR, CCPA). + +### 4.5. Secure API Communication + +- Use API keys or tokens to authenticate API requests. +- Implement rate limiting to prevent abuse and denial-of-service attacks. +- Validate API request data and sanitize API responses to prevent vulnerabilities. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +- Unit test model methods, query logic, and other database-related functionality in isolation. Use mocking and stubbing to isolate units of code from external dependencies (e.g., the database itself). +- Utilize an in-memory SQLite database for unit tests to avoid modifying the production database. Bind the models to the in-memory database during testing. + +python +import unittest +from peewee import * + +db = SqliteDatabase(':memory:') + +class BaseModel(Model): + class Meta: + database = db + +class User(BaseModel): + username = CharField() + +class TestUser(unittest.TestCase): + def setUp(self): + db.bind([User]) + db.create_tables([User]) + + def tearDown(self): + db.drop_tables([User]) + db.close() + + def test_user_creation(self): + user = User.create(username='test_user') + self.assertEqual(user.username, 'test_user') + +if __name__ == '__main__': + unittest.main() + + +### 5.2. Integration Testing + +- Integration test the interaction between different components of the application, including database interactions. +- Use a separate test database (e.g., a dedicated SQLite file or a testing instance of your production database) for integration tests. +- Ensure that the test database is properly initialized and cleaned up before and after each test. + +### 5.3. End-to-End Testing + +- End-to-end tests verify the entire application workflow, including database interactions, from the user's perspective. +- Use tools like Selenium or Cypress to automate end-to-end tests. +- Ensure that the end-to-end tests cover critical business scenarios and data flows. + +### 5.4. Test Organization + +- Organize tests into separate modules or directories based on the component or functionality being tested. +- Use descriptive test names to clearly indicate the purpose of each test. +- Follow the Arrange-Act-Assert pattern to structure tests. + +### 5.5. Mocking and Stubbing + +- Use mocking and stubbing to isolate units of code during unit testing and to simulate database behavior. +- Use libraries like `unittest.mock` or `pytest-mock` for mocking and stubbing. +- Mock database connections, queries, and model methods as needed. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +- **Forgetting to Handle Exceptions:** Neglecting to handle database exceptions can lead to application crashes or data corruption. +- **Incorrect Data Types:** Using incorrect data types for model fields can result in data truncation or validation errors. +- **Not Closing Connections:** Failing to close database connections can lead to connection leaks and performance issues. +- **N+1 Query Problem:** Inefficiently fetching related data can result in the N+1 query problem, where a query is executed for each related record. Use `prefetch()` to mitigate this. + +### 6.2. Edge Cases + +- **Concurrency Issues:** Be aware of concurrency issues when multiple threads or processes access the database simultaneously. Use transactions and locking mechanisms to ensure data consistency. +- **Large Datasets:** Handle large datasets efficiently by using pagination, streaming, or batch processing techniques. +- **Database-Specific Features:** Be aware of database-specific features and syntax differences when writing queries. Use Peewee's database-specific extensions when necessary. + +### 6.3. Version-Specific Issues + +- Consult the Peewee documentation and release notes for version-specific issues and compatibility concerns. +- Test your application thoroughly after upgrading Peewee or your database driver. + +### 6.4. Compatibility Concerns + +- Ensure that the Peewee version is compatible with the Python version and the database driver being used. +- Be aware of potential compatibility issues when integrating Peewee with other libraries or frameworks. + +### 6.5. Debugging Strategies + +- Enable logging to track database queries and errors. +- Use a database profiler to analyze query performance. +- Use a debugger to step through code and inspect variables. +- Simplify complex queries to isolate the source of the problem. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +- **Integrated Development Environment (IDE):** VS Code, PyCharm, or other IDEs with Python support. +- **Database Client:** DB Browser for SQLite, pgAdmin, MySQL Workbench, or other database clients for interacting with the database. +- **Migration Tool:** Alembic or `peewee-migrate` for managing database schema changes. + +### 7.2. Build Configuration + +- Use `requirements.txt` or `pyproject.toml` to manage project dependencies. +- Specify the Peewee version and database driver versions in the dependencies file. +- Use a virtual environment to isolate project dependencies from the system environment. + +### 7.3. Linting and Formatting + +- Use a linter like Flake8 or Pylint to enforce code style and identify potential errors. +- Use a code formatter like Black to automatically format your code according to a consistent style. +- Configure the linter and formatter to adhere to the Peewee coding standards. + +### 7.4. Deployment + +- Use a deployment tool like Docker or Kubernetes to containerize and deploy your application. +- Configure the application to connect to the production database using environment variables or configuration files. +- Ensure that the database schema is up-to-date before deploying the application. Run database migrations as part of the deployment process. +- Monitor the application and database for performance and security issues. + +### 7.5. CI/CD Integration + +- Integrate Peewee tests into your CI/CD pipeline to automatically run tests whenever code is committed or deployed. +- Use a CI/CD tool like Jenkins, GitLab CI, or GitHub Actions to automate the build, test, and deployment process. +- Configure the CI/CD pipeline to run database migrations and seed the database with test data. + +This guide provides a comprehensive overview of Peewee ORM best practices. By following these guidelines, you can create robust, efficient, and maintainable database-driven applications. \ No newline at end of file diff --git a/.cursor/rules/phoenix.mdc b/.cursor/rules/phoenix.mdc new file mode 100644 index 000000000..c85f0933e --- /dev/null +++ b/.cursor/rules/phoenix.mdc @@ -0,0 +1,270 @@ +--- +description: This rule outlines the best practices and coding standards for developing Elixir applications with the Phoenix framework, covering code organization, performance, security, testing, and common pitfalls. +globs: *.ex,*.exs,*.eex,*.leex,*.sface +--- +- # Phoenix Framework Best Practices + + This document outlines general best practices for developing Elixir applications with the Phoenix Framework. It aims to promote maintainable, scalable, and performant codebases. + +- ## 1. Code Organization and Structure + + - ### 1.1. Directory Structure Best Practices + + - **`lib/`**: Contains the core application logic, including contexts, schemas, and supporting modules. + - **`lib/_web/`**: Holds the web-related components, such as controllers, views, templates, channels, and live views. + - **`lib//`**: Contains your application’s core domain logic. Favor placing most of your code here. + - **`test/`**: Contains all test-related files, including unit, integration, and acceptance tests. + - **`priv/repo/migrations/`**: Stores database migration files. + - **`config/`**: Contains configuration files for different environments (dev, test, prod). + - **`assets/`**: Contains static assets like JavaScript, CSS, and images. Managed by tools like esbuild or Webpack. + - **`deps/`**: Automatically generated folder containing project dependencies. + - **`rel/`**: Used for building releases with distillery or mix release. Contains configuration for releases. + + - ### 1.2. File Naming Conventions + + - Use `snake_case` for file names (e.g., `user_controller.ex`, `user_schema.ex`). + - Match the file name to the module name (e.g., `UserController` module should be in `user_controller.ex`). + - For LiveView components, use `_live` suffix (e.g., `user_live.ex`). + - For schema files, use `_schema` suffix(e.g. `user_schema.ex`). + + - ### 1.3. Module Organization + + - Organize modules into contexts, representing logical domains within the application (e.g., `Accounts`, `Blog`, `Payments`). + - Keep modules small and focused, adhering to the Single Responsibility Principle (SRP). + - Use namespaces to group related modules (e.g., `MyApp.Accounts.User`, `MyApp.Blog.Post`). + - Favor explicit dependencies between modules to promote clarity and reduce coupling. + + - ### 1.4. Component Architecture (LiveView) + + - Break down complex LiveView pages into smaller, reusable components. + - Use functional components (HEEx templates with function definitions) for simple UI elements. + - Use LiveComponent modules for more complex, stateful components with event handling. + - Organize components into directories based on their purpose or domain. + - Define attributes and slots for LiveView components to ensure type safety. + - Utilize PubSub for communication between LiveView components. + + - ### 1.5. Code Splitting Strategies + + - Separate concerns by using contexts. Controllers delegate to contexts, which handle the business logic, and contexts rely on schemas, which handle the data validations and structure. + - Utilize umbrella projects to divide a large application into smaller, independent applications. + - Consider code splitting at the LiveView level, using separate LiveView modules for different sections of a page. + - Use dynamic imports for JavaScript modules to load code on demand. + +- ## 2. Common Patterns and Anti-patterns + + - ### 2.1. Design Patterns + + - **MVC (Model-View-Controller)**: The core architectural pattern in Phoenix. Ensure proper separation of concerns between models (schemas), views (templates), and controllers. + - **Context Pattern**: Encapsulate business logic and data access within contexts to provide a clear API for controllers and other parts of the application. + - **Repository Pattern**: Abstract away database interactions behind a repository interface to allow for easier testing and potential database changes. While contexts often serve this purpose in Phoenix, a dedicated repository layer can be useful for very complex data access logic. + - **PubSub**: Use `Phoenix.PubSub` for real-time communication between different parts of the application (e.g., LiveView components, channels). + - **Ecto Changesets**: Employ Ecto changesets for data validation and sanitization before persisting to the database. + + - ### 2.2. Recommended Approaches for Common Tasks + + - **Authentication**: Use a library like `Pow` or `Ueberauth` for handling user authentication and authorization. + - **Authorization**: Implement role-based access control (RBAC) or attribute-based access control (ABAC) using libraries like `Pomegranate` or custom logic. + - **Form Handling**: Use `Phoenix.HTML.Form` helpers and Ecto changesets for building and validating forms. + - **Real-time Updates**: Leverage Phoenix Channels or LiveView for real-time updates and interactive features. + - **Background Jobs**: Use `Oban` for reliable background job processing. + + - ### 2.3. Anti-patterns and Code Smells + + - **Fat Controllers**: Avoid putting too much logic in controllers. Delegate to contexts or services. + - **Direct Repo Access in Controllers**: Do not directly call `Repo` functions in controllers. Use contexts. + - **Ignoring Changeset Errors**: Always handle changeset errors and display appropriate messages to the user. + - **Over-Complicated Queries**: Keep Ecto queries simple and composable. Use views or functions in schema modules to build more complex queries. + - **Unnecessary Global State**: Minimize the use of global state. Prefer passing data explicitly between functions and components. + + - ### 2.4. State Management Best Practices (LiveView) + + - Store only the minimal required state in LiveView's `assigns`. + - Use `handle_params` to load data based on URL parameters. + - Employ `handle_info` for handling asynchronous events and updates. + - Implement proper state cleanup in `mount` and `terminate` callbacks. + - Consider using a global state management library like `global` only when truly necessary for cross-component communication. + + - ### 2.5. Error Handling Patterns + + - Use pattern matching to handle different error scenarios. + - Raise exceptions only for truly exceptional cases. For expected errors, return `{:error, reason}` tuples. + - Implement error logging and monitoring using tools like `Sentry` or `Bugsnag`. + - Use `try...rescue` blocks for handling potential exceptions during external API calls or file operations. + - Define custom error types using atoms or structs to provide more context for error handling. + +- ## 3. Performance Considerations + + - ### 3.1. Optimization Techniques + + - **Database Queries**: Optimize Ecto queries by using indexes, preloading associations, and avoiding N+1 queries. + - **Caching**: Implement caching for frequently accessed data using `ETS` tables or a dedicated caching library like `Cachex`. + - **Concurrency**: Leverage Elixir's concurrency features (e.g., `Task`, `GenServer`) to handle concurrent requests and background tasks efficiently. + - **Code Profiling**: Use tools like `Erlang's observer` or `fprof` to identify performance bottlenecks in your code. + - **Connection Pooling**: Ensure proper database connection pooling to minimize connection overhead. + + - ### 3.2. Memory Management + + - **Avoid Memory Leaks**: Be mindful of memory leaks, especially in long-running processes like `GenServers`. + - **Use Streams**: Employ Elixir streams for processing large datasets in a memory-efficient manner. + - **Garbage Collection**: Understand Erlang's garbage collection behavior and optimize code to minimize garbage collection overhead. + + - ### 3.3. Rendering Optimization (LiveView) + + - **Minimize DOM Updates**: Reduce the number of DOM updates in LiveView by using `phx-update` attributes and smart diffing. + - **Use Static Content**: Serve static content (e.g., images, CSS, JavaScript) directly from the web server without involving LiveView. + - **Optimize Templates**: Optimize HEEx templates for efficient rendering. + + - ### 3.4. Bundle Size Optimization + + - **Code Splitting**: Use code splitting to reduce the initial bundle size and load code on demand. + - **Tree Shaking**: Enable tree shaking in esbuild or Webpack to remove unused code from the bundle. + - **Minification**: Minify CSS and JavaScript files to reduce their size. + - **Image Optimization**: Optimize images for web use by compressing them and using appropriate formats. + + - ### 3.5. Lazy Loading + + - **Lazy Load Images**: Implement lazy loading for images to improve initial page load time. + - **Lazy Load Components**: Use dynamic imports or conditional rendering to lazy load LiveView components. + +- ## 4. Security Best Practices + + - ### 4.1. Common Vulnerabilities and Prevention + + - **Cross-Site Scripting (XSS)**: Sanitize user input and use proper escaping in templates to prevent XSS attacks. Phoenix's HEEx templates automatically escape variables, but be careful when using `raw/1` or `safe/1`. + - **Cross-Site Request Forgery (CSRF)**: Protect against CSRF attacks by using Phoenix's built-in CSRF protection mechanisms. Include the CSRF token in forms and AJAX requests. + - **SQL Injection**: Use Ecto's parameterized queries to prevent SQL injection vulnerabilities. + - **Authentication and Authorization Issues**: Implement robust authentication and authorization mechanisms to protect sensitive data and functionality. + + - ### 4.2. Input Validation + + - **Use Ecto Changesets**: Always validate user input using Ecto changesets to ensure data integrity and prevent malicious input. + - **Whitelist Input**: Define a whitelist of allowed input values instead of a blacklist of disallowed values. + - **Sanitize Input**: Sanitize user input to remove potentially harmful characters or code. + + - ### 4.3. Authentication and Authorization + + - **Use Strong Passwords**: Enforce strong password policies and use proper hashing algorithms (e.g., `bcrypt`) to store passwords securely. + - **Implement Multi-Factor Authentication (MFA)**: Add an extra layer of security by requiring users to authenticate using multiple factors. + - **Use JWT (JSON Web Tokens)**: Use JWT for secure API authentication and authorization. + - **Implement Role-Based Access Control (RBAC)**: Define roles and permissions to control access to different parts of the application. + + - ### 4.4. Data Protection + + - **Encrypt Sensitive Data**: Encrypt sensitive data at rest and in transit using appropriate encryption algorithms. + - **Use HTTPS**: Always use HTTPS to encrypt communication between the client and the server. + - **Protect API Keys**: Store API keys securely and restrict access to them. + + - ### 4.5. Secure API Communication + + - **Use HTTPS**: Enforce HTTPS for all API communication. + - **Validate API Requests**: Validate API requests to prevent malicious input and unauthorized access. + - **Rate Limiting**: Implement rate limiting to prevent abuse and denial-of-service attacks. + - **API Versioning**: Use API versioning to ensure backward compatibility and allow for future API changes. + +- ## 5. Testing Approaches + + - ### 5.1. Unit Testing + + - **Test Contexts and Schemas**: Write unit tests for contexts and schemas to ensure that business logic and data validation work as expected. + - **Mock External Dependencies**: Use mocking libraries like `Mock` or `Meck` to isolate units under test and avoid dependencies on external services. + - **Test Edge Cases**: Test edge cases and boundary conditions to ensure that code handles unexpected input correctly. + + - ### 5.2. Integration Testing + + - **Test Controllers and Channels**: Write integration tests for controllers and channels to ensure that they interact correctly with contexts and other parts of the application. + - **Use a Test Database**: Use a dedicated test database to avoid affecting the production database. + - **Verify Database Interactions**: Verify that database interactions are performed correctly by inspecting the database state after running tests. + + - ### 5.3. End-to-End Testing + + - **Use Wallaby or Cypress**: Use end-to-end testing frameworks like Wallaby or Cypress to simulate user interactions and verify that the application works as expected from the user's perspective. + - **Test Critical User Flows**: Test critical user flows to ensure that the most important features of the application are working correctly. + + - ### 5.4. Test Organization + + - **Organize Tests by Module**: Organize tests into directories that mirror the application's module structure. + - **Use Descriptive Test Names**: Use descriptive test names to clearly indicate what each test is verifying. + - **Keep Tests Small and Focused**: Keep tests small and focused to make them easier to understand and maintain. + + - ### 5.5. Mocking and Stubbing + + - **Use Mock Libraries**: Use mocking libraries like `Mock` or `Meck` to create mock objects and stub function calls. + - **Avoid Over-Mocking**: Avoid over-mocking, as it can make tests less realistic and harder to maintain. + - **Use Stubs for External Dependencies**: Use stubs to replace external dependencies with simplified versions that are easier to control during testing. + +- ## 6. Common Pitfalls and Gotchas + + - ### 6.1. Frequent Mistakes + + - **Not Using Contexts**: Directly accessing the Repo in controllers or other modules, bypassing the business logic encapsulation provided by contexts. + - **Ignoring Changeset Validation Errors**: Neglecting to handle and display changeset validation errors to the user, leading to confusing behavior. + - **N+1 Queries**: Failing to preload associations in Ecto queries, resulting in performance bottlenecks. + - **Over-reliance on Global State**: Using global state when it's not necessary, making code harder to reason about and test. + + - ### 6.2. Edge Cases + + - **Handling Timezones**: Dealing with timezones correctly, especially when storing and displaying dates and times to users in different locations. + - **Concurrency Issues**: Avoiding race conditions and other concurrency issues when using Elixir's concurrency features. + - **Large File Uploads**: Handling large file uploads efficiently and securely. + + - ### 6.3. Version-Specific Issues + + - **LiveView Updates**: Staying up-to-date with LiveView updates and understanding potential breaking changes. + - **Ecto Version Compatibility**: Ensuring compatibility between Ecto and other dependencies. + + - ### 6.4. Compatibility Concerns + + - **JavaScript Framework Integration**: Integrating Phoenix with JavaScript frameworks like React or Vue.js can introduce compatibility issues. + - **Database Driver Compatibility**: Ensuring compatibility between Ecto and the chosen database driver. + + - ### 6.5. Debugging Strategies + + - **Use IEx.pry**: Use `IEx.pry` to pause execution and inspect variables. + - **Enable Debug Logging**: Enable debug logging to get more information about what's happening in the application. + - **Use Remote Debugging**: Use remote debugging tools to debug applications running in production environments. + - **Analyze Logs**: Analyze logs to identify errors and performance issues. + +- ## 7. Tooling and Environment + + - ### 7.1. Recommended Development Tools + + - **Visual Studio Code**: VS Code with the ElixirLS extension provides excellent Elixir support, including code completion, linting, and debugging. + - **IntelliJ IDEA**: IntelliJ IDEA with the Elixir plugin also provides excellent Elixir support. + - **Mix**: Elixir's build tool, used for creating, compiling, testing, and managing dependencies. + - **IEx**: Elixir's interactive shell, used for exploring code and debugging. + - **Erlang Observer**: Erlang's GUI tool for monitoring and debugging Erlang/Elixir applications. + + - ### 7.2. Build Configuration + + - **Use `mix.exs`**: Use `mix.exs` to manage project dependencies, compilation settings, and other build configurations. + - **Define Environments**: Define separate environments for development, testing, and production. + - **Use Configuration Files**: Use configuration files (`config/config.exs`, `config/dev.exs`, `config/test.exs`, `config/prod.exs`) to configure the application for different environments. + - **Secrets Management**: Use environment variables or dedicated secrets management tools to store sensitive information like API keys and database passwords. + + - ### 7.3. Linting and Formatting + + - **Use `mix format`**: Use `mix format` to automatically format Elixir code according to a consistent style. + - **Use Credo**: Use Credo to analyze Elixir code for style and potential issues. + - **Configure Editor**: Configure the editor to automatically format code on save. + + - ### 7.4. Deployment + + - **Use Mix Release**: Use `mix release` to build and deploy Elixir applications. Distillery is deprecated. + - **Use Docker**: Use Docker to containerize Elixir applications for easy deployment and scaling. + - **Deploy to Cloud Platforms**: Deploy Elixir applications to cloud platforms like Heroku, AWS, Google Cloud, or Azure. + - **Use a Process Manager**: Use a process manager like `systemd` or `pm2` to manage Elixir application processes. + + - ### 7.5. CI/CD Integration + + - **Use GitHub Actions, GitLab CI, or CircleCI**: Use CI/CD platforms to automate the build, test, and deployment processes. + - **Run Tests on CI**: Run tests on CI to ensure that code changes don't introduce regressions. + - **Automate Deployments**: Automate deployments to production environments after successful tests. + +- ## Additional Resources + + - [Phoenix Framework Documentation](https://www.phoenixframework.org/docs) + - [Elixir Language Documentation](https://elixir-lang.org/docs.html) + - [Ecto Documentation](https://hexdocs.pm/ecto/Ecto.html) + - [Phoenix LiveView Documentation](https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.html) + - [Oban Documentation](https://github.com/sorentwo/oban) + - [Credo Documentation](https://hexdocs.pm/credo/Credo.html) \ No newline at end of file diff --git a/.cursor/rules/php.mdc b/.cursor/rules/php.mdc new file mode 100644 index 000000000..32d3fd474 --- /dev/null +++ b/.cursor/rules/php.mdc @@ -0,0 +1,334 @@ +--- +description: This rule provides guidelines for PHP coding best practices, covering code structure, security, performance, and testing to improve code quality and maintainability. +globs: *.php +--- +# PHP Best Practices and Coding Standards + +This document outlines best practices and coding standards for PHP development to ensure code quality, maintainability, security, and performance. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure Best Practices + +* **`src/`:** Contains the core source code of your application or library. +* **`public/`:** The document root of your application, containing publicly accessible files like `index.php`, CSS, JavaScript, and images. All requests should be routed through this directory. +* **`config/`:** Stores configuration files for different environments (development, testing, production). +* **`tests/`:** Contains unit, integration, and functional tests. +* **`vendor/`:** Managed by Composer, containing third-party libraries and dependencies. Do not modify contents manually. +* **`cache/`:** Directory used for caching data. +* **`logs/`:** Directory for storing application logs. Use descriptive names for log files. +* **`data/`:** Stores data files (e.g., SQLite databases). +* **`resources/`:** Contains view templates, language files, and other resources. +* **`migrations/`:** Database migration files for version control of your database schema. + + +project/ +├── src/ +│ ├── Controller/ +│ │ └── UserController.php +│ ├── Model/ +│ │ └── User.php +│ └── ... +├── public/ +│ └── index.php +├── config/ +│ └── config.php +├── tests/ +│ ├── Unit/ +│ │ └── UserTest.php +│ └── ... +├── vendor/ +│ └── ... +├── cache/ +│ └── ... +├── logs/ +│ └── app.log +├── data/ +│ └── database.sqlite +├── resources/ +│ ├── views/ +│ │ └── user.php +│ └── lang/ +│ └── en.php +└── migrations/ + └── 2024_01_01_create_users_table.php + + +### 1.2. File Naming Conventions + +* **Classes:** Use PascalCase (e.g., `UserController.php`). The filename should match the class name. +* **Functions and methods:** Use camelCase (e.g., `getUserById()`). +* **Variables:** Use camelCase (e.g., `$userName`). +* **Constants:** Use UPPER_SNAKE_CASE (e.g., `MAX_USER_AGE`). +* **Configuration files:** Use lowercase with underscores (e.g., `database_config.php`). +* **View files:** Use lowercase with hyphens (e.g., `user-profile.php`). + +### 1.3. Module Organization + +* **Separate concerns:** Organize code into modules based on functionality (e.g., authentication, user management, payment processing). +* **Use namespaces:** Group related classes and interfaces within namespaces to avoid naming conflicts and improve code organization (e.g., `namespace App\Controllers;`). Follow PSR-4 autoloading standards. +* **Dependency Injection:** Use dependency injection to decouple modules and make them easier to test and maintain. +* **Interfaces and Abstract Classes:** Define interfaces for modules to ensure loose coupling and allow for different implementations. + +### 1.4. Component Architecture + +* **Components:** Independent, reusable pieces of functionality that can be composed to build larger applications. +* **Thin Controllers:** Keep controllers light, delegate business logic to service classes or model layer. +* **Repositories:** Abstract data access logic from the rest of the application. This allows you to easily switch between different data sources (e.g., databases, APIs) without modifying the core application code. +* **Services:** Encapsulate complex business logic. + +### 1.5. Code Splitting + +* **Lazy loading:** Load modules or components only when they are needed to improve initial load time. +* **Composer autoloading:** Composer efficiently handles autoloading of classes based on namespaces, splitting code into separate files. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +* **Singleton:** Ensures that a class has only one instance and provides a global point of access to it. Use sparingly, as overuse can lead to tight coupling and make testing difficult. +* **Factory:** Creates objects without specifying the exact class to create. Useful for abstracting object creation. +* **Observer:** Defines a one-to-many dependency between objects so that when one object changes state, all its dependents are notified and updated automatically. Useful for event handling. +* **Strategy:** Defines a family of algorithms, encapsulates each one, and makes them interchangeable. Lets the algorithm vary independently from clients that use it. +* **Model-View-Controller (MVC):** Separates application logic (Model), user interface (View), and input handling (Controller). Most PHP frameworks are based on MVC. +* **Repository Pattern:** Abstract data access logic. Useful when you want to be able to easily switch data sources or when your data access logic is complex. + +### 2.2. Recommended Approaches + +* **Database interactions:** Use prepared statements with parameterized queries to prevent SQL injection. +* **Templating:** Use a templating engine like Twig or Blade to separate presentation logic from PHP code. Helps to maintain code clean and readable. +* **Routing:** Use a routing component to map URLs to controllers and actions. Simplifies URL handling. +* **Error handling:** Implement a consistent error handling strategy using exceptions and logging. +* **Dependency Injection:** Use a dependency injection container to manage dependencies and promote loose coupling. Frameworks often include DI containers. + +### 2.3. Anti-patterns and Code Smells + +* **Global state:** Avoid using global variables as they can lead to unpredictable behavior and make code difficult to test. +* **Tight coupling:** Minimize dependencies between classes and modules to improve reusability and maintainability. +* **Long methods/functions:** Break down large methods into smaller, more focused functions. +* **Copy-pasted code:** Extract common functionality into reusable functions or classes. +* **Ignoring errors:** Always handle exceptions and log errors appropriately. +* **Over-use of static methods:** Limit the use of static methods, which can make testing more difficult. +* **Code duplication:** Follow the DRY (Don't Repeat Yourself) principle. + +### 2.4. State Management + +* **Sessions:** Use PHP sessions for managing user authentication and temporary data. Configure session security settings (e.g., `session.cookie_httponly`, `session.cookie_secure`). +* **Cookies:** Use cookies for storing small amounts of data on the client-side. Be mindful of cookie size limits and security. +* **Caching:** Use caching mechanisms (e.g., Memcached, Redis, file-based caching) to improve performance by storing frequently accessed data. +* **Database:** Store persistent data in a database. Use appropriate data types and indexes for optimal performance. +* **Statelessness:** Design APIs to be stateless whenever possible to improve scalability. + +### 2.5. Error Handling + +* **Exceptions:** Use exceptions to handle errors and exceptional situations. Throw exceptions when something goes wrong. +* **Try-catch blocks:** Wrap code that might throw an exception in try-catch blocks to handle errors gracefully. +* **Custom exception classes:** Create custom exception classes to represent specific error conditions in your application. +* **Logging:** Log errors, warnings, and informational messages using a logging library (e.g., Monolog). Configure logging levels. +* **Error reporting:** Configure PHP error reporting settings (`error_reporting`, `display_errors`) appropriately for different environments. Disable displaying errors in production. +* **Global Exception Handler:** Set a global exception handler to catch uncaught exceptions and log them. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +* **Opcode caching:** Use an opcode cache like OPcache (built into PHP) to store compiled PHP code in memory, reducing the need to recompile code on each request. +* **Caching:** Implement caching strategies at different levels (e.g., database caching, object caching, page caching). +* **Database optimization:** Optimize database queries, use indexes, and avoid N+1 query problems. +* **Code profiling:** Use a profiler (e.g., Xdebug) to identify performance bottlenecks in your code. +* **Minimize file I/O:** Reduce the number of file read/write operations. +* **Use appropriate data structures:** Choose the right data structures for your needs (e.g., arrays, linked lists, hash tables). +* **Avoid resource-intensive operations:** Minimize the use of functions that consume a lot of resources (e.g., regular expressions). +* **Use PHP 7.x/8.x features:** Take advantage of performance improvements in newer versions of PHP. + +### 3.2. Memory Management + +* **Unset variables:** Unset variables that are no longer needed to free up memory. +* **Avoid large arrays:** Avoid storing large amounts of data in arrays. Use iterators or generators if possible. +* **Garbage collection:** PHP automatically manages memory using garbage collection. Ensure that your code doesn't create memory leaks (e.g., circular references). +* **Limit session size:** Be mindful of the amount of data stored in sessions, as large sessions can consume a lot of memory. + +### 3.3. Rendering Optimization + +* **Minimize HTTP requests:** Reduce the number of HTTP requests by combining CSS and JavaScript files, using CSS sprites, and inlining small images. +* **Optimize images:** Compress images and use appropriate image formats (e.g., JPEG, PNG, WebP). +* **Use browser caching:** Configure browser caching to store static assets (e.g., CSS, JavaScript, images) in the browser cache. +* **Content Delivery Network (CDN):** Use a CDN to serve static assets from geographically distributed servers. +* **Gzip compression:** Enable Gzip compression to reduce the size of HTTP responses. +* **Minify HTML, CSS, and JavaScript:** Remove unnecessary characters from HTML, CSS, and JavaScript files. + +### 3.4. Bundle Size Optimization (Less relevant for traditional PHP, but important for frontend assets) + +* **Code splitting:** Split frontend code into smaller chunks that can be loaded on demand. (Relevant if using a frontend framework like React, Vue, or Angular alongside PHP). +* **Tree shaking:** Remove unused code from your JavaScript bundles. (Relevant if using a frontend framework). + +### 3.5. Lazy Loading + +* **Lazy-load images:** Load images only when they are visible in the viewport. +* **Lazy-load modules:** Load modules or components only when they are needed. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities + +* **SQL injection:** Occurs when untrusted data is used to construct SQL queries. Prevent by using prepared statements with parameterized queries. +* **Cross-Site Scripting (XSS):** Occurs when malicious scripts are injected into a website. Prevent by sanitizing user input and escaping output. +* **Cross-Site Request Forgery (CSRF):** Occurs when a malicious website tricks a user into performing unwanted actions on another website. Prevent by using CSRF tokens. +* **Remote Code Execution (RCE):** Occurs when an attacker can execute arbitrary code on the server. Prevent by avoiding dangerous functions (e.g., `eval()`, `system()`) and carefully validating user input. +* **File Inclusion:** Occurs when an attacker can include arbitrary files on the server. Prevent by carefully validating user input and restricting file access. +* **Session Hijacking:** Occurs when an attacker steals a user's session ID. Prevent by using secure session management practices. +* **Denial of Service (DoS):** Occurs when an attacker overwhelms a server with requests, making it unavailable to legitimate users. Prevent by implementing rate limiting and other security measures. + +### 4.2. Input Validation + +* **Sanitize user input:** Remove or encode potentially dangerous characters from user input. +* **Validate user input:** Verify that user input conforms to expected formats and values. +* **Use whitelisting:** Allow only known good values for user input. Avoid blacklisting, which can be easily bypassed. +* **Escape output:** Escape output to prevent XSS attacks. + +### 4.3. Authentication and Authorization + +* **Use strong passwords:** Enforce strong password policies and use a secure password hashing algorithm (e.g., bcrypt, Argon2). +* **Salt passwords:** Add a unique salt to each password before hashing it. +* **Store passwords securely:** Store password hashes in a secure database. +* **Implement access control:** Restrict access to resources based on user roles and permissions. +* **Use multi-factor authentication (MFA):** Add an extra layer of security by requiring users to provide multiple forms of authentication. +* **Secure cookies:** Set the `HttpOnly` and `Secure` flags on cookies to prevent XSS and man-in-the-middle attacks. +* **Use HTTPS:** Encrypt all communication between the client and server using HTTPS. + +### 4.4. Data Protection + +* **Encrypt sensitive data:** Encrypt sensitive data at rest and in transit. +* **Use encryption keys securely:** Store encryption keys in a secure location and rotate them regularly. +* **Comply with data privacy regulations:** Comply with data privacy regulations like GDPR and CCPA. +* **Implement data masking:** Mask sensitive data when it is not needed for processing. +* **Regularly back up data:** Regularly back up data to protect against data loss. + +### 4.5. Secure API Communication + +* **Use HTTPS:** Enforce HTTPS for all API endpoints. +* **Authenticate API requests:** Use API keys, OAuth 2.0, or other authentication mechanisms to verify the identity of API clients. +* **Authorize API requests:** Restrict access to API endpoints based on user roles and permissions. +* **Rate limit API requests:** Implement rate limiting to prevent abuse. +* **Validate API input:** Validate all API input to prevent injection attacks. +* **Sanitize API output:** Sanitize all API output to prevent XSS attacks. +* **Log API requests:** Log all API requests for auditing and security monitoring. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +* **Test individual units of code:** Focus on testing individual functions, methods, or classes in isolation. +* **Use a testing framework:** Use a unit testing framework like PHPUnit or Codeception. +* **Write testable code:** Design code to be easily testable (e.g., use dependency injection). +* **Follow the Arrange-Act-Assert pattern:** Arrange the test data, act on the code being tested, and assert the expected results. +* **Test edge cases:** Test edge cases and boundary conditions to ensure that code handles unexpected input correctly. + +### 5.2. Integration Testing + +* **Test interactions between components:** Focus on testing how different components of the application interact with each other. +* **Use a testing framework:** Use an integration testing framework like PHPUnit or Codeception. +* **Test database interactions:** Test database interactions to ensure that data is stored and retrieved correctly. +* **Test API integrations:** Test API integrations to ensure that data is exchanged correctly. + +### 5.3. End-to-End Testing + +* **Test the entire application flow:** Focus on testing the entire application flow from start to finish. +* **Use a testing framework:** Use an end-to-end testing framework like Selenium or Cypress. +* **Automate tests:** Automate end-to-end tests to ensure that the application continues to work correctly as it evolves. + +### 5.4. Test Organization + +* **Create a `tests/` directory:** Store all test files in a `tests/` directory. +* **Mirror the source code structure:** Organize test files to mirror the source code structure (e.g., `tests/Unit/UserControllerTest.php`). +* **Use namespaces:** Use namespaces to organize test classes. +* **Use descriptive test names:** Use descriptive test names that clearly indicate what is being tested. + +### 5.5. Mocking and Stubbing + +* **Use mocks to isolate units of code:** Use mocks to replace dependencies with controlled test doubles. +* **Use stubs to provide canned responses:** Use stubs to provide canned responses from dependencies. +* **Use a mocking framework:** Use a mocking framework like Mockery or Prophecy. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +* **Not escaping output:** Failing to escape output, leading to XSS vulnerabilities. +* **Using `mysql_*` functions:** Using deprecated `mysql_*` functions instead of `mysqli_*` or PDO. +* **Not using prepared statements:** Not using prepared statements with parameterized queries, leading to SQL injection vulnerabilities. +* **Ignoring errors:** Ignoring errors and warnings, making it difficult to debug problems. +* **Not using namespaces:** Not using namespaces, leading to naming conflicts. +* **Over-complicating code:** Writing overly complex code that is difficult to understand and maintain. +* **Not following coding standards:** Not following coding standards, leading to inconsistent code. + +### 6.2. Edge Cases + +* **Handling different character encodings:** Handling different character encodings correctly. +* **Dealing with time zones:** Dealing with time zones correctly. +* **Handling large file uploads:** Handling large file uploads without running out of memory. +* **Dealing with concurrency:** Handling concurrency correctly in multi-threaded applications (less common in typical web applications). + +### 6.3. Version-Specific Issues + +* **Deprecated features:** Being aware of deprecated features and replacing them with their recommended alternatives. +* **Compatibility issues:** Ensuring that code is compatible with different versions of PHP. + +### 6.4. Compatibility Concerns + +* **Database drivers:** Ensuring that the correct database drivers are installed and configured. +* **Web server configuration:** Configuring the web server (e.g., Apache, Nginx) correctly to serve PHP applications. +* **Operating system compatibility:** Ensuring that the application works correctly on different operating systems. + +### 6.5. Debugging Strategies + +* **Use a debugger:** Use a debugger (e.g., Xdebug) to step through code and inspect variables. +* **Log messages:** Insert log messages to track the execution flow and identify problems. +* **Use `var_dump()` or `print_r()`:** Use `var_dump()` or `print_r()` to inspect variables and data structures (use with caution in production). +* **Read error messages:** Carefully read error messages and warnings to understand the cause of problems. +* **Use a linter:** Use a linter to identify potential errors and code style violations. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +* **IDE:** PhpStorm, VS Code with PHP extensions. +* **Debugger:** Xdebug. +* **Package manager:** Composer. +* **Database client:** Dbeaver, phpMyAdmin. +* **Version control:** Git. +* **Virtualization:** Docker, Vagrant. +* **Profiler:** Xdebug, Blackfire.io + +### 7.2. Build Configuration + +* **Use Composer:** Use Composer to manage dependencies. +* **Define dependencies in `composer.json`:** List all dependencies in the `composer.json` file. +* **Use semantic versioning:** Use semantic versioning to specify dependency versions. +* **Use a `composer.lock` file:** Commit the `composer.lock` file to version control to ensure that everyone is using the same versions of dependencies. +* **Configure autoloading:** Configure autoloading to automatically load classes. + +### 7.3. Linting and Formatting + +* **Use a linter:** Use a linter (e.g., PHPStan, Psalm) to identify potential errors and code style violations. +* **Use a code formatter:** Use a code formatter (e.g., PHP CS Fixer) to automatically format code according to coding standards. +* **Configure linting and formatting rules:** Configure linting and formatting rules to enforce coding standards. +* **Integrate linting and formatting into the development workflow:** Integrate linting and formatting into the development workflow to automatically check code for errors and style violations. + +### 7.4. Deployment + +* **Use version control:** Use version control to track changes to code. +* **Automate deployments:** Automate deployments using a deployment tool (e.g., Deployer, Capistrano). +* **Use environment variables:** Use environment variables to configure application settings for different environments. +* **Separate code and configuration:** Separate code and configuration to make it easier to deploy applications to different environments. +* **Use a build process:** Use a build process to prepare code for deployment (e.g., run linters, formatters, and tests). +* **Test deployments:** Test deployments in a staging environment before deploying to production. + +### 7.5. CI/CD + +* **Use a CI/CD platform:** Use a CI/CD platform (e.g., Jenkins, Travis CI, CircleCI, GitHub Actions) to automate the build, test, and deployment process. +* **Configure CI/CD pipelines:** Configure CI/CD pipelines to run tests, linters, and formatters automatically on every commit. +* **Automate deployments:** Automate deployments using CI/CD pipelines. +* **Use infrastructure as code:** Use infrastructure as code (e.g., Terraform, Ansible) to manage infrastructure. + +This document aims to provide a comprehensive overview of PHP best practices and coding standards. By following these guidelines, developers can create high-quality, maintainable, secure, and performant PHP applications. \ No newline at end of file diff --git a/.cursor/rules/pillow.mdc b/.cursor/rules/pillow.mdc new file mode 100644 index 000000000..2a2dfc13e --- /dev/null +++ b/.cursor/rules/pillow.mdc @@ -0,0 +1,169 @@ +--- +description: This rule provides best practices for using the Pillow image processing library in Python, covering code organization, performance, security, testing, and common pitfalls. It aims to help developers write maintainable, efficient, and secure image processing applications. +globs: *.py +--- +# Pillow Library Best Practices and Coding Standards + +This document outlines the recommended best practices and coding standards for effectively using the Pillow library in Python for image processing tasks. These guidelines aim to promote maintainable, efficient, and secure applications. + +## 1. Installation and Setup + +- **Use a Virtual Environment:** Always create a virtual environment for your Pillow projects to isolate dependencies and avoid conflicts. Use `python -m venv .venv` and activate it. +- **Install with pip:** Install Pillow using `pip install Pillow`. Consider using `pip install -U Pillow` for upgrades. +- **Verify Installation:** Verify the installation using `python -c "from PIL import Image; print(Image.VERSION)"`. This also serves as a smoke test for the library. +- **Pin Dependencies:** Specify Pillow and other dependencies in a `requirements.txt` file, pinning the versions for reproducible builds. Use `pip freeze > requirements.txt`. +- **Install dependencies with UV:** Use the uv package manager when installing dependencies: `pip install uv` then `uv pip install -r requirements.txt`. + +## 2. Code Organization and Structure + +- **Directory Structure:** Organize your project into logical directories. A typical structure might include: + + project_name/ + ├── src/ + │ ├── __init__.py + │ ├── image_processing.py # Pillow-related functions + │ ├── utils.py # Helper functions + ├── tests/ + │ ├── __init__.py + │ ├── test_image_processing.py + ├── data/ # Input and output images + ├── notebooks/ # Jupyter notebooks for experimentation + ├── requirements.txt + ├── pyproject.toml # Optional: For poetry or flit + ├── README.md + +- **File Naming Conventions:** Use descriptive and consistent file names: + - Python modules: `image_utils.py`, `image_filters.py` + - Test files: `test_image_utils.py`, `test_image_filters.py` + - Image data: `input_image.jpg`, `output_image.png` +- **Module Organization:** Group related functions and classes into modules. For example: + - `image_processing.py`: Contains functions for core image manipulations (resizing, cropping, color conversions). + - `image_filters.py`: Contains functions for applying image filters (blur, sharpen, edge detection). + - `image_io.py`: Contains functions for image loading and saving. +- **Component Architecture:** Consider using a component-based architecture for larger projects: + - **Data Access Layer:** Handles image loading and saving. + - **Business Logic Layer:** Implements image processing algorithms. + - **Presentation Layer:** Provides a user interface (if applicable) or API endpoint. +- **Code Splitting:** For large image processing pipelines, split the code into smaller, manageable functions. Use generators for lazy processing of large image datasets. + +## 3. Coding Standards and Best Practices + +- **Importing:** Always import Pillow using `from PIL import Image`. This is the standard and recommended way. +- **Explicit Imports:** Use explicit imports (`from PIL import Image, ImageFilter`) rather than wildcard imports (`from PIL import *`). This improves code readability and avoids namespace pollution. +- **Clear Naming:** Use descriptive variable and function names: + - `image = Image.open("input.jpg")` + - `resized_image = image.resize((width, height))` +- **Error Handling:** Use `try...except` blocks to handle potential errors, such as `FileNotFoundError` when opening images or `IOError` when saving images. +- **Context Managers:** Use `with` statements to ensure proper resource management (especially file closing) when working with images: + python + try: + with Image.open("input.jpg") as image: + # Process the image + image.save("output.png") + except FileNotFoundError: + print("Error: Input image not found.") + except IOError: + print("Error: Could not save the image.") + +- **Image Modes:** Be mindful of image modes (RGB, RGBA, L, CMYK) and convert images to the appropriate mode when necessary using `image.convert("RGB")`. +- **Resampling Filters:** Choose appropriate resampling filters for resizing operations: + - `Image.LANCZOS`: High-quality resampling for downscaling. + - `Image.NEAREST`: Fastest resampling, but lowest quality. + - `Image.BILINEAR`, `Image.BICUBIC`: Intermediate quality and speed. + - Use `Image.Resampling.LANCZOS` (or `Image.Resampling.BOX`, `Image.Resampling.NEAREST`, etc.) if using Pillow 9.2.0 or newer. +- **Saving Images:** Always specify the format when saving images, especially if the filename extension is ambiguous. Use `image.save("output.png", format="PNG")`. +- **Documentation:** Write clear and concise docstrings for all functions and classes, explaining their purpose, arguments, and return values. +- **Type Hints:** Use type hints to improve code readability and help catch errors early. + python + from PIL import Image + +def resize_image(image_path: str, width: int, height: int) -> Image.Image: + """Resizes an image to the specified dimensions.""" + with Image.open(image_path) as image: + resized_image = image.resize((width, height)) + return resized_image + +- **Linting and Formatting:** Use a linter (e.g., pylint, flake8) and a formatter (e.g., black, autopep8) to maintain consistent code style. + +## 4. Common Patterns and Anti-patterns + +- **Factory Pattern:** Use a factory pattern to create different types of image objects based on input data. +- **Strategy Pattern:** Implement different image processing algorithms as strategies, allowing you to easily switch between them. +- **Anti-pattern: Ignoring Errors:** Avoid ignoring exceptions without proper handling. Always log errors or raise them appropriately. +- **Anti-pattern: Excessive Memory Usage:** Avoid loading entire image datasets into memory at once. Use generators or iterators for large datasets. +- **State Management:** For applications with image editing features, use a state management system to track changes and enable undo/redo functionality. Redux, Zustand, or custom state management are possibilities. +- **Error Handling Patterns:** Use specific exception types (e.g., `FileNotFoundError`, `ValueError`) to handle different error scenarios. Provide informative error messages to the user or log them for debugging. + +## 5. Performance Considerations + +- **Optimize Image Formats:** Choose the appropriate image format for your needs. JPEG is good for photographs, PNG is good for images with transparency or sharp edges, and WebP offers good compression and quality. +- **Image Optimization Libraries:** Utilize libraries like `optipng` or `jpegoptim` to further optimize images for web delivery. These tools can reduce file size without significant quality loss. +- **Lazy Loading:** Load images only when they are needed, especially for web applications. Use placeholders or low-resolution previews while the full image is loading. +- **Caching:** Cache frequently accessed images to reduce loading times. Use a memory-based cache (e.g., `functools.lru_cache`) or a disk-based cache (e.g., `diskcache`). +- **Thumbnail Generation:** Generate thumbnails for large images to improve performance in image galleries or previews. Use `image.thumbnail()`. +- **Efficient Resizing:** Use `image.thumbnail()` when creating thumbnails as it preserves aspect ratio. If ignoring aspect ratio, use `image.resize()` but be mindful of performance implications. +- **Memory Management:** + - **Use `del` to release memory:** Explicitly delete large image objects when they are no longer needed using `del image`. + - **Limit Image Size:** Avoid loading extremely large images that can consume excessive memory. Consider resizing images to a reasonable size before processing. + - **Chunked Processing:** For very large images, process them in chunks to reduce memory usage. +- **Rendering Optimization:** When displaying images in a GUI application, use optimized rendering techniques to improve performance. Consider using hardware acceleration if available. +- **Asynchronous Processing:** Offload image processing tasks to separate threads or processes to avoid blocking the main thread. Use `threading` or `multiprocessing` modules. +- **Numpy integration:** Convert Pillow images to NumPy arrays for faster calculations. + +## 6. Security Best Practices + +- **Input Validation:** Validate all image file names and paths to prevent directory traversal attacks and other security vulnerabilities. Use `os.path.abspath()` and `os.path.normpath()` to sanitize paths. +- **File Extension Validation:** Verify that the file extension matches the actual image format. Don't rely solely on the extension. +- **Limit File Size:** Limit the maximum file size of uploaded images to prevent denial-of-service attacks. +- **Prevent Image Bomb Attacks:** Implement checks to prevent image bomb attacks, which involve maliciously crafted images designed to consume excessive resources. +- **Use a Security Linter:** Incorporate a security linter (e.g., bandit) into your CI/CD pipeline to identify potential security vulnerabilities in your code. +- **Authentication and Authorization:** Implement proper authentication and authorization mechanisms to protect access to image processing services. +- **Data Protection:** Encrypt sensitive image data at rest and in transit. +- **Avoid Executing Untrusted Code:** Be cautious when using Pillow to process images from untrusted sources, as vulnerabilities in the library could be exploited. Regularly update Pillow to the latest version. +- **Regularly Update Pillow:** Stay up-to-date with the latest Pillow releases to patch security vulnerabilities. +- **Safe Image Handling:** Consider using a sandboxed environment to handle untrusted images, further isolating potential security risks. + +## 7. Testing Approaches + +- **Unit Testing:** Write unit tests for individual functions and classes in your Pillow-related code. Use a testing framework like `pytest` or `unittest`. + - **Test Image Transformations:** Verify that image transformations (resizing, cropping, color conversions) produce the expected results. + - **Test Error Handling:** Ensure that your code handles errors gracefully (e.g., invalid image files, incorrect parameters). +- **Integration Testing:** Test the interaction between different components of your image processing pipeline. +- **End-to-End Testing:** Verify that the entire application works as expected, including image loading, processing, and saving. +- **Test Organization:** Organize your tests into logical directories that mirror your code structure. +- **Mocking and Stubbing:** Use mocking and stubbing to isolate components during testing. For example, mock the `Image.open()` function to avoid accessing real image files during unit tests. +- **Property-based Testing:** Use property-based testing (e.g., Hypothesis) to automatically generate test cases and verify that your code satisfies certain properties. +- **Golden Image Tests:** Compare the output of your image processing functions with known "golden" images to ensure that the results are consistent. + +## 8. Common Pitfalls and Gotchas + +- **Incorrect Image Mode:** Forgetting to convert images to the correct mode (e.g., RGB) before performing certain operations can lead to unexpected results. +- **Integer Division:** Be aware of integer division in Python 2. Use `from __future__ import division` to ensure that division always returns a float. +- **File Closing:** Forgetting to close image files after processing can lead to resource leaks. Use `with` statements to ensure that files are closed automatically. +- **Out-of-Memory Errors:** Processing extremely large images can lead to out-of-memory errors. Use techniques like chunked processing and lazy loading to avoid this. +- **Color Profile Issues:** Be aware of color profile issues when working with images that have embedded color profiles. Consider using `image.convert('RGB')` to strip color profiles or use a color management library like `littlecms`. +- **Pillow Version Compatibility:** Be aware of compatibility issues between different Pillow versions. Check the Pillow documentation for version-specific changes and bug fixes. +- **Floating point errors:** Be aware of the loss of precision due to floating point math and how to properly fix it. + +## 9. Tooling and Environment + +- **Development Tools:** + - **IDE:** Use a powerful IDE like VS Code with the Python extension or PyCharm. + - **Debugging:** Use a debugger to step through your code and identify errors. + - **Profiling:** Use a profiler to identify performance bottlenecks. +- **Build Configuration:** + - **`setup.py`:** Use `setup.py` or `pyproject.toml` to manage your project's dependencies and build process. + - **Virtual Environments:** Use virtual environments to isolate your project's dependencies. +- **Linting and Formatting:** + - **Pylint:** Use Pylint to check your code for style errors and potential problems. + - **Black:** Use Black to automatically format your code to a consistent style. + - **Flake8:** Use Flake8 to check your code for style errors and potential problems. +- **Deployment:** + - **Docker:** Use Docker to containerize your application and ensure consistent deployment across different environments. + - **Cloud Platforms:** Deploy your application to a cloud platform like AWS, Google Cloud, or Azure. +- **CI/CD Integration:** + - **GitHub Actions:** Use GitHub Actions to automate your build, test, and deployment process. + - **Jenkins:** Use Jenkins for continuous integration and continuous delivery. + - **CircleCI:** Use CircleCI for continuous integration and continuous delivery. + +By following these best practices and coding standards, you can create robust, efficient, and secure image processing applications using the Pillow library in Python. \ No newline at end of file diff --git a/.cursor/rules/playwright.mdc b/.cursor/rules/playwright.mdc new file mode 100644 index 000000000..7914bb958 --- /dev/null +++ b/.cursor/rules/playwright.mdc @@ -0,0 +1,151 @@ +--- +description: This rule enforces best practices and coding standards for Playwright tests, including stable selectors, test isolation, user-centric testing, and performance considerations. +globs: *.spec.ts +--- +- **General Principles** + - **Test User-Visible Behavior:** Focus tests on how users interact with your application, not on internal implementation details. + - **Isolate Tests:** Ensure tests are independent of each other to prevent cascading failures and ensure predictable results. + - **Avoid Testing Third-Party Dependencies:** Mock or stub external services and APIs to isolate your application's behavior. + +- **Code Organization and Structure** + - **Directory Structure:** + - `tests/`: Contains all test files. + - `tests/e2e/`: End-to-end tests. + - `tests/unit/`: Unit tests (if applicable, though Playwright is primarily for E2E). + - `tests/utils/`: Helper functions and page object models. + - **File Naming Conventions:** + - Use `.spec.ts` or `.spec.js` for test files (e.g., `login.spec.ts`). + - Group related tests in the same file. + - **Module Organization:** + - Employ Page Object Model (POM) to encapsulate UI elements and interactions. + - **Component Architecture:** + - Structure tests around components or features of your application. + - **Code Splitting Strategies:** + - Not directly applicable to tests, but keep test files concise and focused. + +- **Common Patterns and Anti-patterns** + - **Design Patterns:** + - **Page Object Model (POM):** A common pattern where each page is represented as a class, with methods for interacting with the page's elements. This improves reusability and maintainability. Example: + typescript + class LoginPage { + constructor(private readonly page: Page) {} + + async goto() { + await this.page.goto('/login'); + } + + async login(username: string, password: string) { + await this.page.fill('#username', username); + await this.page.fill('#password', password); + await this.page.click('#login-button'); + } + + async getErrorMessage() { + return await this.page.textContent('#error-message'); + } + } + + - **Fixture pattern:** Use Playwright's built-in fixtures to manage test setup and teardown. This ensures each test starts in a clean state. + - **Recommended Approaches:** + - Use `baseURL` in `playwright.config.ts` to avoid hardcoding URLs in tests. + - Utilize `expect` matchers for assertions (e.g., `expect(page.locator('#success')).toBeVisible()`). + - Use auto-waiting features for improved stability. + - **Anti-patterns:** + - Hardcoding URLs. + - Using brittle selectors (e.g., XPath based on DOM structure). + - Writing tests that depend on each other. + - **State Management:** + - Keep tests stateless. Reset the application state before each test. + - Use database transactions or API calls to seed data for tests. + - **Error Handling:** + - Use `try...catch` blocks to handle expected errors. + - Log errors and failures with descriptive messages. + - Use `expect.soft()` for non-critical assertions that shouldn't fail the test immediately. + +- **Performance Considerations** + - **Optimization Techniques:** + - Run tests in parallel to reduce overall test execution time. + - Use `reuseExistingServer: true` in `playwright.config.ts` during development to speed up debugging. + - Use `codegen` to generate selectors automatically. + - **Memory Management:** + - Close pages and browsers after each test or group of tests to release resources. + - **Rendering Optimization:** + - Not directly applicable but optimize your application's rendering for faster testing. + - **Bundle Size Optimization:** + - Not directly applicable, but optimize your application's bundle size for faster loading. + - **Lazy Loading Strategies:** + - Not directly applicable to tests. + +- **Security Best Practices** + - **Common Vulnerabilities:** + - Avoid exposing sensitive data (e.g., passwords, API keys) in test code or logs. + - **Input Validation:** + - Test input validation to ensure your application handles invalid data correctly. + - **Authentication and Authorization:** + - Test different user roles and permissions. + - **Data Protection:** + - Ensure sensitive data is encrypted in the database. + - **Secure API Communication:** + - Test that API calls are made over HTTPS. + +- **Testing Approaches** + - **Unit Testing:** + - While Playwright primarily focuses on E2E testing, unit tests can be written for utility functions or components. + - **Integration Testing:** + - Test the interaction between different parts of your application. + - **End-to-End Testing:** + - Simulate user flows to test the entire application. + - **Test Organization:** + - Group tests by feature or functionality. + - Use `describe` blocks to organize tests. + - **Mocking and Stubbing:** + - Use Playwright's `route` API to mock API responses. + - Use `locator.evaluate` to stub JavaScript functions. + +- **Common Pitfalls and Gotchas** + - **Frequent Mistakes:** + - Using XPath instead of CSS selectors. + - Not using auto-waiting features. + - Writing flaky tests. + - **Edge Cases:** + - Handling different screen sizes and devices. + - Testing error conditions and edge cases. + - **Version-Specific Issues:** + - Stay up-to-date with Playwright's release notes and upgrade guides. + - **Compatibility Concerns:** + - Test on different browsers and operating systems. + - **Debugging Strategies:** + - Use Playwright Inspector to debug tests visually. + - Use `console.log` statements to log information during test execution. + - Use `pause()` to halt test execution and inspect the page. + +- **Tooling and Environment** + - **Recommended Development Tools:** + - VS Code with the Playwright extension. + - **Build Configuration:** + - Use TypeScript for type safety and autocompletion. + - **Linting and Formatting:** + - Use ESLint and Prettier to enforce code style. + - **Deployment Best Practices:** + - Run tests in CI/CD pipeline before deploying to production. + - **CI/CD Integration:** + - Integrate Playwright with CI/CD tools like GitHub Actions, Jenkins, or GitLab CI. + +- **Specific Best Practices & Details** + - **Stable Selectors:** Prefer CSS selectors based on attributes like `data-testid` or `data-test-id` over XPath or fragile CSS classnames. + - **Leverage Auto-waiting:** Playwright automatically waits for elements to be actionable before performing actions. Avoid explicit waits where possible. However, use explicit waits (e.g. `waitForSelector`) when necessary. + - **Web-First Assertions:** Use `expect` assertions, which retry and wait for conditions to be met. They help to avoid flakiness. + - **Configure Debugging Highlights:** Configure `playwright.config.ts` to highlight actions performed by playwright in the browser during debugging to see what's happening step by step. Example: + typescript + use: { + /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */ + trace: 'on-first-retry', + video: 'on', + screenshot: 'only-on-failure', + } + + +- **Additional Notes** + - Regularly review and update your test suite to reflect changes in your application. + - Document your tests to make them easier to understand and maintain. + - Use a consistent naming convention for your tests. \ No newline at end of file diff --git a/.cursor/rules/plotly.mdc b/.cursor/rules/plotly.mdc new file mode 100644 index 000000000..8f57f3282 --- /dev/null +++ b/.cursor/rules/plotly.mdc @@ -0,0 +1,343 @@ +--- +description: This rule file provides best practices and coding standards for using the Plotly library, focusing on code organization, performance, security, testing, and common pitfalls. It aims to guide developers in creating maintainable, efficient, and secure Plotly applications. +globs: *.py +--- +# Plotly Best Practices and Coding Standards + +This document provides comprehensive guidelines for using the Plotly library in Python for AI, ML, data science, and other development contexts. It covers various aspects, including code organization, common patterns, performance considerations, security best practices, testing approaches, common pitfalls, and tooling. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure Best Practices + +Organizing your project with a clear and consistent directory structure enhances maintainability and collaboration. Here's a suggested structure: + + +project_root/ +├── data/ +│ ├── raw/ +│ ├── processed/ +│ └── external/ +├── src/ +│ ├── visualizations/ +│ │ ├── __init__.py +│ │ ├── charts.py # Contains functions for creating different charts +│ │ ├── layouts.py # Defines layouts and themes +│ │ └── utils.py # Utility functions for visualizations +│ ├── models/ +│ │ ├── __init__.py +│ │ ├── model_training.py +│ │ └── model_evaluation.py +│ ├── data_processing/ +│ │ ├── __init__.py +│ │ ├── data_cleaning.py +│ │ └── feature_engineering.py +│ ├── utils/ +│ │ ├── __init__.py +│ │ ├── config.py # Configuration settings +│ │ └── helpers.py # General helper functions +│ └── main.py # Entry point of the application +├── tests/ +│ ├── visualizations/ +│ ├── models/ +│ ├── data_processing/ +│ └── conftest.py # Pytest configuration file +├── notebooks/ # Jupyter notebooks for exploration +├── requirements.txt # Project dependencies +├── pyproject.toml # Configuration file for dependencies +└── README.md + + +* `data/`: Stores data-related files. + * `raw/`: Contains the original, untouched data. + * `processed/`: Stores cleaned and transformed data. + * `external/`: Includes data from external sources. +* `src/`: Houses the source code of your application. + * `visualizations/`: Contains modules for creating Plotly charts and layouts. + * `models/`: Includes modules for training and evaluating machine learning models. + * `data_processing/`: Contains modules for data cleaning and feature engineering. + * `utils/`: Contains utility modules, such as configuration settings and helper functions. + * `main.py`: The entry point of your application. +* `tests/`: Stores unit tests for different modules. +* `notebooks/`: Contains Jupyter notebooks for data exploration and prototyping. +* `requirements.txt`: Lists project dependencies. +* `README.md`: Provides a high-level overview of the project. + +### 1.2. File Naming Conventions + +Follow consistent naming conventions for files and directories to improve readability and maintainability: + +* Use lowercase letters for file names. +* Separate words with underscores (`_`). +* Be descriptive and concise. + +Example: + +* `charts.py` +* `data_cleaning.py` +* `feature_engineering.py` +* `model_training.py` + +### 1.3. Module Organization Best Practices + +Organize your code into modules based on functionality. For Plotly-specific code, consider the following modules: + +* `charts.py`: Contains functions for creating different types of Plotly charts (e.g., scatter plots, bar charts, line charts). +* `layouts.py`: Defines layouts and themes for Plotly charts. +* `utils.py`: Includes utility functions for visualizations, such as data formatting and color palettes. + +Example `charts.py`: + +python +import plotly.express as px + +def create_scatter_plot(df, x_col, y_col, color_col=None, title="Scatter Plot"): + """Creates a scatter plot using Plotly Express.""" + fig = px.scatter(df, x=x_col, y=y_col, color=color_col, title=title) + return fig + +def create_bar_chart(df, x_col, y_col, color_col=None, title="Bar Chart"): + """Creates a bar chart using Plotly Express.""" + fig = px.bar(df, x=x_col, y=y_col, color=color_col, title=title) + return fig + +# Add other chart functions here + + +### 1.4. Component Architecture Recommendations + +For larger applications, consider a component-based architecture. This involves breaking down the application into reusable components. Components can be: + +* Chart components: Reusable functions or classes for creating specific types of charts. +* Layout components: Reusable layouts and themes. +* Data components: Functions or classes for fetching and processing data. + +Example: + +python +# visualizations/components.py + +import plotly.graph_objects as go + +class ScatterPlot: + def __init__(self, df, x_col, y_col, color_col=None, title="Scatter Plot"): + self.df = df + self.x_col = x_col + self.y_col = y_col + self.color_col = color_col + self.title = title + + def create_plot(self): + fig = go.Figure(data=go.Scatter(x=self.df[self.x_col], y=self.df[self.y_col], mode='markers', marker=dict(color=self.df[self.color_col] if self.color_col else None))) + fig.update_layout(title=self.title, xaxis_title=self.x_col, yaxis_title=self.y_col) + return fig + + +### 1.5. Code Splitting Strategies + +For web applications built with Dash, code splitting can improve performance by loading only the necessary code for each page or component. Strategies include: + +* Using Dash's `dcc.Location` component to conditionally render different components based on the URL. +* Implementing lazy loading for large datasets or complex visualizations. +* Breaking down your Dash app into multiple files and importing only the necessary components. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns Specific to Plotly + +* **Factory Pattern:** Use a factory function to create different types of charts based on input parameters. This simplifies the creation of charts and promotes code reuse. +* **Template Method Pattern:** Define a base class for chart components with a template method that outlines the steps for creating a chart. Subclasses can then implement specific steps. +* **Observer Pattern:** In Dash applications, use the Observer pattern to update charts dynamically based on user interactions or data changes. + +### 2.2. Recommended Approaches for Common Tasks + +* **Creating Interactive Charts:** Use `plotly.express` for quick interactive charts. For more control, use `plotly.graph_objects`. +* **Updating Chart Layouts:** Use `fig.update_layout()` to modify chart titles, axes labels, legends, and annotations. +* **Handling Large Datasets:** Use techniques like data aggregation, sampling, or WebGL rendering to improve performance. +* **Creating Dashboards:** Use the Dash framework to build interactive web applications with Plotly charts. + +### 2.3. Anti-patterns and Code Smells + +* **Overloading Charts:** Avoid including too much information in a single chart, which can make it difficult to interpret. +* **Ignoring Performance:** Neglecting to optimize charts for large datasets can lead to slow rendering and poor user experience. +* **Hardcoding Values:** Avoid hardcoding values in your chart definitions. Use configuration files or environment variables instead. +* **Not Handling Errors:** Failing to handle errors can lead to unexpected behavior and poor user experience. +* **Inconsistent Styling:** Ensure consistent styling across all charts in your application. + +### 2.4. State Management Best Practices for Plotly Applications (Dash) + +* **Use Dash's `dcc.Store` component:** Store application-wide state in `dcc.Store` components. This allows you to share state between different callbacks. +* **Avoid Storing Large Datasets in State:** Instead, load data on-demand or use a server-side caching mechanism. +* **Use Callbacks to Update State:** Only update state in response to user interactions or data changes. +* **Immutable State Updates:** Treat state as immutable and create new state objects instead of modifying existing ones. + +### 2.5. Error Handling Patterns + +* **Use `try-except` Blocks:** Wrap Plotly code in `try-except` blocks to catch potential exceptions, such as data errors or rendering issues. +* **Log Errors:** Log error messages to a file or console for debugging purposes. +* **Display User-Friendly Error Messages:** Display informative error messages to the user instead of showing raw exceptions. +* **Implement Fallback Mechanisms:** Provide fallback mechanisms in case of chart rendering failures, such as displaying a static image or a placeholder message. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +* **Data Aggregation:** Aggregate data to reduce the number of data points displayed in the chart. +* **Sampling:** Use sampling techniques to reduce the size of large datasets. +* **WebGL Rendering:** Utilize Plotly's WebGL backend for faster rendering of complex 3D and dense 2D plots. Add `renderer="webgl"` argument in the `show()` function +* **Caching:** Implement caching mechanisms to store frequently accessed data and reduce the number of database queries or API calls. + +### 3.2. Memory Management Considerations + +* **Release Unused Resources:** Ensure that you release any unused resources, such as large datasets or chart objects, to prevent memory leaks. +* **Use Generators:** Use generators to process large datasets in chunks instead of loading the entire dataset into memory. +* **Avoid Creating Unnecessary Copies:** Avoid creating unnecessary copies of data objects, as this can consume additional memory. + +### 3.3. Rendering Optimization + +* **Reduce Chart Complexity:** Simplify your charts by reducing the number of traces, data points, or annotations. +* **Use Efficient Data Structures:** Use efficient data structures, such as NumPy arrays or Pandas DataFrames, to store and manipulate data. +* **Optimize Callback Logic:** Optimize the logic in your Dash callbacks to reduce the amount of time spent processing data or updating charts. + +### 3.4. Bundle Size Optimization Strategies + +* **Use a Virtual Environment:** Create a virtual environment for your project to isolate dependencies and reduce the overall bundle size. +* **Remove Unused Dependencies:** Remove any unused dependencies from your project. +* **Minify JavaScript and CSS:** Minify JavaScript and CSS files to reduce their size. +* **Use Code Splitting:** Split your code into multiple bundles and load only the necessary code for each page or component. + +### 3.5. Lazy Loading Strategies + +* **Load Data On-Demand:** Load data only when it is needed, such as when a user interacts with a chart or navigates to a new page. +* **Use Asynchronous Loading:** Use asynchronous loading techniques to load data in the background without blocking the main thread. +* **Implement Placeholder Content:** Display placeholder content while data is loading to improve the user experience. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities and Prevention + +* **Cross-Site Scripting (XSS):** Prevent XSS attacks by sanitizing user inputs and escaping any data displayed in the charts. +* **Injection Attacks:** Prevent injection attacks by using parameterized queries or ORM frameworks to interact with databases. +* **Denial of Service (DoS):** Implement rate limiting and input validation to prevent DoS attacks. + +### 4.2. Input Validation + +* **Validate User Inputs:** Validate user inputs to ensure that they conform to the expected format and range. +* **Sanitize User Inputs:** Sanitize user inputs to remove any potentially malicious code or characters. +* **Use Whitelisting:** Use whitelisting to allow only specific characters or patterns in user inputs. + +### 4.3. Authentication and Authorization + +* **Implement Authentication:** Implement authentication to verify the identity of users accessing your application. +* **Use Strong Passwords:** Enforce the use of strong passwords and implement password hashing and salting. +* **Implement Authorization:** Implement authorization to control access to different parts of your application based on user roles or permissions. + +### 4.4. Data Protection Strategies + +* **Encrypt Sensitive Data:** Encrypt sensitive data at rest and in transit to protect it from unauthorized access. +* **Use Secure Protocols:** Use secure protocols, such as HTTPS, to communicate with servers and APIs. +* **Implement Access Controls:** Implement access controls to restrict access to sensitive data based on user roles or permissions. + +### 4.5. Secure API Communication + +* **Use API Keys:** Use API keys to authenticate requests to external APIs. +* **Validate API Responses:** Validate API responses to ensure that they are in the expected format and do not contain any malicious content. +* **Implement Rate Limiting:** Implement rate limiting to prevent abuse of your APIs. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +* **Test Individual Components:** Write unit tests to test individual components of your Plotly code, such as chart functions or layout definitions. +* **Use Mock Data:** Use mock data to isolate components from external dependencies. +* **Verify Chart Properties:** Verify that the chart properties, such as titles, axes labels, and data values, are correct. + +### 5.2. Integration Testing + +* **Test Interactions Between Components:** Write integration tests to test the interactions between different components of your application. +* **Use a Test Database:** Use a test database to isolate your tests from the production database. +* **Verify Data Flow:** Verify that data flows correctly between components. + +### 5.3. End-to-End Testing + +* **Test the Entire Application:** Write end-to-end tests to test the entire application from the user's perspective. +* **Use a Testing Framework:** Use a testing framework, such as Selenium or Cypress, to automate end-to-end tests. +* **Verify User Interactions:** Verify that user interactions, such as clicking buttons or entering data, produce the expected results. + +### 5.4. Test Organization + +* **Organize Tests by Module:** Organize your tests into directories that correspond to the modules in your application. +* **Use Descriptive Test Names:** Use descriptive test names that clearly indicate what the test is verifying. +* **Write Clear Assertions:** Write clear assertions that verify the expected results. + +### 5.5. Mocking and Stubbing + +* **Use Mock Objects:** Use mock objects to replace external dependencies, such as databases or APIs, with controlled test doubles. +* **Use Stub Functions:** Use stub functions to replace complex or time-consuming operations with simple, predictable implementations. +* **Verify Interactions with Mocks:** Verify that your code interacts with mock objects in the expected way. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +* **Incorrect Data Formatting:** Ensure that your data is formatted correctly for Plotly charts. For example, ensure that dates are in the correct format and that numerical data is not stored as strings. +* **Missing Dependencies:** Ensure that all required dependencies are installed. +* **Incorrect Chart Types:** Choose the correct chart type for your data and the message you want to convey. +* **Ignoring Layout Customization:** Customize the chart layout to improve readability and visual appeal. + +### 6.2. Edge Cases + +* **Handling Missing Data:** Handle missing data gracefully by either removing it from the chart or replacing it with a default value. +* **Dealing with Outliers:** Deal with outliers appropriately by either removing them from the chart or using a chart type that is less sensitive to outliers, such as a box plot. +* **Handling Large Numbers:** Handle large numbers by using appropriate formatting and scaling. + +### 6.3. Version-Specific Issues + +* **Check Release Notes:** Check the release notes for each Plotly version to be aware of any breaking changes or bug fixes. +* **Test with Different Versions:** Test your code with different Plotly versions to ensure compatibility. + +### 6.4. Compatibility Concerns + +* **Dash Version Compatibility:** Ensure that your Dash version is compatible with your Plotly version. +* **Browser Compatibility:** Test your charts in different browsers to ensure compatibility. + +### 6.5. Debugging Strategies + +* **Use Debugging Tools:** Use debugging tools, such as print statements or a debugger, to identify and fix issues in your code. +* **Check Error Messages:** Check error messages for clues about what is going wrong. +* **Simplify the Chart:** Simplify the chart by removing traces or annotations to isolate the issue. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +* **VS Code:** A popular code editor with excellent support for Python and Plotly. +* **Jupyter Notebook:** An interactive environment for data exploration and prototyping. +* **PyCharm:** A full-featured IDE for Python development. +* **Dash Enterprise:** A platform for deploying and managing Dash applications. + +### 7.2. Build Configuration Best Practices + +* **Use a Virtual Environment:** Create a virtual environment for your project to isolate dependencies. +* **Use a Dependency Manager:** Use a dependency manager, such as pip or poetry, to manage project dependencies. +* **Specify Dependencies:** Specify the exact versions of your dependencies in your `requirements.txt` or `pyproject.toml` file. + +### 7.3. Linting and Formatting + +* **Use a Linter:** Use a linter, such as pylint or flake8, to identify and fix code style issues. +* **Use a Formatter:** Use a formatter, such as black or autopep8, to automatically format your code according to a consistent style. +* **Configure Your Editor:** Configure your editor to automatically run the linter and formatter when you save your code. + +### 7.4. Deployment Best Practices + +* **Use a Production Environment:** Deploy your application to a production environment that is separate from your development environment. +* **Use a Web Server:** Use a web server, such as nginx or Apache, to serve your application. +* **Use a Process Manager:** Use a process manager, such as gunicorn or uwsgi, to manage your application processes. + +### 7.5. CI/CD Integration Strategies + +* **Use a CI/CD Pipeline:** Use a CI/CD pipeline, such as Jenkins or GitHub Actions, to automate the build, test, and deployment process. +* **Run Tests Automatically:** Run tests automatically as part of the CI/CD pipeline. +* **Deploy Automatically:** Deploy your application automatically to the production environment when all tests pass. + +By following these best practices and coding standards, you can create maintainable, efficient, and secure Plotly applications that meet the needs of your users and stakeholders. \ No newline at end of file diff --git a/.cursor/rules/poetry.mdc b/.cursor/rules/poetry.mdc new file mode 100644 index 000000000..67fcf7c49 --- /dev/null +++ b/.cursor/rules/poetry.mdc @@ -0,0 +1,333 @@ +--- +description: This rule set provides comprehensive guidance on best practices for using Poetry in Python projects, including dependency management, project structure, and coding standards. It covers various aspects such as code organization, performance considerations, security, testing, and tooling. +globs: *.py +--- +# Poetry Python Library Best Practices + +This document outlines best practices for utilizing Poetry, a dependency management and packaging tool for Python. Following these guidelines will result in more maintainable, scalable, and robust Python projects. + +## 1. Code Organization and Structure + +A well-organized project structure is critical for maintainability and collaboration. Poetry provides the tools to enforce a standardized structure. + +### 1.1 Directory Structure Best Practices + +* **Root Directory:** The root directory contains the `pyproject.toml` file, `README.md`, and potentially other top-level configuration files (e.g., `.gitignore`, `mypy.ini`). +* **Source Directory:** The main source code should reside in a dedicated directory. The typical layout is to place your package in a top-level directory, e.g., `my_package/`, or within a `src/` directory (e.g., `src/my_package/`). Using `src/` allows for cleaner separation of concerns, facilitating testing and package building. + * **Choosing Between Flat and `src` Layouts:** + * **Flat Layout (default):** `my_project/my_package/__init__.py` + * **`src` Layout:** `my_project/src/my_package/__init__.py`. The `src` layout avoids potential import ambiguity when running scripts from the project root and provides a clearer separation between project-level files and package code. +* **Tests Directory:** Tests should be in a `tests/` directory, mirroring the source structure. This helps in easily locating and running tests. Include an `__init__.py` in the directory to make it a package. +* **Docs Directory (Optional):** If the project requires extensive documentation, a `docs/` directory can be included to house documentation files. +* **Examples Directory (Optional):** Provides example usage scenarios to help end-users easily start using your library. + +Example directory structure (using the `src` layout): + + +my_project/ +├── src/ +│ └── my_package/ +│ ├── __init__.py +│ ├── module1.py +│ └── module2.py +├── tests/ +│ ├── __init__.py +│ ├── test_module1.py +│ └── test_module2.py +├── pyproject.toml +├── poetry.lock +├── README.md +└── .gitignore + + +### 1.2 File Naming Conventions + +* **Python Files:** Use lowercase names with underscores (snake_case) for Python files (e.g., `my_module.py`, `data_processing.py`). +* **Test Files:** Test files should mirror the module being tested, prefixed with `test_`. For example, `test_my_module.py` tests `my_module.py`. +* **`__init__.py`:** Use an empty `__init__.py` file or add initialization logic if needed. The presence of `__init__.py` makes the directory a Python package. +* **Configuration Files:** Use descriptive names for configuration files (e.g., `config.py`, `settings.toml`). + +### 1.3 Module Organization + +* **Grouping Functionality:** Group related functions and classes into modules. Each module should represent a specific functionality or component. +* **Avoiding Circular Dependencies:** Design modules to minimize dependencies on each other. Circular dependencies can lead to import errors and make code harder to understand. +* **Explicit Imports:** Use explicit imports (e.g., `from my_package.module1 import function_x`) instead of wildcard imports (`from my_package.module1 import *`). Explicit imports improve code readability and reduce namespace pollution. +* **Relative Imports (Where Appropriate):** Within a package, use relative imports (e.g., `from . import module2` or `from .. import another_package`) to reference other modules in the same package. This makes the package more self-contained. + +### 1.4 Component Architecture + +* **Layered Architecture:** For larger projects, consider a layered architecture (e.g., presentation layer, business logic layer, data access layer). This promotes separation of concerns and makes the codebase more modular. +* **Microservices (If Applicable):** If the project is complex, consider splitting it into microservices, each managed as a separate Poetry project. +* **Dependency Injection:** Employ dependency injection to decouple components. This enhances testability and flexibility. +* **Interfaces:** Define clear interfaces between components. This allows for easier swapping of implementations without affecting other parts of the system. + +### 1.5 Code Splitting Strategies + +* **By Functionality:** Split code into modules based on functionality (e.g., `data_processing.py`, `api_client.py`, `ui_components.py`). +* **By Component:** If using a component-based architecture, split code into modules representing individual components. +* **Lazy Loading:** Use lazy loading for modules that are not immediately required at startup. This can improve application startup time. +* **Dynamic Imports:** Use dynamic imports (e.g., `importlib.import_module`) for optional dependencies or features. Handle potential `ImportError` exceptions gracefully. + +## 2. Common Patterns and Anti-patterns + +Understanding common patterns and anti-patterns is essential for writing clean, efficient, and maintainable code. + +### 2.1 Design Patterns + +* **Factory Pattern:** Use factory patterns to create objects without specifying their concrete classes. This promotes loose coupling and makes it easier to switch between different implementations. +* **Strategy Pattern:** Use strategy patterns to define a family of algorithms and make them interchangeable at runtime. This allows for flexible algorithm selection based on different conditions. +* **Observer Pattern:** Implement the observer pattern for event handling and decoupled communication between components. +* **Singleton Pattern (Use Sparingly):** Avoid overuse of the singleton pattern. If used, ensure thread safety. + +### 2.2 Recommended Approaches for Common Tasks + +* **Dependency Injection:** Using dependency injection to manage dependencies improves testability and flexibility. +* **Configuration Management:** Using configuration files (e.g., TOML, YAML) to store application settings simplifies management and deployment. +* **Logging:** Using a logging library (e.g., `logging`) for application logging enables easy debugging and monitoring. +* **Data Validation:** Use a data validation library (e.g., `pydantic`, `marshmallow`) to validate input data and ensure data integrity. +* **API Clients:** Encapsulate API interactions within dedicated modules, enabling reuse and reducing code duplication. Use libraries like `requests` or `httpx` for HTTP requests. Consider asynchronous clients with `asyncio` for enhanced performance in I/O bound scenarios. + +### 2.3 Anti-patterns and Code Smells + +* **Global State:** Avoid using global variables excessively. Global state makes code harder to reason about and test. +* **Magic Numbers:** Avoid using magic numbers in code. Define constants with descriptive names instead. +* **Duplicated Code:** Avoid code duplication. Extract common code into functions or classes. +* **Long Functions:** Avoid writing long functions. Break them down into smaller, more manageable functions. +* **Over-Engineering:** Don't over-engineer solutions. Keep the code simple and focused on the problem at hand. +* **Ignoring Exceptions:** Never leave `except:` blocks empty. Always handle exceptions appropriately. +* **Wildcard Imports:** Avoid wildcard imports (`from module import *`). They pollute the namespace and make it harder to understand where symbols come from. + +### 2.4 State Management + +* **Stateless Components:** Prefer stateless components whenever possible. Stateless components are easier to test and reason about. +* **Immutable Data:** Use immutable data structures (e.g., `namedtuple`, `dataclasses` with `frozen=True`) to avoid accidental state modifications. +* **Explicit State:** Make state transitions explicit and well-defined. Use a state management library (e.g., `transitions`) for complex state machines. +* **Centralized State (If Needed):** For complex applications, consider using a centralized state management solution (e.g., Redux-like pattern). However, be mindful of the increased complexity. + +### 2.5 Error Handling + +* **Specific Exceptions:** Catch specific exceptions instead of using a broad `except Exception:`. +* **Logging Errors:** Log error messages with relevant context (e.g., function name, input arguments). +* **Raising Exceptions:** Raise exceptions when errors occur to signal failures to the caller. +* **Context Managers:** Use context managers (`with` statement) to ensure proper resource management (e.g., closing files, releasing locks). +* **Retry Mechanism:** Implement a retry mechanism for transient errors (e.g., network errors, API rate limits). Use libraries like `tenacity` to easily implement retry logic with customizable backoff strategies. +* **Error Boundaries:** Design error boundaries in your application that prevent errors in one part of the system from crashing the entire application. This can be achieved through exception handling at appropriate levels and the use of circuit breaker patterns. + +## 3. Performance Considerations + +Optimizing performance is crucial for providing a smooth user experience and reducing resource consumption. + +### 3.1 Optimization Techniques + +* **Profiling:** Use profiling tools (e.g., `cProfile`, `line_profiler`) to identify performance bottlenecks. +* **Algorithm Optimization:** Choose efficient algorithms and data structures for critical tasks. +* **Caching:** Implement caching for frequently accessed data. Use libraries like `cachetools` or `diskcache` for caching. +* **Asynchronous Programming:** Use asynchronous programming (`asyncio`) for I/O-bound operations (e.g., network requests, file I/O). This can significantly improve concurrency and responsiveness. +* **Multiprocessing:** Use multiprocessing (`multiprocessing`) for CPU-bound operations to take advantage of multiple cores. However, be mindful of the increased complexity of inter-process communication. +* **Vectorization (NumPy):** Use NumPy for vectorized operations on arrays. Vectorization can significantly speed up numerical computations. +* **Just-In-Time (JIT) Compilation:** Consider using a JIT compiler (e.g., Numba) for computationally intensive functions. JIT compilation can significantly improve performance. + +### 3.2 Memory Management + +* **Garbage Collection:** Understand how Python's garbage collection works. Avoid creating circular references that can prevent garbage collection. +* **Generators and Iterators:** Use generators and iterators for processing large datasets. Generators and iterators consume memory on demand, avoiding the need to load the entire dataset into memory. +* **Data Structure Efficiency:** Choose efficient data structures for storing data. Consider using specialized data structures (e.g., `collections.deque`, `heapq`) when appropriate. +* **Memory Profiling:** Use memory profiling tools (e.g., `memory_profiler`) to identify memory leaks and excessive memory consumption. +* **Object Reuse:** Reuse objects when possible to minimize object creation and destruction overhead. This can be particularly relevant for long-running processes. + +### 3.3 Bundle Size Optimization (If Applicable) + +* **Tree Shaking:** If using a bundler, enable tree shaking to remove unused code from the bundle. +* **Code Minification:** Minify code to reduce the bundle size. Tools like `Terser` can be used for code minification. +* **Image Optimization:** Optimize images to reduce their size without compromising quality. +* **Dependency Optimization:** Analyze dependencies to identify and remove unnecessary dependencies. Tools like `depcheck` can help with dependency optimization. +* **Compression:** Use compression algorithms (e.g., gzip, Brotli) to compress the bundle before deployment. This reduces the download size and improves loading time. + +### 3.4 Lazy Loading + +* **Module Lazy Loading:** Use dynamic imports to lazy load modules that are not immediately required. +* **Data Lazy Loading:** Use generators or iterators to lazy load data from databases or files. +* **Component Lazy Loading:** Load UI components or application features on demand, improving initial startup time. + +## 4. Security Best Practices + +Security should be a top priority in any software project. Following security best practices can help protect your application from vulnerabilities. + +### 4.1 Common Vulnerabilities and Prevention + +* **Injection Attacks:** Prevent injection attacks (e.g., SQL injection, command injection) by validating and sanitizing user inputs. Use parameterized queries or ORMs to prevent SQL injection. +* **Cross-Site Scripting (XSS):** Prevent XSS attacks by escaping user-generated content before displaying it in the browser. +* **Cross-Site Request Forgery (CSRF):** Protect against CSRF attacks by using anti-CSRF tokens. +* **Authentication and Authorization:** Implement strong authentication and authorization mechanisms to protect sensitive resources. +* **Dependency Vulnerabilities:** Regularly scan dependencies for vulnerabilities and update them to the latest versions. Tools like `Safety` or Snyk can be used to detect and remediate dependency vulnerabilities. +* **Sensitive Data Exposure:** Avoid storing sensitive data (e.g., passwords, API keys) in plaintext. Encrypt sensitive data at rest and in transit. +* **Denial of Service (DoS):** Implement rate limiting and input validation to prevent DoS attacks. + +### 4.2 Input Validation + +* **Validate All Inputs:** Validate all user inputs, including form data, URL parameters, and API requests. +* **Use a Validation Library:** Use a validation library (e.g., `pydantic`, `marshmallow`) to define validation schemas and validate data against them. +* **Sanitize Inputs:** Sanitize inputs to remove potentially harmful characters or code. +* **Limit Input Length:** Limit the length of input fields to prevent buffer overflows. +* **Check Data Types:** Verify that input data is of the expected data type. + +### 4.3 Authentication and Authorization + +* **Strong Passwords:** Enforce strong password policies (e.g., minimum length, complexity requirements). +* **Hashing Passwords:** Hash passwords using a strong hashing algorithm (e.g., bcrypt, Argon2) and store them securely. +* **Salting Passwords:** Salt passwords to prevent rainbow table attacks. +* **Two-Factor Authentication (2FA):** Implement two-factor authentication for increased security. +* **Role-Based Access Control (RBAC):** Use role-based access control to restrict access to sensitive resources based on user roles. +* **JSON Web Tokens (JWT):** Use JSON Web Tokens for stateless authentication and authorization. +* **OAuth 2.0:** Use OAuth 2.0 for secure delegation of authorization. + +### 4.4 Data Protection + +* **Encryption:** Encrypt sensitive data at rest and in transit using strong encryption algorithms (e.g., AES, TLS). +* **Data Masking:** Mask sensitive data when displaying it to users. +* **Data Anonymization:** Anonymize data when it is not necessary to identify individual users. +* **Regular Backups:** Perform regular backups of data to prevent data loss. +* **Secure Storage:** Store data in a secure location with appropriate access controls. + +### 4.5 Secure API Communication + +* **HTTPS:** Use HTTPS for all API communication to encrypt data in transit. +* **API Keys:** Protect API keys and store them securely. Rotate API keys regularly. +* **Rate Limiting:** Implement rate limiting to prevent API abuse. +* **Input Validation:** Validate all API requests to prevent injection attacks. +* **Output Sanitization:** Sanitize API responses to prevent XSS attacks. +* **Authentication and Authorization:** Implement strong authentication and authorization mechanisms for API access. +* **CORS:** Configure Cross-Origin Resource Sharing (CORS) to restrict API access to authorized domains. + +## 5. Testing Approaches + +Thorough testing is essential for ensuring the quality and reliability of software. + +### 5.1 Unit Testing + +* **Test-Driven Development (TDD):** Consider using test-driven development (TDD) to write tests before writing code. +* **Test Coverage:** Aim for high test coverage to ensure that all parts of the code are tested. +* **Test Independence:** Write tests that are independent of each other. Avoid sharing state between tests. +* **Assertion Libraries:** Use assertion libraries (e.g., `unittest`, `pytest`) to write clear and concise assertions. +* **Edge Cases:** Test edge cases and boundary conditions to ensure that the code handles them correctly. + +### 5.2 Integration Testing + +* **Component Integration:** Test the integration between different components of the system. +* **API Integration:** Test the integration with external APIs. +* **Database Integration:** Test the integration with databases. +* **Mock External Dependencies:** Use mocks or stubs to isolate the system under test from external dependencies. +* **Real-World Scenarios:** Design integration tests to simulate real-world usage scenarios. + +### 5.3 End-to-End Testing + +* **User Interface Testing:** Test the user interface to ensure that it is working as expected. +* **User Flows:** Test complete user flows to ensure that the system is functioning correctly from start to finish. +* **Automated Testing:** Automate end-to-end tests to ensure that they are run regularly. +* **Browser Automation:** Use browser automation tools (e.g., Selenium, Playwright) to automate user interface tests. +* **Continuous Integration:** Integrate end-to-end tests into the continuous integration pipeline. + +### 5.4 Test Organization + +* **Mirror Source Structure:** Organize tests in a directory structure that mirrors the source code structure. +* **Descriptive Test Names:** Use descriptive names for test functions and test classes. +* **Grouping Tests:** Group related tests into test classes or modules. +* **Test Suites:** Create test suites to group related tests together. +* **Test Runners:** Use test runners (e.g., `unittest`, `pytest`) to discover and run tests. + +### 5.5 Mocking and Stubbing + +* **Mock External Dependencies:** Use mocks to simulate the behavior of external dependencies (e.g., databases, APIs). +* **Stub Internal Dependencies:** Use stubs to replace internal dependencies with simpler implementations. +* **Isolate the System Under Test:** Use mocks and stubs to isolate the system under test from external factors. +* **Verify Interactions:** Use mocks to verify that the system under test is interacting with its dependencies in the expected way. +* **Mocking Libraries:** Use mocking libraries (e.g., `unittest.mock`, `pytest-mock`) to create mocks and stubs. + +## 6. Common Pitfalls and Gotchas + +Being aware of common pitfalls can save time and prevent headaches. + +### 6.1 Frequent Mistakes + +* **Forgetting to Update `poetry.lock`:** Failing to update the `poetry.lock` file after adding or updating dependencies can lead to inconsistent environments. +* **Using Wildcard Dependencies:** Using wildcard dependencies can lead to unexpected updates that break the code. +* **Ignoring Virtual Environments:** Not using virtual environments can lead to conflicts between different projects. +* **Committing Virtual Environment Directories:** Committing the virtual environment directory to version control is unnecessary and can lead to problems. +* **Using Incompatible Python Versions:** Using incompatible Python versions can lead to errors. + +### 6.2 Edge Cases + +* **Platform-Specific Dependencies:** Handling platform-specific dependencies requires careful consideration of package availability and compatibility. +* **Optional Dependencies:** Managing optional dependencies that may not be available on all systems. +* **Conflicting Dependencies:** Resolving conflicts between dependencies with overlapping requirements can be challenging. +* **Cyclic Dependencies:** Cyclic dependencies can lead to import errors and make code harder to understand. +* **Large Datasets:** Handling large datasets requires careful consideration of memory management and performance. + +### 6.3 Version-Specific Issues + +* **Dependency Compatibility:** Ensuring that dependencies are compatible with the target Python version and each other. +* **Deprecated Features:** Handling deprecated features and migrating to newer versions of dependencies. +* **Security Updates:** Staying up-to-date with security updates for Poetry and its dependencies. + +### 6.4 Compatibility Concerns + +* **Operating System Compatibility:** Ensuring that the code works correctly on different operating systems. +* **Python Interpreter Compatibility:** Ensuring that the code works correctly with different Python interpreters (e.g., CPython, PyPy). +* **Package Compatibility:** Ensuring that packages are compatible with each other. +* **Hardware Compatibility:** Considering hardware compatibility, especially when dealing with platform-specific dependencies or performance-critical code. +* **Backward Compatibility:** Striving to maintain backward compatibility to minimize disruption for existing users when introducing new features or changes. + +### 6.5 Debugging Strategies + +* **Logging:** Using logging to track the flow of execution and identify errors. +* **Debuggers:** Using debuggers (e.g., `pdb`, `ipdb`) to step through code and inspect variables. +* **Unit Tests:** Writing unit tests to isolate and test individual components. +* **Print Statements (Use Sparingly):** Using print statements to display the values of variables (use sparingly and remove them before committing code). +* **Profiling:** Using profiling tools to identify performance bottlenecks. +* **Remote Debugging:** Utilize remote debugging techniques to debug code running on remote servers or containers. This can be particularly helpful for cloud deployments. +* **Post-Mortem Debugging:** Leverage Python's post-mortem debugging capabilities to analyze errors after they have occurred. This can provide valuable insights into the root cause of the problem. + +## 7. Tooling and Environment + +Using the right tools and configuring the environment correctly can significantly improve developer productivity. + +### 7.1 Recommended Development Tools + +* **Integrated Development Environment (IDE):** Use an IDE (e.g., VS Code, PyCharm) with Python support for code completion, syntax highlighting, and debugging. +* **Linters:** Use linters (e.g., `flake8`, `pylint`) to enforce code style and identify potential errors. +* **Formatters:** Use formatters (e.g., `black`, `autopep8`) to automatically format code according to style guidelines. +* **Type Checkers:** Use type checkers (e.g., `mypy`) to catch type errors early on. +* **Version Control System (VCS):** Use a version control system (e.g., Git) to track changes to the code. + +### 7.2 Build Configuration + +* **`pyproject.toml`:** The cornerstone of Poetry projects, this file defines project metadata, dependencies, and build configuration. +* **`.gitignore`:** Excludes unnecessary files and directories (e.g., virtual environment directories, build artifacts) from version control. +* **`Makefile` (Optional):** Use a `Makefile` to automate common tasks (e.g., running tests, building documentation). +* **Dockerfiles:** Use Dockerfiles to create container images for deployment. This ensures consistent environments across different systems. +* **Configuration Files:** Use configuration files (e.g., TOML, YAML) to store application settings and environment variables. + +### 7.3 Linting and Formatting + +* **`flake8` Configuration:** Configure `flake8` to enforce coding style guidelines and identify potential errors. Configure in `pyproject.toml` if the tool supports it, otherwise use a separate config file. +* **`black` Configuration:** Configure `black` to automatically format code according to style guidelines. Configure in `pyproject.toml`. +* **`isort` Configuration:** Configure `isort` to automatically sort imports according to style guidelines. Configure in `pyproject.toml`. +* **Pre-commit Hooks:** Use pre-commit hooks to automatically run linters and formatters before committing code. This helps to maintain code quality and consistency. + +### 7.4 Deployment + +* **Containerization:** Containerize the application using Docker to ensure consistent environments across different systems. +* **Cloud Platforms:** Deploy the application to a cloud platform (e.g., AWS, Azure, Google Cloud) for scalability and reliability. +* **Continuous Delivery:** Use continuous delivery pipelines to automate the deployment process. +* **Configuration Management:** Use configuration management tools (e.g., Ansible, Chef, Puppet) to manage infrastructure configuration. +* **Monitoring:** Implement monitoring to track the performance and health of the application. + +### 7.5 CI/CD Integration + +* **Continuous Integration (CI):** Integrate the code with a continuous integration system (e.g., Jenkins, GitLab CI, GitHub Actions) to automate testing and building. +* **Continuous Delivery (CD):** Integrate the CI system with a continuous delivery system to automate the deployment process. +* **Automated Testing:** Automate unit tests, integration tests, and end-to-end tests in the CI/CD pipeline. +* **Automated Code Analysis:** Integrate code analysis tools (e.g., linters, formatters, type checkers) into the CI/CD pipeline. +* **Automated Deployment:** Automate the deployment process to ensure that code is deployed consistently and reliably. + +By adhering to these best practices, developers can create Python projects with Poetry that are more maintainable, scalable, secure, and reliable. \ No newline at end of file diff --git a/.cursor/rules/pony.mdc b/.cursor/rules/pony.mdc new file mode 100644 index 000000000..38216c87d --- /dev/null +++ b/.cursor/rules/pony.mdc @@ -0,0 +1,275 @@ +--- +description: Comprehensive guidelines for Pony ORM best practices, covering code organization, patterns, performance, security, testing, common pitfalls, and tooling. +globs: *.py +--- +# Pony ORM Best Practices + +This document outlines best practices for developing Python applications using Pony ORM. Following these guidelines will improve code maintainability, readability, performance, and security. + +## 1. Code Organization and Structure + +### Directory Structure + +Adopt a logical directory structure to organize your project. A common pattern is to separate data access logic (Pony entities and queries) from business logic. + + +project/ +├── models/ +│ ├── __init__.py +│ ├── entities.py # Pony entity definitions +│ └── queries.py # Reusable queries +├── services/ +│ ├── __init__.py +│ ├── user_service.py # Business logic for users +│ └── product_service.py # Business logic for products +├── utils/ +│ └── db.py # Database connection and session management +├── tests/ +│ ├── __init__.py +│ ├── test_entities.py +│ └── test_user_service.py +├── main.py # Application entry point +└── requirements.txt + + +### File Naming Conventions + +* Use descriptive names for files and modules. +* `entities.py`: Contains Pony entity definitions. +* `queries.py`: Contains reusable database queries. +* `[module]_service.py`: Contains business logic for a specific module. +* `test_[module].py`: Contains unit tests for a specific module. + +### Module Organization + +* Group related entities and queries into separate modules. +* Use `__init__.py` to define module interfaces and control imports. + +### Component Architecture + +* **Entities Layer**: Contains database models and relationships. Entities should primarily define data structure and basic validation. +* **Data Access Layer**: Manages database interactions. Implement CRUD operations (Create, Read, Update, Delete) within this layer. Avoid business logic in this layer, focusing solely on data retrieval and persistence. +* **Business Logic Layer**: This layer contains the core application logic. It orchestrates data flow between the data access layer and the presentation layer. +* **Presentation Layer**: (If applicable) Handles user interaction and data display. This could be a web framework, GUI application or CLI. + +### Code Splitting + +* Break down large modules into smaller, more manageable files. +* Use subpackages to group related modules. +* Consider splitting modules based on functionality or entity type. + +## 2. Common Patterns and Anti-patterns + +### Design Patterns + +* **Repository Pattern**: Encapsulate data access logic behind a repository interface. This allows for easier testing and switching of data storage implementations. +* **Unit of Work Pattern**: Track changes to entities within a single unit of work (database transaction) and commit all changes at once. +* **Data Mapper Pattern**: Map data between the database and the entity model. Pony ORM inherently implements this pattern. +* **Factory Pattern**: Use factories to create entity instances, especially when instantiation logic is complex or requires external dependencies. +* **Singleton Pattern**: Ensure only one instance of the `Database` object exists for each database connection. This is often used in conjunction with a dependency injection framework. + +### Recommended Approaches for Common Tasks + +* **Fetching data efficiently**: Utilize Pony's query optimization features to minimize database round trips. +* **Handling relationships**: Properly define relationships (One-to-many, Many-to-many) using `Set`, `Required`, and `Optional` attributes. +* **Performing complex queries**: Leverage Pony's Pythonic query syntax for readable and maintainable queries. +* **Managing transactions**: Always use the `db_session` decorator or context manager to wrap database interactions. +* **Implementing validation**: Define validation rules within entity classes or using separate validation functions. + +### Anti-patterns and Code Smells + +* **Mixing business logic with entity definitions**: Keep entity classes focused on data representation, not application logic. +* **Performing database operations outside of a `db_session`**: This will lead to `TransactionError` exceptions. +* **Over-fetching data**: Only select the data that is needed for a specific operation. Use projections in your queries. +* **Ignoring Pony's query optimization**: Write inefficient queries that lead to performance bottlenecks. +* **Directly manipulating the database connection**: Rely on Pony ORM for database interactions. +* **Not handling exceptions**: Lack of exception handling can lead to application instability and data corruption. +* **Using global database sessions without proper context management.** This can lead to concurrency issues. +* **Not using indexes where appropriate.** This will lead to slow query performance. + +### State Management + +* Within a `db_session`, Pony ORM manages the state of entities automatically. +* For long-running processes, be mindful of memory usage and consider clearing the `db_session` cache periodically. +* Avoid passing entity instances outside of the `db_session` scope if you intend to modify them. Consider extracting relevant data and passing simple data objects (dictionaries, data transfer objects) instead. + +### Error Handling + +* Use `try...except` blocks to handle potential exceptions during database operations. +* Rollback transactions in case of errors using `db.rollback()` or let `db_session` handle it. +* Log errors for debugging and monitoring purposes. +* Define custom exceptions for specific error scenarios. +* Consider retrying transactions for recoverable errors. + +## 3. Performance Considerations + +### Optimization Techniques + +* **Query optimization**: Use Pony's query syntax effectively to generate optimized SQL queries. Analyze generated SQL queries to identify potential performance bottlenecks and optimize using indexes. Use `set_sql_debug(True)` to observe the queries. +* **Caching**: Pony ORM includes Identity Map, automatic caching of queries and objects. Understand its limitations and potential benefits. +* **Lazy loading**: Avoid loading unnecessary data by using lazy loading for related entities. +* **Eager loading**: For well-known queries where related entities are always required, use eager loading to reduce the number of database queries. +* **Batch operations**: Perform bulk inserts, updates, and deletes to minimize database round trips. +* **Indexing**: Add indexes to frequently queried columns to improve query performance. +* **Connection pooling**: Pony ORM uses connection pooling. Configure connection pool size based on application needs. +* **Data Type Selection**: Choose the appropriate data types for your entities to minimize storage space and improve query performance. + +### Memory Management + +* Be aware of the size of the `db_session` cache and clear it periodically for long-running processes. +* Avoid loading large datasets into memory at once. +* Use generators for processing large datasets in chunks. + +### Rendering Optimization + +* If applicable, optimize rendering logic to minimize the amount of data that needs to be displayed. +* Use pagination for displaying large lists of data. +* Consider caching rendered output to reduce database load. + +### Bundle Size Optimization + +* If applicable to your project, minimize the size of your application bundle by removing unused code and dependencies. +* Use code splitting to load only the code that is needed for a specific page or feature. + +### Lazy Loading + +* Pony ORM supports lazy loading for related entities. +* Use lazy loading to avoid loading unnecessary data upfront. +* Be mindful of the N+1 problem when using lazy loading and consider eager loading for frequently accessed related entities. + +## 4. Security Best Practices + +### Common Vulnerabilities and Prevention + +* **SQL injection**: Prevent SQL injection by using Pony's parameterized queries. Never construct SQL queries by concatenating strings directly. Pony ORM avoids SQL injection by using generator expressions or lambda functions. +* **Cross-site scripting (XSS)**: If applicable to your project, sanitize user input to prevent XSS attacks. +* **Cross-site request forgery (CSRF)**: If applicable to your project, protect against CSRF attacks by using CSRF tokens. +* **Authentication and authorization vulnerabilities**: Implement secure authentication and authorization mechanisms to protect sensitive data. +* **Denial-of-service (DoS) attacks**: Protect against DoS attacks by implementing rate limiting and other security measures. + +### Input Validation + +* Validate all user input to prevent malicious data from being stored in the database. +* Use appropriate validation rules for each entity attribute. +* Sanitize input to remove potentially harmful characters. + +### Authentication and Authorization + +* Use a secure authentication mechanism to verify user identities. +* Implement authorization rules to control access to sensitive data and functionality. +* Consider using a role-based access control (RBAC) system. + +### Data Protection + +* Encrypt sensitive data at rest and in transit. +* Use strong passwords and store them securely. +* Implement data masking to protect sensitive data from unauthorized access. + +### Secure API Communication + +* Use HTTPS for all API communication. +* Implement API authentication and authorization. +* Protect against API attacks, such as rate limiting and input validation. + +## 5. Testing Approaches + +### Unit Testing + +* Write unit tests for individual components, such as entity classes, queries, and services. +* Use mocking and stubbing to isolate components from external dependencies. +* Test edge cases and error conditions. +* Use `in-memory` SQLite database for faster and isolated testing. + +### Integration Testing + +* Write integration tests to verify the interaction between different components. +* Test the integration between Pony ORM and the database. +* Use a test database that is separate from the production database. + +### End-to-End Testing + +* Write end-to-end tests to verify the entire application workflow. +* Test the application from the user's perspective. +* Use a testing framework such as Selenium or Cypress. + +### Test Organization + +* Organize tests into separate modules based on the component or functionality being tested. +* Use a consistent naming convention for test files and functions. +* Keep tests small and focused. + +### Mocking and Stubbing + +* Use mocking to replace external dependencies with controlled substitutes. +* Use stubbing to provide predefined responses for external dependencies. +* Use mocking frameworks such as `unittest.mock` or `pytest-mock`. + +## 6. Common Pitfalls and Gotchas + +### Frequent Mistakes + +* Forgetting to use `db_session` for database interactions. +* Writing inefficient queries that lead to performance bottlenecks. +* Not handling exceptions properly. +* Ignoring security best practices. +* Using deprecated features or APIs. +* Incorrectly configuring database connections. +* Misunderstanding how the Identity Map cache works. + +### Edge Cases + +* Handling concurrent database access. +* Dealing with large datasets. +* Managing database migrations. +* Handling database errors and exceptions. +* Working with different database vendors. + +### Version-Specific Issues + +* Be aware of compatibility issues between different versions of Pony ORM. +* Consult the Pony ORM documentation for version-specific information. + +### Compatibility Concerns + +* Be aware of compatibility issues between Pony ORM and other technologies, such as web frameworks and database drivers. +* Consult the Pony ORM documentation for compatibility information. + +### Debugging Strategies + +* Use Pony's SQL debugging feature to inspect generated SQL queries. +* Use logging to track application behavior and identify errors. +* Use a debugger to step through code and inspect variables. +* Check the database logs for errors and warnings. + +## 7. Tooling and Environment + +### Recommended Development Tools + +* Python IDE (e.g., PyCharm, VS Code). +* Database client (e.g., DBeaver, pgAdmin). +* Testing framework (e.g., pytest, unittest). +* Mocking framework (e.g., unittest.mock, pytest-mock). +* Linting and formatting tools (e.g., flake8, black). + +### Build Configuration + +* Use a build system such as `setuptools` or `poetry` to manage project dependencies. +* Specify Pony ORM as a dependency in your project's `requirements.txt` or `pyproject.toml` file. + +### Linting and Formatting + +* Use linting tools such as `flake8` to enforce code style guidelines. +* Use formatting tools such as `black` to automatically format code. +* Configure your IDE to run linting and formatting tools automatically. + +### Deployment + +* Deploy your application to a production environment that is separate from the development environment. +* Configure your application to use a production database. +* Monitor your application for errors and performance issues. + +### CI/CD Integration + +* Use a continuous integration and continuous delivery (CI/CD) system to automate the build, test, and deployment process. +* Integrate Pony ORM tests into your CI/CD pipeline. +* Automate database migrations as part of your CI/CD process. \ No newline at end of file diff --git a/.cursor/rules/postgresql.mdc b/.cursor/rules/postgresql.mdc new file mode 100644 index 000000000..a2a6ffe82 --- /dev/null +++ b/.cursor/rules/postgresql.mdc @@ -0,0 +1,107 @@ +--- +description: Enforces PostgreSQL coding standards, best practices, and performance optimization techniques to ensure maintainable, efficient, and secure database interactions. This rule covers code formatting, data integrity, security, and performance considerations. +globs: *.sql,*.plpgsql,*.c,*.h +--- +- **Code Formatting and Comments:** + - Maintain consistent code formatting using a tool like `pgformatter` or similar. + - Use clear and concise comments to explain complex logic and intentions. Update comments regularly to avoid confusion. + - Use inline comments sparingly; prefer block comments for detailed explanations. + - Write comments in plain, easy-to-follow English. + - Add a space after line comments (`-- a comment`); do not add a space for commented-out code (`--raise notice`). + - Keep comments up-to-date; incorrect comments are worse than no comments. + +- **Naming Conventions:** + - Use `snake_case` for identifiers (e.g., `user_id`, `customer_name`). + - Use plural nouns for table names (e.g., `customers`, `products`). + - Use consistent naming conventions for functions, procedures, and triggers. + - Choose descriptive and meaningful names for all database objects. + +- **Data Integrity and Data Types:** + - Use appropriate data types for columns to ensure data integrity (e.g., `INTEGER`, `VARCHAR`, `TIMESTAMP`). + - Use constraints (e.g., `NOT NULL`, `UNIQUE`, `CHECK`, `FOREIGN KEY`) to enforce data integrity. + - Define primary keys for all tables. + - Use foreign keys to establish relationships between tables. + - Utilize domains to enforce data type constraints reusable across multiple columns. + +- **SQL Injection Prevention:** + - Always use parameterized queries or prepared statements to prevent SQL injection attacks. + - Sanitize user inputs before using them in SQL queries. + - Avoid constructing SQL queries by concatenating strings directly. + - Use the `quote_literal()` and `quote_ident()` functions to properly escape strings and identifiers. + +- **Transaction Management:** + - Use explicit transactions to ensure data consistency and atomicity. + - Start transactions with `BEGIN;` and end them with `COMMIT;` or `ROLLBACK;`. + - Handle transaction errors properly to prevent data corruption. + - Use savepoints to allow partial rollbacks within a transaction. + +- **Indexing:** + - Create indexes on columns frequently used in `WHERE` clauses and `JOIN` conditions. + - Avoid over-indexing, as it can slow down write operations. + - Consider using partial indexes for specific query patterns. + - Use appropriate index types (e.g., `B-tree`, `Hash`, `GIN`, `GiST`) based on the data and query requirements. + - Regularly analyze and maintain indexes using `ANALYZE` and `VACUUM`. + +- **Query Optimization:** + - Use `EXPLAIN ANALYZE` to analyze query execution plans and identify performance bottlenecks. + - Avoid using `SELECT *` and specify only the necessary columns. + - Use `JOIN` operations instead of subqueries where possible. + - Optimize `WHERE` clauses to reduce the number of rows processed. + - Consider using materialized views for frequently executed, complex queries. + - Rewrite slow performing queries with more efficient alternatives. + +- **PL/pgSQL Best Practices:** + - Keep PL/pgSQL functions and procedures short and focused. + - Use exception handling to gracefully handle errors. + - Use `RAISE NOTICE`, `RAISE WARNING`, and `RAISE EXCEPTION` for logging and error reporting. + - Avoid using cursors unless absolutely necessary; prefer set-based operations. + - Use the `STRICT` attribute to ensure that a function returns a value. + +- **Security Best Practices:** + - Grant the least privileges necessary to database users. + - Use roles to manage permissions. + - Regularly audit database activity. + - Encrypt sensitive data at rest and in transit. + - Use the `SECURITY DEFINER` attribute for functions that require elevated privileges. + - Configure appropriate authentication mechanisms (e.g., `SCRAM-SHA-256`). + +- **Connection Management:** + - Use connection pooling to reduce the overhead of establishing new connections. + - Close connections when they are no longer needed. + - Configure connection timeouts to prevent idle connections from consuming resources. + +- **Backup and Recovery:** + - Implement a robust backup and recovery strategy. + - Regularly back up the database. + - Test the recovery process to ensure it works as expected. + - Consider using replication for high availability. + +- **Code Organization and Structure:** + - Organize database objects into schemas based on functionality or application modules. + - Use version control for database scripts and migrations. + - Follow a consistent directory structure for database-related files. + +- **Common Pitfalls and Gotchas:** + - Avoid using reserved keywords as identifiers. + - Be aware of the limitations of data types (e.g., maximum length of `VARCHAR` columns). + - Handle null values carefully. + - Test database changes thoroughly before deploying them to production. + +- **Tooling and Environment:** + - Use a code editor with SQL syntax highlighting and autocompletion. + - Use a database client tool for querying and managing the database (e.g., `pgAdmin`, `DBeaver`, `psql`). + - Use a version control system (e.g., Git) to manage database scripts and migrations. + - Use a CI/CD pipeline to automate database deployments. + +- **C Coding Standards (When Extending PostgreSQL):** + - Adhere to the C99 standard. Only use language features available in the C99 standard. + - Follow PostgreSQL's source code conventions ([PostgreSQL Documentation](https://www.postgresql.org/docs/current/source-conventions.html)). + - Manage memory carefully; avoid memory leaks. + - Properly handle errors and exceptions. + - Document code thoroughly. + +- **Use Case specific optimizations:** + - Time series data: Consider timescaledb extension + - Geospatial data: Consider postgis extension + +@file ./rules/postgresql_security_rules.mdc \ No newline at end of file diff --git a/.cursor/rules/postman.mdc b/.cursor/rules/postman.mdc new file mode 100644 index 000000000..dc071e5d3 --- /dev/null +++ b/.cursor/rules/postman.mdc @@ -0,0 +1,100 @@ +--- +description: This rule provides best practices for effectively using Postman for API testing, covering code organization, common patterns, performance, security, testing, and tooling to ensure robust and maintainable API tests. +globs: *.postman_collection.json +--- +# Postman API Testing Best Practices + +This document outlines best practices for using Postman for API testing, covering aspects like project structure, common patterns, performance optimization, security considerations, testing strategies, common pitfalls, and tooling. + +## 1. Code Organization and Structure + +Effective organization is crucial for maintainability and collaboration. While Postman doesn't directly enforce code organization like a traditional codebase, the way you structure your collections and environments significantly impacts your workflow. + +- **Workspaces:** Utilize workspaces to separate projects and collections logically. Different workspaces can represent different projects, teams, or stages of development. +- **Collections:** Group related API requests into collections. Each collection should represent a specific feature or API domain. +- **Folders:** Organize requests within collections using folders. Folders can represent different workflows or API endpoints. For example: + + MyProject + └── Collections + └── User Authentication + ├── Register User + ├── Login User + ├── Forgot Password + └── Reset Password + +- **Environment Variables:** Use environments to manage different configurations (development, staging, production) without hardcoding sensitive information. +- **File Naming Conventions:** Use consistent and descriptive names for collections, folders, and requests. + - `Collection`: `[ProjectName] - [APIDomain] API Tests` (e.g., `ECommerce - User Authentication API Tests`) + - `Folder`: `[Endpoint]` or `[Workflow]` (e.g., `/users` or `Register User Workflow`) + - `Request`: `[HTTPMethod] [EndpointDescription]` (e.g., `POST Register User`) + +## 2. Common Patterns and Anti-patterns + +- **Environment-Specific Variables:** Use environment variables to store values that change based on the environment (e.g., API URLs, authentication tokens). Create separate environments for development, staging, and production. + + - **Pattern:** Create environment variables for the base URL, API keys, and other environment-specific settings. + - **Anti-pattern:** Hardcoding URLs or API keys directly into requests. + +- **Data-Driven Testing:** Use data-driven testing to run the same request multiple times with different data sets. This can be achieved by importing data from CSV or JSON files. + + - **Pattern:** Use the Collection Runner to iterate through data files and automatically populate request parameters. + - **Anti-pattern:** Manually creating multiple requests with slightly different data. + +- **Reusing Code with Pre-request Scripts and Tests:** Use pre-request scripts to dynamically generate request parameters or perform setup tasks. Use test scripts to validate API responses and set variables for subsequent requests. + + - **Pattern:** Use pre-request scripts to generate authentication tokens or timestamps. Use test scripts to extract data from responses and store them in environment variables for use in other requests. + - **Anti-pattern:** Duplicating the same code in multiple requests. + +- **Error Handling:** Implement robust error handling to gracefully handle unexpected API responses or network issues. + + - **Pattern:** Check for error status codes (e.g., 4xx, 5xx) in your test scripts and log detailed error messages. + - **Anti-pattern:** Ignoring error responses or assuming that all requests will succeed. + +## 3. Performance Considerations + +- **Minimize Request Size:** Reduce the size of your requests by sending only the necessary data. Avoid including unnecessary headers or body parameters. +- **Optimize Test Scripts:** Write efficient test scripts to minimize execution time. Avoid using complex or unnecessary logic in your test scripts. +- **Use Monitors:** Utilize Postman Monitors to schedule regular performance tests and track API response times. +- **Parallel Execution:** The collection runner in Postman executes requests sequentially. If parallel execution is required (for load testing, for example), consider Newman and external load testing tools. + +## 4. Security Best Practices + +- **Input Validation:** While Postman itself doesn't directly handle server-side input validation, your tests should validate that the API responds appropriately to invalid input. + - **Pattern:** Send requests with invalid or malicious input and verify that the API returns appropriate error messages and status codes. + +- **Authentication and Authorization:** Implement secure authentication and authorization mechanisms to protect your APIs. + - **Pattern:** Use environment variables to store API keys, tokens, and other credentials. Implement authentication flows (e.g., OAuth 2.0) in your collections. + - **Anti-pattern:** Hardcoding credentials directly into requests or storing them in plain text. + +- **Data Protection:** Ensure that sensitive data is encrypted in transit and at rest. + - **Pattern:** Use HTTPS to encrypt communication between Postman and your APIs. Implement data masking or redaction in your test scripts to prevent sensitive data from being exposed. + +## 5. Testing Approaches + +- **Unit Testing:** While traditional unit testing is not directly applicable to Postman, you can create individual requests to test specific API endpoints in isolation. This verifies the individual endpoint is returning expected data. +- **Integration Testing:** Use collections to test the integration between different API endpoints. This verifies that data is passed correctly between different parts of your application. +- **End-to-End Testing:** Create workflows to test the entire user experience, from start to finish. This verifies that all parts of your application are working together correctly. +- **Test Organization:** + - Group tests by functionality. + - Use descriptive names for tests. + - Keep tests small and focused. + +- **Mocking and Stubbing:** + - Use Postman's mock server feature to create mock APIs for testing purposes. This allows you to test your application without relying on a live API. + +## 6. Common Pitfalls and Gotchas + +- **Hardcoding Values:** Avoid hardcoding values directly into requests. Use environment variables instead. +- **Ignoring Error Responses:** Always check for error responses and handle them appropriately. +- **Not Versioning Collections:** Version your collections to track changes and ensure that your tests are up to date. +- **Not Documenting Tests:** Document your tests to make them easier to understand and maintain. +- **Misunderstanding Scope of Variables**: Be mindful of variable scope (global, collection, environment, local) and how it affects test execution. + +## 7. Tooling and Environment + +- **Postman CLI (Newman):** Use Newman to run Postman collections from the command line. This is essential for CI/CD integration. +- **Version Control (Git):** Store your Postman collections in a Git repository to track changes and collaborate with team members. +- **Linting and Formatting:** While Postman doesn't have built-in linting, ensure consistency in request structures and test scripts. + + - **Build Configuration:** Use Newman with CI/CD tools (Jenkins, GitLab CI, GitHub Actions) to automate API testing. + - **Deployment:** Deploy API specifications and Postman collections alongside your API for easier testing and documentation. \ No newline at end of file diff --git a/.cursor/rules/prisma.mdc b/.cursor/rules/prisma.mdc new file mode 100644 index 000000000..2e190b6f0 --- /dev/null +++ b/.cursor/rules/prisma.mdc @@ -0,0 +1,190 @@ +--- +description: Enforces Prisma best practices for schema design, data access, and application security. Provides guidelines for writing efficient, secure, and maintainable Prisma applications. +globs: *.prisma +--- +- **General Principles:** + - **Never expose the raw Prisma client directly in APIs.** Instead, create abstraction layers (e.g., repositories or services) to handle data access logic. This protects you from accidentally exposing sensitive database details and allows you to easily change your data access implementation in the future. + - **Always use input validation** before performing any database operations. Use a validation library like Zod or Yup to ensure that user input conforms to your expected schema and data types. This helps prevent injection attacks and data corruption. + - **Implement row-level security (RLS) where applicable.** If your application handles sensitive data, use Prisma policies or database-level RLS to restrict data access based on user roles or permissions. Carefully design your policies to prevent unauthorized data access. + - **Sanitize and validate all user inputs** to prevent injection attacks, such as SQL injection. Use Prisma's built-in features to escape user input and prevent it from being interpreted as code. + +- **Code Organization and Structure:** + - **Directory Structure:** + - `prisma/`: Contains the `schema.prisma` file and any seed scripts or migrations. + - `src/`: Contains the main application code. + - `src/lib/prisma.ts`: A single module that exports an instance of the Prisma client. This promotes reuse and simplifies dependency injection. + - `src/models/`: Database model definitions as classes, abstracting Prisma calls. + - `src/repositories/`: Contains data access logic, abstracting Prisma queries. + - `src/services/`: Contains business logic, orchestrating data access through repositories. + - **File Naming Conventions:** + - Use descriptive names for files and directories that reflect their purpose. + - Use consistent naming conventions (e.g., camelCase for variables, PascalCase for components). + - **Module Organization:** + - Organize code into logical modules with clear responsibilities. + - Use dependency injection to decouple modules and improve testability. + - **Component Architecture:** + - Design components that are reusable, testable, and maintainable. + - Follow the single responsibility principle for each component. + - **Code Splitting Strategies:** + - Use dynamic imports to load code on demand and reduce initial bundle size. + - Split code based on routes, features, or user roles. + +- **Common Patterns and Anti-patterns:** + - **Design Patterns:** + - **Repository Pattern:** Abstract data access logic behind repositories. + - **Service Layer Pattern:** Encapsulate business logic in service classes. + - **Unit of Work Pattern:** Manage transactions across multiple repositories. + - **Recommended Approaches:** + - Use Prisma's relation features to model complex relationships between entities. + - Use Prisma's transaction features to ensure data consistency. + - Use Prisma's filtering and sorting options to optimize queries. + - **Anti-patterns:** + - **Exposing the Prisma client directly to the client-side.** This is a major security risk. + - **Writing complex queries directly in components.** Move query logic to repositories or services. + - **Ignoring error handling.** Always handle potential errors when interacting with the database. + - **State Management:** + - Consider using a state management library like Redux, Zustand, or Jotai for complex applications. + - Use server-side data fetching for initial data loading and hydration. + - **Error Handling:** + - Use try-catch blocks to handle potential errors when interacting with the database. + - Log errors to a centralized logging system for monitoring and debugging. + - Return meaningful error messages to the client-side. + +- **Performance Considerations:** + - **Optimization Techniques:** + - Use Prisma's connection pooling to reduce database connection overhead. + - Use Prisma's batching features to reduce the number of database roundtrips. + - Optimize database queries by using indexes and filtering data on the server-side. + - **Memory Management:** + - Avoid loading large amounts of data into memory at once. Use pagination or streaming to process data in smaller chunks. + - Clean up resources after use, such as database connections and file handles. + - **Rendering Optimization:** + - Use virtualization for large lists or tables. + - Optimize images and other assets. + - **Bundle Size Optimization:** + - Use code splitting to reduce the initial bundle size. + - Remove unused code and dependencies. + - **Lazy Loading Strategies:** + - Load data on demand when it is needed. + - Use placeholders or skeletons to improve the user experience during loading. + +- **Security Best Practices:** + - **Common Vulnerabilities:** + - SQL injection: Prevented by using Prisma's prepared statements and escaping user input. + - Cross-site scripting (XSS): Prevented by sanitizing user input and output. + - Cross-site request forgery (CSRF): Prevented by using CSRF tokens. + - **Input Validation:** + - Use a validation library to validate all user input. + - Validate data types, lengths, and formats. + - **Authentication and Authorization:** + - Use a secure authentication library like NextAuth.js or Clerk. + - Implement role-based access control (RBAC) to restrict access to sensitive data and functionality. + - **Data Protection:** + - Encrypt sensitive data at rest and in transit. + - Use HTTPS to secure communication between the client and server. + - **Secure API Communication:** + - Use API keys or tokens to authenticate API requests. + - Rate limit API requests to prevent abuse. + +- **Testing Approaches:** + - **Unit Testing:** + - Test individual components and functions in isolation. + - Use mocking and stubbing to isolate components from external dependencies. + - **Integration Testing:** + - Test the interaction between different components and modules. + - Test database interactions using a test database. + - **End-to-End Testing:** + - Test the entire application from end to end. + - Use a testing framework like Cypress or Playwright. + - **Test Organization:** + - Organize tests into logical suites based on functionality. + - Use descriptive names for tests. + - **Mocking and Stubbing:** + - Use mocking to replace external dependencies with test doubles. + - Use stubbing to control the behavior of dependencies. + +- **Common Pitfalls and Gotchas:** + - **Frequent Mistakes:** + - Forgetting to handle errors when interacting with the database. + - Exposing the Prisma client directly to the client-side. + - Writing complex queries directly in components. + - **Edge Cases:** + - Handling large datasets. + - Dealing with concurrency issues. + - Managing database migrations. + - **Version-Specific Issues:** + - Be aware of breaking changes between Prisma versions. + - Use version control to manage dependencies. + - **Compatibility Concerns:** + - Ensure that your code is compatible with different browsers and devices. + - Test your application on different environments. + - **Debugging Strategies:** + - Use logging to track down errors. + - Use a debugger to step through code. + - Use Prisma's query logging feature to inspect database queries. + +- **Tooling and Environment:** + - **Recommended Development Tools:** + - VS Code with the Prisma extension. + - Prisma Studio for visualizing and managing data. + - Docker for containerizing the application. + - **Build Configuration:** + - Use a build tool like Webpack, Parcel, or esbuild to bundle and optimize code. + - Configure the build tool to generate production-ready code. + - **Linting and Formatting:** + - Use ESLint and Prettier to enforce code style and prevent errors. + - Configure the linter and formatter to use consistent settings across the project. + - **Deployment Best Practices:** + - Use a CI/CD pipeline to automate deployments. + - Deploy to a cloud platform like Vercel, Netlify, or AWS. + - **CI/CD Integration:** + - Integrate with CI/CD tools like GitHub Actions, GitLab CI, or CircleCI. + - Run tests and linters as part of the CI/CD pipeline. + +- **Specific Prisma Configuration:** + - **Schema Design:** + - Use clear and concise names for models and fields. + - Define relationships between models using Prisma's relation features. + - Use indexes to optimize query performance. + - Consider using enums for fields with a limited set of values. + - Properly use `@id`, `@unique`, and `@relation` directives for optimal schema design. + - **Data Types:** + - Use the appropriate data types for each field. + - Use `DateTime` for storing timestamps. + - Use `Json` for storing complex data structures. + - **Prisma Client Generation:** + - Ensure your `.env` file is correctly configured for database connection. + - Regularly update the Prisma client to the latest version. + - Use `prisma generate` to generate the client after schema changes. + +- **Query Optimization:** + - **Select only the necessary fields** to reduce the amount of data transferred from the database. + - **Use `include` and `select` carefully** to optimize the query shape. + - **Avoid N+1 queries** by using Prisma's `include` or `join` features. + - **Use pagination** to limit the number of results returned by a query. + - **Use `take` and `skip`** for efficient pagination. + +- **Transaction Management:** + - **Use Prisma's `prisma.$transaction` method** to ensure atomicity of database operations. + - **Handle potential errors** within the transaction to prevent data inconsistency. + - **Keep transactions short** to minimize the impact on database performance. + +- **Seed Data Management:** + - **Create a seed script** to populate the database with initial data. + - **Use Prisma's `prisma.create` method** to create seed data. + - **Use environment variables** to configure seed data for different environments. + +- **Migration Management:** + - **Use Prisma Migrate** to manage database schema changes. + - **Create migrations** for each schema change. + - **Apply migrations** to the database using `prisma migrate deploy`. + - **Use shadow database** to test migrations before deploying them to production. + +- **Monitoring and Logging:** + - **Use Prisma's query logging feature** to monitor database queries. + - **Log errors and warnings** to a centralized logging system. + - **Monitor database performance** using tools like Prometheus or Grafana. + +- **Further Study:** + - Always refer to the [official Prisma documentation](https://www.prisma.io/docs/) for the most up-to-date information. + - Consider taking an [official Prisma course](https://www.prisma.io/learn/). \ No newline at end of file diff --git a/.cursor/rules/puppeteer.mdc b/.cursor/rules/puppeteer.mdc new file mode 100644 index 000000000..26d9eba48 --- /dev/null +++ b/.cursor/rules/puppeteer.mdc @@ -0,0 +1,284 @@ +--- +description: This rule file outlines best practices for Puppeteer, covering code organization, performance, security, testing, and common pitfalls. It aims to guide developers in building robust and maintainable Puppeteer applications. +globs: *.js +--- +# Puppeteer Best Practices + +This document outlines best practices for using Puppeteer, covering code organization, common patterns, performance, security, testing, and common pitfalls. Following these guidelines will help you write more robust, maintainable, and efficient Puppeteer applications. + +## 1. Code Organization and Structure + +### Directory Structure + +Consistent directory structure is crucial for maintainability. Here's a recommended structure: + + +project-root/ +├── src/ +│ ├── scrapers/ # Directory for specific scraping logic +│ │ ├── product_scraper.js +│ │ ├── news_scraper.js +│ │ └── ... +│ ├── utils/ # Utility functions (e.g., CSV export, error handling) +│ │ ├── csv_utils.js +│ │ ├── error_handler.js +│ │ └── ... +│ ├── config/ # Configuration files (e.g., URLs, selectors) +│ │ ├── urls.js +│ │ ├── selectors.js +│ │ └── ... +│ ├── models/ # Data models (e.g., Product, Article) +│ │ ├── product.js +│ │ ├── article.js +│ │ └── ... +│ ├── core/ # Core puppeteer setup and helper functions +│ │ ├── browser_manager.js # Browser launch and closing +│ │ ├── page_actions.js # Reusable page interaction functions +│ │ └── ... +│ └── index.js # Main entry point +├── tests/ +│ ├── unit/ +│ ├── integration/ +│ └── e2e/ +├── .eslintrc.js # ESLint configuration +├── .prettierrc.js # Prettier configuration +├── package.json +└── README.md + + +### File Naming Conventions + +* **Scrapers:** `[target_site]_scraper.js` (e.g., `amazon_scraper.js`, `twitter_scraper.js`) +* **Utility functions:** `[utility_name]_utils.js` (e.g., `csv_utils.js`, `string_utils.js`) +* **Configuration files:** Use descriptive names like `urls.js`, `selectors.js`, `api_keys.js`. +* **Data models:** Singular nouns in PascalCase (e.g., `Product.js`, `Article.js`). +* **Core Files:** `browser_manager.js`, `page_actions.js`, `request_interceptor.js` + +### Module Organization + +* **Separate concerns:** Each module should have a single responsibility. For example, one module handles browser launching, another handles navigation, and another handles data extraction. +* **Use ES modules:** Use `import` and `export` for better readability and maintainability. +* **Avoid circular dependencies:** Ensure modules don't depend on each other in a circular manner. + +### Component Architecture + +* **Reusable components:** Create reusable components for common tasks like navigating to a page, filling out a form, or waiting for an element to load. These can be organized into functions within the `core` directory and reused across multiple scrapers. +* **Configuration-driven:** Design components to be configurable via parameters or configuration files. This allows for easy adaptation to different websites or scraping scenarios. + +### Code Splitting + +* **Scraper-specific bundles:** If you have multiple scrapers with significant code differences, consider creating separate bundles for each scraper to reduce initial load time. +* **Dynamic imports:** Use dynamic imports (`import()`) to load modules only when they are needed. This can be useful for loading scraper-specific configurations or utility functions. + +## 2. Common Patterns and Anti-patterns + +### Design Patterns + +* **Strategy Pattern:** Use the strategy pattern to handle different scraping strategies for different websites or content types. Create an interface for scraping, and implement concrete strategies for each website. This promotes code reusability and flexibility. +* **Factory Pattern:** Employ the factory pattern to create browser instances with specific configurations. This is especially useful when dealing with different browser profiles or proxy settings. +* **Page Object Model (POM):** Encapsulate page elements and interactions into separate page objects. This makes tests more readable and maintainable. +* **Observer Pattern:** Implement an observer pattern to monitor browser events (e.g., network requests, console messages) and react accordingly. This can be helpful for debugging and error handling. + +### Recommended Approaches + +* **Resource Blocking:** Block unnecessary resources (images, CSS, fonts) to improve scraping speed and reduce bandwidth usage. Use `page.route()` to intercept requests and abort those that are not needed. +* **Dynamic Content Handling:** Use `page.waitForSelector()`, `page.waitForResponse()`, or `page.waitForFunction()` instead of fixed delays to ensure elements are loaded before interacting with them. Always use try/catch blocks around these to gracefully handle timeouts. +* **Error Handling:** Implement robust error handling with `try...catch` blocks and logging. Ensure browser instances are properly closed even if errors occur. +* **User-Agent Spoofing:** Set the `User-Agent` header to mimic real user behavior and avoid detection by anti-bot systems. +* **Proxy Rotation:** Use a proxy server and rotate it periodically to avoid IP bans. +* **Headless Mode vs. Headed Mode:** Use headless mode for faster execution but switch to headed mode for debugging when necessary. Consider `slowMo` to slow down execution in headed mode for better visibility. + +### Anti-patterns and Code Smells + +* **Hardcoding selectors:** Avoid hardcoding CSS selectors or XPath expressions directly in your code. Instead, store them in a configuration file or a separate module and reuse them. +* **Using fixed timeouts:** Avoid using `setTimeout` or `sleep` for waiting. Use `waitForSelector`, `waitForResponse`, or `waitForFunction` instead. +* **Ignoring errors:** Never ignore errors. Always log errors and handle them gracefully. +* **Overly complex XPath expressions:** Avoid using overly complex XPath expressions that are prone to breaking when the page structure changes. Use CSS selectors whenever possible. +* **Direct DOM scraping:** Try to extract data directly from API responses instead of scraping the DOM whenever possible. This is more efficient and stable. +* **Not closing the browser:** Failing to close the browser instance after the scraping task is complete leads to memory leaks and resource exhaustion. +* **Parallel scraping without control:** Launching too many parallel scrapers can overload the target server and lead to IP bans or performance issues. Implement rate limiting and concurrency control. + +### State Management + +* **Stateless components:** Design your scrapers to be as stateless as possible. Pass all necessary data as arguments to functions or components. +* **Centralized configuration:** Store global configurations (e.g., API keys, URLs, proxy settings) in a centralized configuration file or environment variables. + +### Error Handling Patterns + +* **Try-Catch Blocks:** Wrap Puppeteer operations within `try...catch` blocks to catch potential errors. Log the errors and handle them appropriately (e.g., retry the request, skip the item, or stop the scraper). +* **Global Error Handling:** Implement a global error handler to catch unhandled exceptions and prevent the application from crashing. +* **Retry Mechanism:** Implement a retry mechanism to automatically retry failed requests. Use exponential backoff to avoid overloading the target server. +* **Circuit Breaker:** Implement a circuit breaker pattern to prevent repeated failures from cascading and bringing down the scraper. + +## 3. Performance Considerations + +### Optimization Techniques + +* **Resource Blocking:** Block unnecessary resources (images, CSS, fonts) using `page.route()`. +* **Viewport Optimization:** Set the viewport to a smaller size to reduce rendering time and memory usage. +* **JavaScript Optimization:** Minimize the amount of JavaScript that needs to be executed on the page. Use `evaluateHandle` for extracting complex data structures. +* **Caching:** Cache frequently accessed data to reduce the number of requests to the target server. +* **Concurrency Control:** Limit the number of concurrent browser instances or pages to avoid overloading the system or the target server. Use libraries like `p-queue` or `async` for managing concurrency. +* **Disable JavaScript if possible**: `await page.setJavaScriptEnabled(false)` can drastically improve performance if the target site doesn't require JavaScript for the data you need. +* **EvaluateHandle instead of evaluate**: For returning complex objects use `evaluateHandle` rather than `evaluate`. `evaluateHandle` returns a reference to the object in the browser, avoiding serialization overhead. + +### Memory Management + +* **Close Browser Instances:** Always close browser instances and pages when they are no longer needed. Use `browser.close()` and `page.close()`. +* **Avoid Memory Leaks:** Be mindful of memory leaks in your code, especially when using closures or event listeners. Remove event listeners when they are no longer needed. +* **Garbage Collection:** Force garbage collection periodically to release unused memory. This can be done using `await page.evaluate(() => gc());`. However, use this sparingly as it can impact performance. + +### Rendering Optimization + +* **Disable GPU:** Disable GPU acceleration for headless browsers to reduce memory usage. Use the `--disable-gpu` flag when launching the browser. +* **Set Viewport:** `await page.setViewport({ width: 800, height: 600 });` Avoid very high resolutions when not needed. + +### Bundle Size Optimization + +* **Tree Shaking:** Use a bundler like Webpack or Rollup with tree shaking enabled to remove unused code. +* **Code Minification:** Minify your JavaScript code to reduce its size. +* **Dependency Optimization:** Analyze your dependencies and remove any that are not needed. Consider using smaller alternatives for large libraries. + +### Lazy Loading + +* **Lazy Loading of Images:** If you are scraping images, consider lazy loading them to reduce initial page load time. Implement this on the scraped site side if applicable. +* **On-Demand Scraping:** Only scrape data that is currently needed. Defer scraping of less important data until it is requested. + +## 4. Security Best Practices + +### Common Vulnerabilities and Prevention + +* **Code Injection:** Prevent code injection by carefully validating any user-supplied input that is used in `evaluate()` or other code execution contexts. Avoid using `eval()` altogether. +* **Cross-Site Scripting (XSS):** Be aware of XSS vulnerabilities when scraping data from untrusted sources. Sanitize data before displaying it to users. +* **Denial of Service (DoS):** Implement rate limiting and concurrency control to prevent your scraper from overloading the target server and causing a denial of service. +* **IP Bans:** Use proxy servers and rotate them periodically to avoid IP bans. Also, respect the target website's `robots.txt` file. + +### Input Validation + +* **Sanitize User Input:** Sanitize any user-supplied input before using it in selectors or other code that interacts with the browser. Use libraries like `DOMPurify` to sanitize HTML. +* **Validate URLs:** Validate URLs before navigating to them using `page.goto()`. Ensure that the URLs are safe and trusted. + +### Authentication and Authorization + +* **Secure Credentials:** Store API keys, passwords, and other sensitive credentials securely using environment variables or a dedicated secrets management tool. Never hardcode credentials in your code. +* **Session Management:** If you need to maintain a session with the target website, use cookies and store them securely. Consider using the `page.cookies()` method to manage cookies. + +### Data Protection + +* **Encryption:** Encrypt sensitive data before storing it. Use libraries like `crypto` to encrypt data. +* **Data Masking:** Mask sensitive data (e.g., credit card numbers, social security numbers) before displaying it to users or storing it in logs. + +### Secure API Communication + +* **HTTPS:** Always use HTTPS for API communication. Verify that the server's SSL certificate is valid. +* **API Rate Limiting:** Implement rate limiting on your API endpoints to prevent abuse. +* **Authentication:** Use authentication to protect your API endpoints. Consider using API keys or JWT tokens. + +## 5. Testing Approaches + +### Unit Testing + +* **Mock Puppeteer Objects:** Mock Puppeteer objects (e.g., `browser`, `page`) to isolate units of code during testing. Use libraries like `jest-puppeteer` or `mock-puppeteer` for mocking. +* **Test Utility Functions:** Unit test utility functions (e.g., CSV export, error handling) to ensure they are working correctly. +* **Verify Logic:** Write unit tests to verify the logic of your scraping algorithms. + +### Integration Testing + +* **Test Interactions:** Write integration tests to verify the interactions between different components of your scraper (e.g., browser launching, page navigation, data extraction). +* **Use Test Servers:** Use a test server or mock API to simulate the target website during integration testing. Libraries like `nock` can be helpful. +* **Verify Data Flow:** Write integration tests to verify that data is flowing correctly through your scraper. + +### End-to-End Testing + +* **Simulate User Flows:** Write end-to-end tests to simulate real user flows and verify that the scraper is working as expected from start to finish. +* **Test Real Websites:** Test your scraper against real websites to ensure that it is robust and reliable. +* **Use Headless Mode:** Run end-to-end tests in headless mode for faster execution. +* **Screenshot Comparison**: Integrate screenshot comparison tools to detect visual regressions in your scraped data. This is particularly useful when target website layouts change frequently. + +### Test Organization + +* **Separate Test Files:** Create separate test files for unit tests, integration tests, and end-to-end tests. +* **Use Descriptive Names:** Use descriptive names for your test files and test cases. +* **Organize Tests by Feature:** Organize your tests by feature or functionality. + +### Mocking and Stubbing + +* **Mock Network Requests:** Mock network requests to avoid making real requests to the target server during testing. Use libraries like `nock` for mocking. +* **Stub Puppeteer Methods:** Stub Puppeteer methods (e.g., `page.goto()`, `page.$()`) to control the behavior of your scraper during testing. + +## 6. Common Pitfalls and Gotchas + +### Frequent Mistakes + +* **Not Handling Dynamic Content:** Failing to handle dynamic content and relying on fixed delays. +* **Using Brittle Selectors:** Using overly complex or fragile CSS selectors or XPath expressions. +* **Ignoring Errors:** Not handling errors gracefully and letting the scraper crash. +* **Not Closing Browser Instances:** Forgetting to close browser instances and leaking memory. +* **Not Respecting `robots.txt`:** Ignoring the `robots.txt` file and scraping restricted areas of the website. +* **Missing Rate Limiting:** Not implementing rate limiting and overloading the target server. +* **Incorrectly Handling Async Operations**: Forgetting to `await` asynchronous calls, leading to race conditions or unexpected behavior. + +### Edge Cases + +* **Website Layout Changes:** Websites frequently change their layout, which can break your scraper. Implement robust selector strategies and monitor your scraper for errors. +* **Anti-Bot Measures:** Websites implement anti-bot measures to detect and block scrapers. Use techniques like user-agent spoofing, proxy rotation, and CAPTCHA solving to avoid detection. +* **Infinite Scroll:** Handling infinite scroll pages requires special attention to avoid scraping the same data multiple times. Use techniques like pagination or scroll-to-bottom with debouncing. +* **Frames and Iframes:** Dealing with content inside frames or iframes requires switching to the frame context before scraping. Use `page.frame()` to get a frame and then interact with it. +* **Shadow DOM:** Accessing elements inside shadow DOM requires using `shadowRoot` property or special selectors. + +### Version-Specific Issues + +* **API Changes:** Be aware of API changes between different versions of Puppeteer. Consult the release notes for each version. +* **Chromium Updates:** Puppeteer is tied to specific versions of Chromium. Chromium updates can sometimes break your scraper. Test your scraper regularly after Chromium updates. + +### Compatibility Concerns + +* **Node.js Version:** Ensure that you are using a compatible version of Node.js. Refer to the Puppeteer documentation for compatibility information. +* **Operating System:** Puppeteer may behave differently on different operating systems. Test your scraper on all target operating systems. + +### Debugging Strategies + +* **Headless Mode:** Switch to headed mode for debugging to visually inspect the browser. Use `headless: false` when launching the browser. +* **Console Logging:** Use `console.log()` to log debugging information to the console. Capture console output from the page using `page.on('console', msg => console.log('PAGE LOG:', msg.text()));` +* **`slowMo` Option:** Use the `slowMo` option when launching the browser to slow down the execution and make it easier to follow along. `const browser = await puppeteer.launch({ headless: false, slowMo: 50 });` +* **Debugger Statement:** Add `debugger;` statements to your code to pause execution and inspect variables in the Chrome DevTools. +* **Screenshot Debugging:** Take screenshots at key points in the script execution to visually debug the page state. `await page.screenshot({path: 'screenshot.png'});` +* **Performance Tracing:** Use Chrome DevTools to record a performance trace and identify performance bottlenecks. `await page.tracing.start({path: 'trace.json'});` and `await page.tracing.stop();` +* **Browser DevTools:** Use `devtools: true` when launching puppeteer to directly show the Browser DevTools. `const browser = await puppeteer.launch({ devtools: true });` + +## 7. Tooling and Environment + +### Recommended Development Tools + +* **IDE:** Visual Studio Code (VS Code) is a popular choice for Puppeteer development due to its excellent JavaScript support, debugging capabilities, and extensions. +* **Debugger:** The Chrome DevTools debugger is essential for debugging Puppeteer code. Use the debugger statement or launch the browser in headed mode to use the DevTools. +* **Linter:** ESLint helps enforce coding standards and identify potential errors. Configure ESLint with appropriate rules for Puppeteer code. +* **Formatter:** Prettier automatically formats your code to ensure consistency. Configure Prettier with appropriate settings for JavaScript code. + +### Build Configuration + +* **Bundler:** Use a bundler like Webpack or Rollup to bundle your code and dependencies into a single file. This simplifies deployment and improves performance. +* **Transpiler:** Use a transpiler like Babel to convert your code to a compatible version of JavaScript. +* **Environment Variables:** Use environment variables to configure your application for different environments (e.g., development, staging, production). + +### Linting and Formatting + +* **ESLint:** Configure ESLint with recommended rules for Puppeteer code. Use the `eslint:recommended` preset and customize it to fit your needs. +* **Prettier:** Configure Prettier with consistent settings for JavaScript code. Use the `prettier` command to automatically format your code. +* **.editorconfig**: Use .editorconfig to enforce consistent coding styles across different editors. + +### Deployment Best Practices + +* **Docker:** Use Docker to containerize your application and ensure consistency across different environments. Create a Dockerfile that specifies the dependencies and configuration for your application. +* **Serverless Functions:** Deploy your scraper as a serverless function (e.g., AWS Lambda, Google Cloud Functions) to avoid managing servers. +* **Environment Variables:** Use environment variables to configure your application in the deployment environment. +* **CI/CD Pipeline:** Automate the deployment process using a CI/CD pipeline. Use tools like Jenkins, GitLab CI, or GitHub Actions to build, test, and deploy your application. + +### CI/CD Integration + +* **Automated Testing:** Integrate automated testing into your CI/CD pipeline to ensure that your scraper is working correctly before deploying it to production. Run unit tests, integration tests, and end-to-end tests as part of the pipeline. +* **Code Quality Checks:** Integrate code quality checks into your CI/CD pipeline to ensure that your code meets coding standards and best practices. Use ESLint and Prettier to check code quality. +* **Deployment Automation:** Automate the deployment process to ensure that your application is deployed consistently and reliably. Use tools like Ansible or Terraform to automate deployment. +* **Monitor Deploys:** Integrate monitoring to immediately notify you of any broken deploys. + +By following these best practices, you can develop robust, scalable, and maintainable Puppeteer applications. \ No newline at end of file diff --git a/.cursor/rules/pydantic.mdc b/.cursor/rules/pydantic.mdc new file mode 100644 index 000000000..bf8a36f58 --- /dev/null +++ b/.cursor/rules/pydantic.mdc @@ -0,0 +1,207 @@ +--- +description: Comprehensive best practices and coding standards for utilizing Pydantic effectively in Python projects, covering code organization, performance, security, and testing. +globs: *.py +--- +- **Model Definition:** + - Use `BaseModel` to define data schemas with type annotations for clarity and automatic validation. + - Prefer simple models that encapsulate a single concept to maintain readability and manageability. + - Use nested models for complex data structures while ensuring each model has clear validation rules. + - Always define a `Config` class within your model to control model behavior. + +- **Validation and Error Handling:** + - Implement built-in and custom validators to enforce data integrity. + - Utilize `@field_validator` for field-specific rules and `@root_validator` for cross-field validation. + - Ensure that validation errors are user-friendly and logged for debugging purposes. + - Custom error messages should be informative and guide the user on how to correct the data. + - Use `ValidationError` to catch and handle validation errors. + +- **Performance Optimization:** + - Consider using lazy initialization and avoid redundant validation where data is already trusted. + - Use Pydantic's configuration options to control when validation occurs, which can significantly enhance performance in high-throughput applications. + - Use `model_rebuild` to dynamically rebuild models when related schema change. + - Consider using `__slots__` in the `Config` class to reduce memory footprint. + +- **Code Organization and Structure:** + - **Directory Structure:** + - Adopt a modular structure: `src/`, `tests/`, `docs/`. + - Models can reside in `src/models/`. + - Validators in `src/validators/`. + - Example: + + project_root/ + ├── src/ + │ ├── __init__.py + │ ├── models/ + │ │ ├── __init__.py + │ │ ├── user.py + │ │ ├── item.py + │ ├── validators/ + │ │ ├── __init__.py + │ │ ├── user_validators.py + │ ├── main.py # Application entry point + ├── tests/ + │ ├── __init__.py + │ ├── test_user.py + │ ├── test_item.py + ├── docs/ + │ ├── ... + ├── .env + ├── pyproject.toml + ├── README.md + + - **File Naming:** + - Use snake_case for file names (e.g., `user_model.py`). + - Name model files after the primary model they define (e.g., `user.py` for `UserModel`). + - **Module Organization:** + - Group related models and validators into separate modules. + - Utilize `__init__.py` to make modules importable. + - **Component Architecture:** + - Employ a layered architecture (e.g., data access, business logic, presentation). + - Pydantic models are primarily used in the data access layer. + - **Code Splitting:** + - Split large models into smaller, manageable components using composition. + - Leverage inheritance judiciously. + +- **Common Patterns and Anti-patterns:** + - **Design Patterns:** + - **Data Transfer Object (DTO):** Pydantic models serve as DTOs. + - **Factory Pattern:** Create model instances using factory functions for complex initialization. + - **Repository Pattern:** Use repositories to abstract data access and validation logic. + - **Recommended Approaches:** + - Centralize validation logic in dedicated validator functions. + - Utilize nested models for complex data structures. + - Use `BaseSettings` for managing application settings. + - **Anti-patterns:** + - Embedding business logic directly into models. + - Overly complex inheritance hierarchies. + - Ignoring validation errors. + - Performing I/O operations within validator functions. + - **State Management:** + - Use immutable models whenever possible to simplify state management. + - Consider using state management libraries like `attrs` or `dataclasses` in conjunction with Pydantic for complex applications. + - **Error Handling:** + - Raise `ValidationError` exceptions when validation fails. + - Provide informative error messages. + - Log validation errors for debugging. + +- **Performance Considerations:** + - **Optimization Techniques:** + - Use `model_rebuild` to recompile models when their schema changes. + - Leverage `__slots__` in the `Config` class to reduce memory footprint. + - Use the `@cached_property` decorator to cache expensive computations. + - **Memory Management:** + - Be mindful of large lists or dictionaries within models, as they can consume significant memory. + - Use generators or iterators for processing large datasets. + - **Efficient Data Parsing:** + - Utilize `model_validate_json` and `model_validate` for efficient data parsing. + - **Controlling Validation:** + - Use `validate_default` and `validate_assignment` options in the `Config` class to control validation occurrence. + +- **Security Best Practices:** + - **Common Vulnerabilities:** + - Injection attacks (e.g., SQL injection) if model data is used directly in database queries. + - Cross-site scripting (XSS) if model data is displayed in web pages without proper escaping. + - Deserialization vulnerabilities if models are deserialized from untrusted sources. + - **Input Validation:** + - Always validate all incoming data using Pydantic models. + - Use appropriate type annotations and constraints to restrict input values. + - Sanitize input data to remove potentially harmful characters or sequences. + - **Authentication and Authorization:** + - Use authentication and authorization mechanisms to restrict access to sensitive data. + - Implement role-based access control (RBAC) to grant different levels of access to different users. + - **Data Protection:** + - Encrypt sensitive data at rest and in transit. + - Use secure storage mechanisms for storing API keys and other secrets. + - Mask sensitive data in logs and error messages. + - **Secure API Communication:** + - Use HTTPS for all API communication. + - Implement API rate limiting to prevent denial-of-service attacks. + +- **Testing Approaches:** + - **Unit Testing:** + - Test individual models and validators in isolation. + - Use parameterized tests to cover different input values and scenarios. + - Verify that validation errors are raised correctly. + - **Integration Testing:** + - Test the interaction between models and other components of the application. + - Use mock objects to simulate external dependencies. + - **End-to-End Testing:** + - Test the entire application flow from end to end. + - Use automated testing tools to simulate user interactions. + - **Test Organization:** + - Organize tests into separate modules based on the component being tested. + - Use descriptive test names to indicate the purpose of each test. + - **Mocking and Stubbing:** + - Use mock objects to simulate external dependencies such as databases or APIs. + - Use stub objects to provide predefined responses for certain functions or methods. + +- **Common Pitfalls and Gotchas:** + - **Frequent Mistakes:** + - Misusing Union Types: Using `Union` incorrectly can complicate type validation and handling. + - Optional Fields without Default Values: Forgetting to provide a default value for optional fields can lead to `None` values causing errors in your application. + - Incorrect Type Annotations: Assigning incorrect types to fields can cause validation to fail. For example, using `str` for a field that should be an `int`. + - **Edge Cases:** + - Handling complex validation logic with dependencies between fields. + - Dealing with large or deeply nested data structures. + - Handling different input formats (e.g., JSON, CSV). + - **Version-Specific Issues:** + - Be aware of breaking changes between Pydantic versions. + - Consult the Pydantic documentation for migration guides. + - **Compatibility Concerns:** + - Ensure compatibility between Pydantic and other libraries used in your project. + - Be mindful of potential conflicts with other validation libraries. + +- **Tooling and Environment:** + - **Development Tools:** + - Use a code editor or IDE with Pydantic support (e.g., VS Code with the Pylance extension). + - Use a static type checker like MyPy to catch type errors. + - Use a linter like Flake8 or Pylint to enforce code style. + - **Build Configuration:** + - Use a build tool like Poetry or Pipenv to manage dependencies. + - Specify Pydantic as a dependency in your project's configuration file. + - **Linting and Formatting:** + - Configure a linter and formatter to enforce consistent code style. + - Use pre-commit hooks to automatically run linters and formatters before committing code. + - **Deployment:** + - Use a deployment platform that supports Python applications (e.g., Heroku, AWS Elastic Beanstalk, Docker). + - Configure your deployment environment to install Pydantic and its dependencies. + - **CI/CD:** + - Integrate Pydantic tests into your CI/CD pipeline. + - Automatically run tests and linters on every commit. + +- **Getting Started with Pydantic:** + - Install Pydantic with `pip install pydantic` + - Define your data models using `BaseModel` and type hints + - Validate your data by instantiating the data models + - Handle validation errors using `try...except ValidationError` + +- **Example:** + python + from pydantic import BaseModel, ValidationError + from typing import List, Optional + + class Address(BaseModel): + street: str + city: str + zip_code: Optional[str] = None + + class User(BaseModel): + id: int + name: str + email: str + addresses: List[Address] + + try: + user_data = { + "id": 1, + "name": "John Doe", + "email": "invalid-email", + "addresses": [{ + "street": "123 Main St", + "city": "Anytown" + }] + } + user = User(**user_data) + print(user) + except ValidationError as e: + print(e.json()) \ No newline at end of file diff --git a/.cursor/rules/pygame.mdc b/.cursor/rules/pygame.mdc new file mode 100644 index 000000000..9af11c2c3 --- /dev/null +++ b/.cursor/rules/pygame.mdc @@ -0,0 +1,321 @@ +--- +description: This rule provides comprehensive guidelines for pygame development, covering code organization, performance, security, testing, and common pitfalls. It aims to establish best practices and coding standards for building maintainable, efficient, and secure pygame applications. +globs: *.py +--- +--- + +# Pygame Development Best Practices + +This document outlines best practices and coding standards for developing games and multimedia applications using the Pygame library in Python. Adhering to these guidelines will result in more maintainable, efficient, and secure code. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure + +A well-organized directory structure is crucial for project maintainability. Consider the following structure: + + +project_name/ +├── assets/ # Images, sounds, fonts, etc. +│ ├── images/ +│ ├── sounds/ +│ ├── fonts/ +│ └── ... +├── src/ # Source code +│ ├── main.py # Entry point +│ ├── modules/ # Custom modules +│ │ ├── game.py # Game logic +│ │ ├── player.py # Player class +│ │ ├── enemy.py # Enemy class +│ │ ├── ui.py # UI elements +│ │ └── ... +│ └── ... +├── tests/ # Unit and integration tests +│ ├── test_game.py +│ ├── test_player.py +│ └── ... +├── data/ # Game data (levels, configurations, etc.) +├── docs/ # Documentation +├── README.md # Project description and instructions +├── LICENSE # License information +├── requirements.txt # Dependencies +└── .gitignore # Files to ignore in Git + + +### 1.2 File Naming Conventions + +* **Python files:** Use lowercase with underscores (e.g., `player.py`, `game_state.py`). +* **Image files:** Use descriptive names (e.g., `player_idle.png`, `enemy_attack.png`). +* **Sound files:** Use descriptive names (e.g., `explosion.wav`, `jump.ogg`). +* **Font files:** Use the font name (e.g., `arial.ttf`). + +### 1.3 Module Organization + +* **Modular design:** Break down your game into logical modules (e.g., `game`, `player`, `enemy`, `ui`). +* **Clear dependencies:** Avoid circular dependencies between modules. +* **Single responsibility principle:** Each module should have a clear and well-defined purpose. + +### 1.4 Component Architecture + +For complex games, consider using a component-based architecture. This involves creating reusable components that can be attached to game objects to provide specific functionality. + +Example: + +python +class Component: + def __init__(self, owner): + self.owner = owner + + def update(self, dt): + pass + +class MovementComponent(Component): + def __init__(self, owner, speed): + super().__init__(owner) + self.speed = speed + + def update(self, dt): + # Update movement logic based on input + ... + +class Player(pygame.sprite.Sprite): + def __init__(self, x, y): + super().__init__() + self.image = pygame.Surface([32, 32]) + self.image.fill((255, 0, 0)) + self.rect = self.image.get_rect() + self.rect.x = x + self.rect.y = y + self.movement = MovementComponent(self, 200) + + def update(self, dt): + self.movement.update(dt) + # Other update logic + + +### 1.5 Code Splitting Strategies + +* **Large files:** Split large files into smaller, more manageable modules. +* **Functional grouping:** Group related functions and classes into modules based on their functionality. +* **Layered architecture:** Separate your code into layers (e.g., presentation layer, logic layer, data access layer) to improve maintainability and testability. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns + +* **Model-View-Controller (MVC):** Separate game data (model), UI (view), and input handling (controller). +* **Observer:** Implement event handling and communication between objects. +* **Factory:** Create objects without specifying their concrete classes. +* **Singleton:** Ensure that a class has only one instance. +* **State:** Encapsulate different states of a game object. +* **Command:** Encapsulate actions as objects to allow for undo/redo functionality. + +### 2.2 Recommended Approaches + +* **Game loop:** Use a well-defined game loop for updating game state and rendering. +* **Event handling:** Implement a clear event handling loop to process user input and other events. +* **Surface management:** Use `Surface.convert()` when loading images to optimize rendering speed. Also ensure image types are optimized for the use case (e.g., .png for transparency, .jpg for photos). +* **Sprite groups:** Use sprite groups for managing and rendering multiple sprites. +* **Timers:** Use `pygame.time.Clock()` to control the frame rate and create timers for game events. Use `pygame.time.set_timer()` for creating custom events on a timer. This avoids blocking the main thread. + +### 2.3 Anti-patterns and Code Smells + +* **God classes:** Avoid creating classes that are too large and complex. +* **Spaghetti code:** Avoid writing code that is difficult to understand and maintain. +* **Tight coupling:** Avoid tight dependencies between modules. +* **Duplicated code:** Avoid repeating the same code in multiple places. +* **Premature optimization:** Avoid optimizing code before identifying performance bottlenecks. +* **Hardcoding values:** Avoid hardcoding values in your code; use constants or configuration files instead. + +### 2.4 State Management + +* **State machines:** Use state machines to manage different game states (e.g., menu, playing, paused). +* **Centralized game state:** Store game state in a central location to make it accessible to all parts of the game. +* **Avoid global variables:** Minimize the use of global variables to reduce the risk of naming conflicts and improve code maintainability. + +### 2.5 Error Handling + +* **Try-except blocks:** Use `try-except` blocks to handle exceptions gracefully. +* **Logging:** Use the `logging` module to log errors and other important events. +* **Custom exceptions:** Define custom exceptions for specific error conditions in your game. +* **Avoid bare except clauses:** Always specify the exception type you are catching to avoid masking unexpected errors. +* **Reraise exceptions when appropriate:** If you catch an exception but cannot handle it, reraise it to allow a higher-level handler to deal with it. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +* **Surface.convert():** As mentioned, use `Surface.convert()` to optimize blitting speed. +* **Rect collision detection:** Use `Rect.colliderect()` for fast collision detection. +* **Sprite groups:** Utilize sprite groups for efficient rendering and collision detection. +* **Profile your code:** Use profiling tools to identify performance bottlenecks. +* **Optimize loops:** Minimize the number of iterations in your game loop. +* **Limit surface creation:** Avoid creating surfaces in your game loop; create them once and reuse them. +* **Pre-calculate values:** Pre-calculate values that are used frequently in your game loop. +* **Use vector math:** `pygame.math.Vector2` is significantly faster than manual tuple based calculations. +* **Use hardware acceleration:** Ensure that hardware acceleration is enabled by using the appropriate display flags (e.g., `pygame.HWSURFACE`, `pygame.DOUBLEBUF`). Note `HWSURFACE` is deprecated in Pygame 2.0.0+ + +### 3.2 Memory Management + +* **Release surfaces:** Release surfaces that are no longer needed by setting them to `None`. +* **Avoid memory leaks:** Be careful not to create memory leaks by holding references to objects that are no longer needed. +* **Use generators:** Use generators to process large datasets without loading them entirely into memory. + +### 3.3 Rendering Optimization + +* **Blitting:** Use `Surface.blit()` for fast image rendering. +* **Dirty rects:** Use dirty rects (updating only changed portions of the screen) to improve rendering performance when the screen is not fully updating every frame. (Less important on modern hardware) +* **Minimize draw calls:** Reduce the number of draw calls by batching them together. +* **Use layers:** Render different parts of the game on different layers to improve performance. +* **Optimize image sizes:** Use appropriately sized images to reduce memory usage and improve rendering speed. +* **Avoid transparency when not needed:** Surfaces without transparency can blit faster. Use `.convert()` or `.convert_alpha()` as needed. + +### 3.4 Bundle Size Optimization + +* **Compress assets:** Use compression techniques to reduce the size of your game assets (images, sounds, fonts). +* **Remove unused assets:** Remove any assets that are not used in your game. +* **Optimize image formats:** Choose the most efficient image format for each asset (e.g., PNG, JPG). +* **Use appropriate audio formats:** Use compressed audio formats like OGG or MP3. + +### 3.5 Lazy Loading + +* **Load assets on demand:** Load assets only when they are needed, instead of loading them all at the beginning of the game. +* **Use loading screens:** Display loading screens while assets are being loaded to provide feedback to the user. +* **Asynchronous loading:** Load assets in the background to avoid blocking the main thread. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities + +* **Code injection:** Prevent code injection by validating user input and avoiding the use of `eval()` or `exec()`. +* **Data tampering:** Protect game data from tampering by using checksums or encryption. +* **Denial of service (DoS):** Protect your game server from DoS attacks by implementing rate limiting and input validation. +* **Save game exploits:** Validate save game data, using encryption and/or checksums, to prevent save game exploits. + +### 4.2 Input Validation + +* **Validate all user input:** Validate all user input to prevent code injection, data tampering, and other security vulnerabilities. +* **Use whitelists:** Use whitelists to specify the allowed characters and values for user input. +* **Sanitize user input:** Sanitize user input to remove potentially harmful characters and code. + +### 4.3 Authentication and Authorization + +* **User authentication:** Implement user authentication to verify the identity of players. +* **Role-based access control:** Use role-based access control to restrict access to certain features and resources based on the player's role. +* **Secure communication:** Use HTTPS to encrypt communication between the game client and server. + +### 4.4 Data Protection + +* **Encrypt sensitive data:** Encrypt sensitive data, such as passwords and credit card numbers. +* **Store data securely:** Store data securely using appropriate storage mechanisms (e.g., databases, key-value stores). +* **Protect against data breaches:** Implement measures to protect against data breaches, such as regular backups and security audits. + +### 4.5 Secure API Communication + +* **Use HTTPS:** Always use HTTPS for API communication to encrypt data in transit. +* **Authenticate requests:** Authenticate all API requests to prevent unauthorized access. +* **Authorize requests:** Authorize all API requests to ensure that users have the necessary permissions to access the requested resources. +* **Validate responses:** Validate API responses to prevent data injection and other security vulnerabilities. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +* **Test individual components:** Write unit tests for individual components to ensure that they function correctly. +* **Use a testing framework:** Use a testing framework like `unittest` or `pytest` to write and run unit tests. +* **Mock dependencies:** Use mocking to isolate components from their dependencies. +* **Test edge cases:** Test edge cases to ensure that your code handles unexpected input and conditions gracefully. +* **Aim for high test coverage:** Aim for high test coverage to ensure that most of your code is tested. + +### 5.2 Integration Testing + +* **Test interactions between components:** Write integration tests to ensure that components work together correctly. +* **Test the game loop:** Test the game loop to ensure that it updates the game state and renders the screen correctly. +* **Test user input:** Test user input to ensure that it is handled correctly. + +### 5.3 End-to-End Testing + +* **Test the entire game:** Write end-to-end tests to test the entire game from start to finish. +* **Automate testing:** Automate testing using tools like Selenium or Appium. +* **Test on multiple platforms:** Test your game on multiple platforms to ensure that it works correctly on all of them. + +### 5.4 Test Organization + +* **Separate test files:** Store test files in a separate directory from your source code. +* **Use descriptive test names:** Use descriptive test names to make it clear what each test is testing. +* **Organize tests by module:** Organize tests by module to make it easier to find and run tests. + +### 5.5 Mocking and Stubbing + +* **Mock dependencies:** Use mocking to replace dependencies with mock objects that can be controlled and inspected during testing. +* **Stub out complex logic:** Use stubbing to replace complex logic with simpler implementations that return predefined values. +* **Use a mocking framework:** Use a mocking framework like `unittest.mock` or `pytest-mock` to create mock objects and stubs. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +* **Forgetting to call `pygame.init()`:** This is a common mistake that can lead to unexpected errors. +* **Not handling events:** Not handling events can cause the game to freeze or crash. +* **Not updating the display:** Not updating the display will cause the screen to remain blank. +* **Using the wrong color format:** Using the wrong color format can lead to incorrect colors being displayed. +* **Not releasing surfaces:** Not releasing surfaces can lead to memory leaks. +* **Inadequate collision detection:** Basic rectangle based collision, while fast, can lead to visually incorrect collisions. Consider using circular collision or per pixel collision in these cases. + +### 6.2 Edge Cases + +* **Window resizing:** Handle window resizing correctly to ensure that the game continues to work as expected. +* **Fullscreen mode:** Test your game in fullscreen mode to ensure that it works correctly. +* **Multiple monitors:** Test your game on systems with multiple monitors to ensure that it works correctly. +* **Different screen resolutions:** Test your game on different screen resolutions to ensure that it scales correctly. + +### 6.3 Version-Specific Issues + +* **Pygame 1 vs Pygame 2:** Be aware of compatibility issues between Pygame 1 and Pygame 2. Some functions and classes have been deprecated or renamed in Pygame 2. + +### 6.4 Compatibility Concerns + +* **Operating systems:** Test your game on different operating systems (Windows, macOS, Linux) to ensure that it works correctly on all of them. +* **Graphics drivers:** Be aware of potential compatibility issues with different graphics drivers. +* **Python versions:** Test your game with different Python versions to ensure compatibility. + +### 6.5 Debugging Strategies + +* **Use print statements:** Use print statements to debug your code and track the values of variables. +* **Use a debugger:** Use a debugger to step through your code and inspect the state of your program. +* **Read error messages:** Pay attention to error messages and use them to identify the cause of the error. +* **Search for solutions:** Search online for solutions to common problems. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +* **Text editor/IDE:** Use a text editor or IDE with Python support (e.g., VS Code, PyCharm). +* **Version control system:** Use a version control system like Git to track changes to your code. +* **Package manager:** Use a package manager like pip to manage dependencies. +* **Debugging tools:** Use debugging tools to step through your code and inspect the state of your program. +* **Profiling tools:** Use profiling tools to identify performance bottlenecks. + +### 7.2 Build Configuration + +* **Use a build system:** Use a build system like Make or CMake to automate the build process. +* **Specify dependencies:** Specify dependencies in a `requirements.txt` file. +* **Use virtual environments:** Use virtual environments to isolate dependencies for each project. + +### 7.3 Linting and Formatting + +* **Use a linter:** Use a linter like pylint or flake8 to identify potential errors and style issues. +* **Use a formatter:** Use a formatter like black or autopep8 to automatically format your code. +* **Follow PEP 8:** Follow the PEP 8 style guide for Python code. + +### 7.4 Deployment + +* **Use a packaging tool:** Use a packaging tool like PyInstaller or cx_Freeze to create standalone executables of your game. +* **Create installers:** Create installers for different platforms to make it easy for users to install your game. +* **Consider using a game distribution platform:** Publish your game on a game distribution platform like Steam or Itch.io. + +### 7.5 CI/CD Integration + +* **Use a CI/CD system:** Use a CI/CD system like Jenkins or GitHub Actions to automate the build, test, and deployment process. +* **Run tests automatically:** Run tests automatically on every commit to ensure that the code is always working correctly. +* **Deploy automatically:** Deploy your game automatically to staging or production environments. \ No newline at end of file diff --git a/.cursor/rules/pylint.mdc b/.cursor/rules/pylint.mdc new file mode 100644 index 000000000..e38537fac --- /dev/null +++ b/.cursor/rules/pylint.mdc @@ -0,0 +1,324 @@ +--- +description: This rule file provides comprehensive best practices for using Pylint to ensure high-quality, maintainable, and secure Python code. It covers code organization, common patterns, performance, security, testing, and tooling. +globs: *.py +--- +# Pylint Best Practices: A Comprehensive Guide + +This document outlines the best practices for using Pylint, a widely used static code analysis tool for Python. Following these guidelines will help you write cleaner, more maintainable, and secure code. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure Best Practices + +* **Project Root:** Contains core project files (`README.md`, `LICENSE`, `.gitignore`, `setup.py` or `pyproject.toml`, `.pylintrc`). +* **Source Directory (`src/` or project_name/`):** Holds the main application code. + * Organize code into modules and subpackages based on functionality (e.g., `src/models`, `src/views`, `src/controllers`, `src/utils`). + * Use clear and descriptive names for modules and packages. +* **Tests Directory (`tests/`):** Contains unit, integration, and end-to-end tests. + * Mirror the source directory structure for easier navigation (e.g., `tests/models`, `tests/views`). +* **Docs Directory (`docs/`):** Stores project documentation (using Sphinx, MkDocs, etc.). +* **Scripts Directory (`scripts/`):** Contains utility scripts for development, deployment, etc. +* **Configuration Directory (`config/`):** Stores configuration files for different environments (development, staging, production). + +Example: + + +my_project/ +├── .gitignore +├── .pylintrc +├── pyproject.toml +├── README.md +├── src/ +│ ├── __init__.py +│ ├── models/ +│ │ ├── __init__.py +│ │ ├── user.py +│ │ └── item.py +│ ├── views/ +│ │ ├── __init__.py +│ │ ├── user_view.py +│ │ └── item_view.py +│ └── utils/ +│ ├── __init__.py +│ └── helper_functions.py +├── tests/ +│ ├── __init__.py +│ ├── models/ +│ │ ├── test_user.py +│ │ └── test_item.py +│ └── views/ +│ ├── __init__.py +│ └── test_user_view.py +└── docs/ + └── ... + + +### 1.2 File Naming Conventions + +* Use lowercase with underscores for module and package names (e.g., `user_model.py`, `data_processing/`). +* Class names should use CamelCase (e.g., `UserModel`). +* Constants should be in uppercase with underscores (e.g., `MAX_USERS`). +* Test files should be named `test_*.py`. + +### 1.3 Module Organization + +* Each module should have a single, well-defined responsibility. +* Avoid circular dependencies between modules. +* Use `__init__.py` files to define packages and control imports. +* Keep modules relatively small (e.g., less than 500 lines of code). +* Follow the "Explicit is better than implicit" principle. Use explicit imports rather than `from module import *`. + +### 1.4 Component Architecture + +* **Layered Architecture:** Separate concerns into distinct layers (e.g., presentation, business logic, data access). +* **Microservices Architecture:** Decompose the application into small, independent services. +* **Hexagonal Architecture (Ports and Adapters):** Decouple the core logic from external dependencies. +* **MVC (Model-View-Controller):** A common pattern, separating data (Model), presentation (View), and user input handling (Controller). + +Choose an architecture that suits the complexity and scale of your project. Ensure that pylint can analyze each component effectively by providing clear interfaces and minimal inter-component dependencies. + +### 1.5 Code Splitting Strategies + +* **Functional Decomposition:** Split code into functions based on specific tasks. +* **Class-Based Decomposition:** Group related functions and data into classes. +* **Module-Based Decomposition:** Organize classes and functions into modules based on their domain. +* **Package-Based Decomposition:** Group related modules into packages. +* **Vertical Slicing**: Create independent, deployable features from end-to-end (UI to database) + +Apply these strategies iteratively as your codebase grows. Keep pylint's static analysis capabilities in mind; avoid excessively complex functions or classes that can become difficult for pylint to analyze effectively. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns + +* **Factory Pattern:** Use factory functions or classes to create objects, promoting loose coupling and testability. +* **Singleton Pattern:** Ensure that a class has only one instance, providing a global point of access. +* **Observer Pattern:** Define a one-to-many dependency between objects, so that when one object changes state, all its dependents are notified and updated automatically. +* **Strategy Pattern:** Define a family of algorithms, encapsulate each one, and make them interchangeable. Strategy lets the algorithm vary independently from clients that use it. +* **Dependency Injection:** Provide dependencies to a component rather than having it create them internally, increasing flexibility and testability. + +### 2.2 Recommended Approaches for Common Tasks + +* **Configuration Management:** Use libraries like `ConfigParser`, `Dynaconf` or `pydantic-settings` to manage application configuration. +* **Logging:** Use the `logging` module for structured logging. +* **Data Validation:** Use libraries like `Pydantic` or `Marshmallow` to validate data. +* **API Requests:** Use the `requests` library for making HTTP requests. +* **Date and Time Handling:** Use the `datetime` module for date and time operations. +* **Use Context Managers (`with` statements) whenever working with external resources like files, network connections, etc. to ensure proper cleanup. + +### 2.3 Anti-patterns and Code Smells + +* **Long Methods/Functions:** Functions or methods exceeding a reasonable length (e.g., 50 lines) are hard to understand and maintain. Split them into smaller, more focused units. +* **Duplicate Code:** Avoid copy-pasting code. Extract common logic into reusable functions or classes. +* **Magic Numbers:** Use named constants instead of hardcoded values. +* **Feature Envy:** A method accesses the data of another object more than its own. Move the method to the object it uses the most. +* **God Class/Object:** A class that knows or does too much. Split it into smaller, more specialized classes. +* **Spaghetti Code:** Code with a complex and tangled control structure, making it difficult to follow the logic. Refactor it into a more structured design. +* **Shotgun Surgery:** Whenever you make one kind of change, you have to make many small changes to a lot of different classes. Consolidate changes into fewer classes. +* **Primitive Obsession:** Using primitive data types to represent domain concepts. Create dedicated classes instead. +* **Data Clumps:** Groups of variables that appear together in many places. Create a class to encapsulate them. + +### 2.4 State Management + +* **Minimize Mutable State:** Favor immutable data structures to reduce complexity and potential errors. +* **Centralized State Management:** Use patterns like Redux or Context API (if applicable) to manage application state in a central location. +* **Clear State Transitions:** Define clear and predictable state transitions to avoid unexpected behavior. +* **Avoid Global State:** Global state can make code harder to reason about and test. Use dependency injection or other techniques to manage dependencies. + +### 2.5 Error Handling + +* **Specific Exception Handling:** Catch specific exceptions rather than generic `Exception` to handle errors appropriately. +* **Use `try...except...finally`:** Ensure that resources are released properly, even if an exception occurs. +* **Avoid Bare `except` Clauses:** Always specify the exception type you are catching. +* **Logging Exceptions:** Log detailed information about exceptions for debugging purposes. +* **Raise Exceptions Responsibly:** Raise exceptions when an error occurs that cannot be handled locally. Provide meaningful error messages. +* **Use Custom Exceptions:** Create custom exception classes for specific error scenarios in your application, improving clarity and maintainability. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +* **Profiling:** Use profiling tools like `cProfile` to identify performance bottlenecks. +* **Algorithmic Optimization:** Choose efficient algorithms and data structures for critical tasks. +* **Caching:** Use caching techniques to store frequently accessed data and reduce computation time (e.g., `functools.lru_cache`). +* **List Comprehensions and Generators:** Use list comprehensions and generators for concise and efficient code. +* **Avoid Premature Optimization:** Optimize code only after identifying actual performance bottlenecks. +* **Use vectorized operations with NumPy when dealing with numerical computations to leverage performance. + +### 3.2 Memory Management + +* **Avoid Memory Leaks:** Ensure that objects are properly deallocated when they are no longer needed. +* **Use Generators:** Use generators to process large datasets in chunks, reducing memory usage. +* **Minimize Object Creation:** Avoid creating unnecessary objects to reduce memory overhead. +* **Use Data Structures Wisely:** Choose appropriate data structures based on memory usage and performance characteristics. +* **Use the `del` keyword when you are completely finished with a variable that is holding a large data structure, allowing the garbage collector to reclaim the memory. + +### 3.3 Rendering Optimization (If Applicable) + +* This is more relevant to UI applications or cases where Pylint is integrated into a tool with a visual output. +* **Virtualization/Windowing**: Only render what is visible on the screen, especially for long lists or tables. +* **Debouncing/Throttling**: Limit the frequency of updates to the display. + +### 3.4 Bundle Size Optimization (If Applicable) + +* This mainly applies if you are building a web application or a desktop app where Pylint analysis is integrated as part of the build process. Minimize dependencies, tree shake (remove unused code), and compress the output. + +### 3.5 Lazy Loading + +* **Lazy Loading Modules:** Use lazy loading to load modules only when they are needed. +* **Lazy Loading Data:** Fetch data only when it is required by the user or the application. +* **Avoid Eager Loading:** Avoid loading large datasets or resources upfront. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and Prevention + +* **SQL Injection:** Use parameterized queries or an ORM to prevent SQL injection attacks. +* **Cross-Site Scripting (XSS):** Sanitize user input to prevent XSS attacks. +* **Cross-Site Request Forgery (CSRF):** Use CSRF tokens to protect against CSRF attacks. +* **Command Injection:** Avoid executing arbitrary commands based on user input. +* **Path Traversal:** Validate file paths to prevent path traversal attacks. +* **Regularly update your dependencies to patch security vulnerabilities. + +### 4.2 Input Validation + +* **Validate All User Input:** Validate all user input to ensure that it is valid and safe. +* **Use Regular Expressions:** Use regular expressions to validate input patterns. +* **Sanitize Input:** Sanitize input to remove or escape potentially harmful characters. +* **Limit Input Length:** Limit the length of input fields to prevent buffer overflows. + +### 4.3 Authentication and Authorization + +* **Use Strong Passwords:** Enforce strong password policies and use password hashing algorithms (e.g., bcrypt, Argon2). +* **Implement Role-Based Access Control (RBAC):** Define roles and permissions to control access to resources. +* **Use Multi-Factor Authentication (MFA):** Add an extra layer of security with MFA. +* **Secure Session Management:** Use secure session management techniques to protect user sessions. + +### 4.4 Data Protection + +* **Encrypt Sensitive Data:** Encrypt sensitive data at rest and in transit. +* **Use Secure Protocols:** Use secure protocols like HTTPS for communication. +* **Data Masking:** Mask sensitive data when it is not needed. +* **Data Anonymization:** Anonymize data to protect user privacy. + +### 4.5 Secure API Communication + +* **Use Authentication Tokens:** Use authentication tokens to authenticate API requests (e.g., JWT). +* **Rate Limiting:** Implement rate limiting to prevent abuse. +* **Input Validation:** Validate all input to API endpoints. +* **Output Sanitization:** Sanitize output from API endpoints to prevent injection attacks. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +* **Test Individual Components:** Write unit tests for individual functions, classes, and modules. +* **Use Assertions:** Use assertions to verify that the code behaves as expected. +* **Test Edge Cases:** Test edge cases and boundary conditions. +* **Test Exception Handling:** Test that exceptions are raised and handled correctly. +* **Isolate Tests:** Ensure that unit tests are isolated and do not depend on external resources. + +### 5.2 Integration Testing + +* **Test Interactions:** Write integration tests to verify that different components work together correctly. +* **Test API Endpoints:** Test API endpoints to ensure that they are functioning as expected. +* **Test Database Interactions:** Test database interactions to ensure that data is being stored and retrieved correctly. + +### 5.3 End-to-End Testing + +* **Simulate User Flows:** Write end-to-end tests to simulate user flows and verify that the application behaves as expected from a user's perspective. +* **Test UI Interactions:** Test UI interactions to ensure that the UI is functioning correctly. +* **Test System Behavior:** Test overall system behavior and performance. + +### 5.4 Test Organization + +* **Mirror Source Directory:** Organize tests in a directory structure that mirrors the source directory structure. +* **Use Descriptive Names:** Use descriptive names for test files and test functions. +* **Separate Test Files:** Separate unit tests, integration tests, and end-to-end tests into separate files. +* **Use a Test Runner:** Use a test runner like `pytest` or `unittest` to run tests. + +### 5.5 Mocking and Stubbing + +* **Use Mocks:** Use mocks to replace external dependencies with controlled test doubles. +* **Use Stubs:** Use stubs to provide predefined responses for external dependencies. +* **Isolate Units Under Test:** Use mocking and stubbing to isolate the unit under test and prevent dependencies from affecting test results. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +* **Ignoring Pylint Warnings:** Ignoring Pylint warnings can lead to code quality issues and potential bugs. Always address Pylint warnings promptly. +* **Over-Suppressing Messages:** Over-suppressing Pylint messages can hide important issues. Only suppress messages when you are certain that they are not relevant. +* **Not Customizing Pylint Configuration:** Not customizing the Pylint configuration to match your project's needs can result in irrelevant warnings and missed issues. Customize the configuration to suit your project. +* **Using Broad Exceptions:** Using broad exception clauses (e.g., `except Exception:`) can catch unintended exceptions and hide potential problems. +* **Not Testing Edge Cases:** Not testing edge cases can lead to unexpected behavior and bugs. +* **Not Using Virtual Environments:** Failing to use virtual environments can lead to dependency conflicts and deployment issues. + +### 6.2 Edge Cases + +* **Dynamic Code Generation:** Pylint may have difficulty analyzing code that is dynamically generated at runtime. +* **Complex Metaclasses:** Complex metaclasses can confuse Pylint and lead to false positives. +* **Third-Party Libraries:** Pylint may not fully support all third-party libraries. +* **Cython Extensions:** Cython extensions may not be analyzed correctly by Pylint. +* **Monkey Patching**: Dynamic modification of code at runtime can confuse pylint and potentially lead to incorrect analysis. + +### 6.3 Version-Specific Issues + +* **Compatibility Issues:** Different versions of Pylint may have compatibility issues with different versions of Python and third-party libraries. +* **Message Changes:** Pylint messages may change between versions, requiring updates to your configuration. +* **Checker Updates:** New checkers may be added in new versions of Pylint, requiring updates to your codebase. + +### 6.4 Compatibility Concerns + +* **Black/Autopep8:** Ensure that Pylint's formatting rules are compatible with formatters like Black or Autopep8 to avoid conflicts. +* **Mypy:** Use Mypy to complement Pylint's static analysis with type checking. +* **Flake8:** Consider using Flake8 alongside Pylint for additional linting checks. + +### 6.5 Debugging Strategies + +* **Read Error Messages Carefully:** Pylint error messages provide valuable information about the cause of the error. +* **Use `--verbose`:** Use the `--verbose` flag to get more detailed output from Pylint. +* **Isolate the Problem:** Try to isolate the problem to a specific function, class, or module. +* **Use a Debugger:** Use a debugger to step through the code and identify the cause of the error. +* **Consult the Pylint Documentation:** The Pylint documentation provides detailed information about all of Pylint's features and messages. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +* **VS Code with Python Extension:** VS Code is a popular code editor with excellent Python support. +* **PyCharm:** PyCharm is a powerful IDE for Python development. +* **Pylance (VS Code extension):** A fast, feature-rich language server for Python in VS Code. +* **Docker:** Use Docker for consistent and reproducible development environments. + +### 7.2 Build Configuration + +* **Use a Build System:** Use a build system like `Make`, `Poetry`, or `tox` to automate build tasks. +* **Define Dependencies:** Define dependencies in a `requirements.txt` or `pyproject.toml` file. +* **Use Virtual Environments:** Use virtual environments to isolate project dependencies. +* **Automate Pylint Execution:** Automate Pylint execution as part of the build process. +* **Include .pylintrc:** Ensure that the `.pylintrc` file is included in the project repository. + +### 7.3 Linting and Formatting + +* **Use a Code Formatter:** Use a code formatter like `Black` or `Autopep8` to automatically format code. +* **Integrate with Editor:** Integrate Pylint and the code formatter with your code editor for real-time feedback. +* **Use Pre-Commit Hooks:** Use pre-commit hooks to run Pylint and the code formatter before committing changes. +* **Address all linting and formatting issues consistently. + +### 7.4 Deployment + +* **Use a Deployment Pipeline:** Use a deployment pipeline to automate the deployment process. +* **Test Before Deployment:** Run unit tests, integration tests, and end-to-end tests before deploying the application. +* **Monitor Application:** Monitor the application after deployment to detect and resolve issues. +* **Use a Configuration Management Tool:** Use a configuration management tool like Ansible or Chef to manage application configuration in production. + +### 7.5 CI/CD Integration + +* **Integrate with CI/CD System:** Integrate Pylint with your CI/CD system (e.g., Jenkins, GitLab CI, GitHub Actions). +* **Run Pylint in CI/CD Pipeline:** Run Pylint as part of the CI/CD pipeline to automatically check code quality. +* **Fail Build on Errors:** Configure the CI/CD pipeline to fail the build if Pylint reports any errors. +* **Collect Metrics:** Collect metrics from Pylint runs to track code quality over time. +* **Use code review processes and tools to enforce coding standards. + +By adhering to these best practices, you can leverage Pylint to create high-quality, maintainable, and secure Python code. \ No newline at end of file diff --git a/.cursor/rules/pyqt.mdc b/.cursor/rules/pyqt.mdc new file mode 100644 index 000000000..f0c125e59 --- /dev/null +++ b/.cursor/rules/pyqt.mdc @@ -0,0 +1,120 @@ +--- +description: This rule file provides comprehensive guidelines for PyQt development, covering code organization, common patterns, performance, security, testing, and tooling. It aims to help developers create maintainable, efficient, and secure PyQt applications. +globs: *.py +--- +- **Follow Consistent Importing:** + - Avoid wildcard imports like `from PyQt5.QtGui import *`. They clutter the namespace and can lead to naming conflicts. + - Instead, import only the necessary modules or classes, e.g., `from PyQt5 import QtGui` and access members with the module prefix: `QtGui.QPushButton`. + - *Rationale:* Improves code readability and avoids potential namespace collisions. + +- **Adhere to Naming Conventions:** + - Follow Qt's camelCase style for method names (e.g., `loadItems()`, `refreshResults()`). This maintains consistency with the Qt API and reduces confusion. + - For signals and slots, consistently use camelCase. + - *Rationale:* Ensures consistency with Qt's API and reduces cognitive load for developers. + +- **Separate GUI and Business Logic (MVC or similar):** + - Keep GUI code separate from business logic using patterns like Model-View-Controller (MVC), Model-View-Presenter (MVP), or Model-View-ViewModel (MVVM). + - *Model:* Represents the data and business logic. + - *View:* Displays the data to the user. + - *Controller/Presenter/ViewModel:* Handles user input and updates the Model and View. + - *Rationale:* Enhances maintainability, testability, and scalability by decoupling concerns. + +- **Leverage Qt Designer (or Alternatives):** + - Use Qt Designer (or alternative tools like QML) for layout management and rapid prototyping. + - Load the generated `.ui` files into your Python code using `PyQt6.uic.loadUi`. + - *Rationale:* Speeds up GUI development and simplifies layout design. + +- **Employ Signals and Slots:** + - Use Qt's signals and slots mechanism for communication between objects. + - Connect signals to slots using `QObject.connect()` or the newer, more Pythonic syntax in PyQt6. + - *Rationale:* Provides a type-safe and flexible way to handle events and user interactions. + +- **Protect UI Members:** + - Treat UI components defined in Qt Designer as protected or private members. + - Provide access to child widgets of your class through methods instead of direct access. + - *Rationale:* Facilitates future modifications to the UI without breaking external code. + +- **Naming Conventions for Designer Components:** + - Adopt a consistent naming convention for UI components created in Qt Designer. + - Use prefixes like `ui_` to indicate that a component was created in Designer (e.g., `ui_project_combo`, `uiProjectCombo`). Append the component type. + - *Rationale:* Improves code readability and makes it easier to identify the origin and purpose of UI components. + +- **Getter and Setter Guidelines:** + - Use getter and setter methods to access and modify internal data instead of directly exposing public members. Use camelCase for both the getter and setter methods. + - Create protected members using a single leading underscore (e.g., `_value`). + - *Rationale:* Provides encapsulation, allows for future changes to the internal implementation, and maintains consistency with Qt's style. + +- **Never Expose Public Members in Qt Classes:** + - In Python, there isn't a strict public/protected/private syntax, but use underscores to flag different types of data. + - Python uses underscores to flag different types of data. + - `No underscores`: public variable/member + - `1 opening underscore`: protected variable/member + - `1 trailing underscore`: used when defining a variable that would otherwise clash with a Python internal + - `2 opening underscores`: private variable/member (does some funkiness under the hood so it becomes pretty hard to access from outside your class or module) + - `2 opening underscores and 2 trailing underscores`: built-in, usually used by internal python variables and methods + - When developing new Qt objects, widgets, or classes you should create protected members, and allow access to them via getter and setter methods. + +- **Code Organization and Structure:** + - *Directory Structure:* Organize your project into logical directories (e.g., `ui`, `models`, `controllers`, `widgets`). + - *File Naming:* Use descriptive file names (e.g., `mainwindow.py`, `user_model.py`). + - *Module Organization:* Break your code into smaller, reusable modules. + - *Component Architecture:* Design your application using a component-based architecture for better modularity. + +- **Common Patterns and Anti-Patterns:** + - *Design Patterns:* Apply design patterns like Observer, Singleton, Factory, and Strategy where appropriate. + - *Anti-Patterns:* Avoid God classes, tight coupling, and code duplication. + - *State Management:* Use signals and slots or dedicated state management libraries for managing application state. + - *Error Handling:* Implement robust error handling using try-except blocks and logging. + +- **Performance Considerations:** + - *Optimization:* Optimize your code by minimizing expensive operations, using caching, and avoiding unnecessary GUI updates. + - *Memory Management:* Be mindful of memory usage, especially when dealing with large images or data sets. Use `del` to remove unwanted objects from memory. + - *Rendering Optimization:* Optimize rendering by using double buffering, avoiding unnecessary repaints, and using optimized drawing methods. + - Deleting objects that are no longer in use. + - Limit the number of widgets. + +- **Security Best Practices:** + - *Vulnerabilities:* Be aware of common vulnerabilities like code injection, cross-site scripting (XSS), and SQL injection if using databases. + - *Input Validation:* Validate all user inputs to prevent malicious data from entering your application. + - *Authentication:* Implement proper authentication and authorization mechanisms to protect sensitive data. + - *Data Protection:* Encrypt sensitive data and use secure communication protocols. + - When fetching external data be mindful about malicious data. + +- **Testing Approaches:** + - *Unit Tests:* Write unit tests for individual components and functions. + - *Integration Tests:* Test the interaction between different components. + - *End-to-End Tests:* Test the entire application flow. + - *Test Organization:* Organize your tests into logical directories and use meaningful names. + - *Mocking:* Use mocking and stubbing to isolate components during testing. + - Using `pytest` to generate tests. + +- **Common Pitfalls and Gotchas:** + - *Mistakes:* Avoid circular dependencies, memory leaks, and neglecting event handling. + - *Edge Cases:* Be aware of edge cases like unexpected user inputs, network errors, and file system issues. + - *Version Issues:* Pay attention to version-specific issues and compatibility concerns when upgrading PyQt or other dependencies. + - Ensure the GUI thread is kept responsive. + +- **Tooling and Environment:** + - *Development Tools:* Use IDEs like VS Code with Python extensions, PyCharm, or Qt Creator. + - *Build Configuration:* Use build tools like CMake or setuptools for managing your project build process. + - *Linting:* Lint your code using tools like Pylint or Flake8. + - *Formatting:* Format your code using tools like Black or autopep8. + - *Deployment:* Use tools like PyInstaller or cx_Freeze for creating standalone executables. + - *CI/CD:* Integrate your project with CI/CD pipelines for automated testing and deployment. + +- **Specific Recommendations:** + - Always make sure that long running process happen in the background. + - Utilize `QThread` to move computationally heavy process off the main thread. + +- **Code Splitting:** + - Break large classes and functions into smaller, more manageable units. + - Extract reusable code into separate modules or libraries. + +- **State Management Best Practices:** + - Use signals and slots effectively to propagate state changes throughout the application. + - Consider using a dedicated state management library or pattern if your application has complex state requirements. + +- **Additional Considerations:** + - If you need to handle asynchronous operations, leverage `QFuture` and `QThreadPool` for efficient execution. + - Use `QSettings` to store application settings and preferences. + - Prioritize accessibility by ensuring your application is usable by people with disabilities (e.g., provide keyboard navigation, use appropriate color contrast). \ No newline at end of file diff --git a/.cursor/rules/pyramid.mdc b/.cursor/rules/pyramid.mdc new file mode 100644 index 000000000..b38ed3298 --- /dev/null +++ b/.cursor/rules/pyramid.mdc @@ -0,0 +1,176 @@ +--- +description: This rule provides comprehensive best practices for developing secure, maintainable, and performant applications using the Pyramid web framework for Python. It covers code structure, security, testing, and deployment considerations. +globs: *.py +--- +- **Project Structure and Code Organization:** + - **Directory Structure:** + - `src/`: Contains the main application code. + - `src//`: Main application package. + - `__init__.py`: Makes the directory a Python package. + - `models.py`: Data models (if using an ORM like SQLAlchemy). + - `views.py`: View callables (controllers). + - `routes.py`: Route configurations. + - `templates/`: Jinja2 or other template files. + - `static/`: Static assets (CSS, JavaScript, images). + - `scripts/`: Management scripts (e.g., database setup, data migration). + - `tests/`: Unit and integration tests. + - `__init__.py`: Makes the directory a Python package. + - `conftest.py`: pytest configuration file. + - `test_*.py`: Test modules. + - `docs/`: Project documentation (Sphinx). + - `venv/`: Virtual environment (should be excluded from version control). + - **File Naming Conventions:** + - Python files: `snake_case.py` (e.g., `database_utils.py`). + - Class names: `CamelCase` (e.g., `User`). + - Variable names: `snake_case` (e.g., `user_id`). + - Constants: `UPPER_SNAKE_CASE` (e.g., `DEFAULT_TIMEOUT`). + - **Module Organization:** + - Group related functionality into modules. + - Avoid circular dependencies. + - Use relative imports within the application package (e.g., `from .models import User`). + - **Component Architecture:** + - MVC (Model-View-Controller) is a common pattern in Pyramid. + - Models represent data and business logic. + - Views handle requests and render responses. + - Controllers (view callables) act as intermediaries between models and views. + - **Code Splitting:** + - Break down large modules into smaller, more manageable files. + - Use packages to group related modules. + - Consider using a service layer to separate business logic from view callables. + +- **Common Patterns and Anti-Patterns:** + - **Design Patterns:** + - **Repository Pattern:** Abstract data access logic behind a repository interface. + - **Service Layer Pattern:** Encapsulate business logic in a separate layer. + - **Factory Pattern:** Use factories to create complex objects. + - **Recommended Approaches:** + - Use URL dispatch for routing requests to view callables. + - Use decorators (`@view_config`) to associate views with routes. + - Use Jinja2 or another templating engine for rendering dynamic content. + - Use SQLAlchemy or another ORM for interacting with databases. + - **Anti-Patterns:** + - **Fat Views:** Avoid putting too much logic in view callables. + - **Tight Coupling:** Design components to be loosely coupled. + - **Ignoring Exceptions:** Always handle exceptions appropriately. + - **State Management:** + - Use request attributes to store request-specific data (e.g., database connection, user session). + - Use sessions to store user-specific data across multiple requests. + - Avoid storing large amounts of data in sessions. + - **Error Handling:** + - Use try-except blocks to handle exceptions. + - Log exceptions with appropriate severity levels (e.g., `error`, `warning`, `info`). + - Provide user-friendly error messages. + - Use Pyramid's exception views to handle exceptions globally. + +- **Performance Considerations:** + - **Optimization Techniques:** + - Use caching to store frequently accessed data (e.g., using `dogpile.cache`). + - Optimize database queries (e.g., using indexes, avoiding N+1 queries). + - Minimize the number of database connections. + - Use efficient data structures and algorithms. + - **Memory Management:** + - Avoid creating unnecessary objects. + - Use generators and iterators to process large datasets. + - Clean up resources properly (e.g., closing database connections). + - **Rendering Optimization:** + - Use template caching. + - Minimize the amount of data passed to templates. + - Optimize template code. + - **Bundle Size Optimization:** (If serving static assets) + - Minify CSS and JavaScript files. + - Use a content delivery network (CDN) to serve static assets. + - **Lazy Loading:** + - Lazy-load images and other resources. + - Defer loading of non-essential JavaScript. + +- **Security Best Practices:** + - **Common Vulnerabilities:** + - **SQL Injection:** Prevent by using parameterized queries or an ORM that escapes input. + - **Cross-Site Scripting (XSS):** Prevent by escaping output in templates. + - **Cross-Site Request Forgery (CSRF):** Use Pyramid's CSRF protection features. + - **Authentication and Authorization Issues:** Implement strong authentication and authorization mechanisms. + - **Session Hijacking:** Use secure cookies and regenerate session IDs after login. + - **Input Validation:** + - Validate all user input. + - Use schema validation libraries (e.g., `colander`) to validate input data. + - Sanitize input to remove potentially malicious characters. + - **Authentication and Authorization:** + - Use a robust authentication system (e.g., `pyramid_jwt`, `pyramid_authsanity`). + - Implement role-based access control (RBAC). + - Secure API endpoints with authentication and authorization checks. + - **Data Protection:** + - Encrypt sensitive data at rest and in transit. + - Use TLS/SSL for secure communication. + - Store passwords securely using strong hashing algorithms (e.g., bcrypt, argon2). + - **Secure API Communication:** + - Use HTTPS for all API requests. + - Authenticate API clients using API keys or tokens. + - Implement rate limiting to prevent abuse. + +- **Testing Approaches:** + - **Unit Testing:** + - Test individual components in isolation. + - Use mocking to isolate components from dependencies. + - Use a testing framework (e.g., `pytest`). + - **Integration Testing:** + - Test the interaction between multiple components. + - Test the application's integration with external systems (e.g., databases). + - **End-to-End Testing:** + - Test the entire application from the user's perspective. + - Use a browser automation tool (e.g., `Selenium`, `Playwright`). + - **Test Organization:** + - Keep tests in a separate `tests/` directory. + - Organize tests into modules that correspond to the application's modules. + - Use clear and descriptive test names. + - **Mocking and Stubbing:** + - Use mocking libraries (e.g., `unittest.mock`, `pytest-mock`) to create mock objects. + - Use stubs to replace dependencies with simplified versions. + +- **Common Pitfalls and Gotchas:** + - **Frequent Mistakes:** + - **Misunderstanding URL Dispatch:** Carefully define routes and view predicates. + - **Insecure Configuration:** Protect sensitive configuration data. + - **Insufficient Testing:** Thoroughly test all parts of the application. + - **Edge Cases:** + - **Handling Unicode:** Ensure proper Unicode handling throughout the application. + - **Dealing with Time Zones:** Use a consistent time zone and handle time zone conversions correctly. + - **Version-Specific Issues:** + - Be aware of compatibility issues when upgrading Pyramid or its dependencies. + - Consult the release notes for any breaking changes. + - **Compatibility Concerns:** + - Ensure compatibility between Pyramid and other technologies used in the application. + - Test the application on all supported platforms and browsers. + - **Debugging Strategies:** + - Use a debugger to step through code and inspect variables. + - Use logging to track the flow of execution and identify errors. + - Use Pyramid's debugging toolbar to inspect requests, responses, and configuration. + +- **Tooling and Environment:** + - **Recommended Tools:** + - **Virtualenv or venv:** For creating isolated Python environments. + - **pip:** For managing dependencies. + - **pytest:** For running tests. + - **flake8:** For linting code. + - **black:** For formatting code. + - **Build Configuration:** + - Use a `setup.py` or `pyproject.toml` file to define the project's metadata and dependencies. + - Use a build system (e.g., `setuptools`, `poetry`) to package the application. + - **Linting and Formatting:** + - Use a linter (e.g., `flake8`) to enforce code style guidelines. + - Use a formatter (e.g., `black`) to automatically format code. + - Configure the linter and formatter to use consistent settings. + - **Deployment:** + - Use a WSGI server (e.g., `gunicorn`, `uWSGI`) to serve the application. + - Use a process manager (e.g., `systemd`, `supervisor`) to manage the WSGI server. + - Deploy the application to a production environment (e.g., cloud platform, virtual machine). + - **CI/CD:** + - Use a continuous integration/continuous deployment (CI/CD) system (e.g., `Jenkins`, `GitLab CI`, `GitHub Actions`) to automate the build, test, and deployment process. + - Run tests automatically on every commit. + - Deploy the application automatically to staging and production environments. + +- **Additional Information:** + - Regularly update dependencies to patch security vulnerabilities. + - Use a security scanner like Bandit to identify potential security flaws in the code. + - Follow the principle of least privilege when granting permissions to users and services. + - Monitor the application's performance and security logs to detect and respond to incidents. + - Document the application's architecture, configuration, and deployment process. \ No newline at end of file diff --git a/.cursor/rules/pyright.mdc b/.cursor/rules/pyright.mdc new file mode 100644 index 000000000..d6dc87a5f --- /dev/null +++ b/.cursor/rules/pyright.mdc @@ -0,0 +1,291 @@ +--- +description: This rule provides comprehensive best practices for using pyright and BasedPyright in Python projects, covering code organization, patterns, performance, security, testing, common pitfalls, and tooling. +globs: *.py +--- +# pyright and BasedPyright Best Practices: A Comprehensive Guide + +This guide provides comprehensive best practices for using pyright and BasedPyright in Python projects. These practices cover code organization, design patterns, performance optimization, security considerations, testing strategies, common pitfalls, and recommended tooling. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure Best Practices + +* **Flat vs. Nested:** Choose a directory structure that balances simplicity and maintainability. For small projects, a flat structure might suffice. For larger projects, a nested structure that mirrors the module hierarchy is often better. For example: + + + my_project/ + ├── src/ + │ ├── my_module/ + │ │ ├── __init__.py + │ │ ├── file1.py + │ │ ├── file2.py + │ ├── main.py + ├── tests/ + │ ├── my_module/ + │ │ ├── test_file1.py + │ │ ├── test_file2.py + │ ├── conftest.py + ├── pyrightconfig.json + ├── pyproject.toml (or setup.py/setup.cfg) + ├── README.md + + +* **`src` Layout:** Use the `src` layout to separate application code from project metadata. This helps prevent accidental imports of test or configuration files. + +### 1.2 File Naming Conventions + +* **Python Naming:** Follow standard Python naming conventions (PEP 8): + * Modules: `lowercase_with_underscores.py` + * Classes: `PascalCase` + * Functions and variables: `lowercase_with_underscores` + * Constants: `UPPERCASE_WITH_UNDERSCORES` +* **Test Files:** Name test files consistently, e.g., `test_.py` or `_test.py`. + +### 1.3 Module Organization + +* **Cohesion:** Group related functions and classes within a single module. +* **Coupling:** Minimize dependencies between modules to improve maintainability. +* **`__init__.py`:** Use `__init__.py` files to define packages and control namespace imports. Consider explicit relative imports within packages (e.g., `from . import module` instead of `import module`). + +### 1.4 Component Architecture + +* **Layered Architecture:** For complex applications, consider a layered architecture (e.g., presentation, business logic, data access). This promotes separation of concerns and testability. +* **Microservices:** For very large projects, consider breaking the application into microservices. + +### 1.5 Code Splitting Strategies + +* **By Feature:** Split code into modules or packages based on features or functionality (e.g., `auth`, `users`, `products`). +* **By Layer:** Split code based on architectural layers (e.g., data access, business logic, presentation). This aligns with Layered architecture. +* **Lazy Loading:** Defer loading modules or components until they are needed. This can improve startup time and reduce memory usage. Use the `importlib` module or dynamic imports for lazy loading. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns + +* **Factory Pattern:** Use factory functions or classes to create objects, especially when the object creation logic is complex or needs to be configurable. +* **Strategy Pattern:** Use the strategy pattern to encapsulate different algorithms or behaviors and switch between them at runtime. This promotes flexibility and testability. +* **Observer Pattern:** Implement the observer pattern for event handling and decoupling components. +* **Singleton Pattern:** Use sparingly and only when a single instance of a class is truly required. Consider dependency injection as an alternative. + +### 2.2 Recommended Approaches + +* **Type Annotations:** Embrace type annotations throughout your codebase. This is essential for effective static analysis with pyright and improves code readability and maintainability. +* **Configuration:** Externalize configuration data (e.g., database connection strings, API keys) using environment variables or configuration files. Use libraries like `python-dotenv` or `dynaconf`. +* **Logging:** Implement comprehensive logging using the `logging` module. Configure logging levels (DEBUG, INFO, WARNING, ERROR, CRITICAL) appropriately. +* **Dependency Injection:** Use dependency injection to decouple components and improve testability. Libraries like `injector` can simplify dependency injection. + +### 2.3 Anti-patterns and Code Smells + +* **Ignoring Pyright Errors:** Treat pyright errors as critical issues that need to be addressed. Do not silence errors without understanding the underlying problem. +* **Overuse of `Any`:** Avoid using `Any` excessively. It defeats the purpose of static typing. Strive to provide specific type annotations. +* **Magic Numbers:** Avoid hardcoding numerical values or strings directly in your code. Use named constants instead. +* **Global State:** Minimize the use of global variables. Global state can make code harder to understand and test. +* **Deeply Nested Code:** Avoid deeply nested conditional statements or loops. Refactor complex code into smaller, more manageable functions. + +### 2.4 State Management + +* **Immutability:** Favor immutable data structures where possible. This can simplify reasoning about state and prevent unintended side effects. Use libraries like `attrs` or `dataclasses` to create immutable classes. +* **Centralized State:** For complex applications, consider using a centralized state management solution (e.g., using Redux-like patterns or libraries). + +### 2.5 Error Handling + +* **Exceptions:** Use exceptions for exceptional situations. Raise specific exception types that accurately describe the error. +* **`try...except`:** Use `try...except` blocks to handle exceptions gracefully. Avoid catching generic `Exception` unless absolutely necessary. +* **Logging Errors:** Log exceptions with sufficient context (e.g., traceback, relevant variable values). This is crucial for debugging. +* **Resource Management:** Use `try...finally` or the `with` statement to ensure resources are properly released, even if an exception occurs. +* **Retry logic:** Implement retry mechanisms for network requests or other operations that may transiently fail. Use libraries like `tenacity`. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +* **Profiling:** Use profiling tools (e.g., `cProfile`) to identify performance bottlenecks in your code. +* **Efficient Data Structures:** Choose appropriate data structures for your tasks. For example, use sets for membership testing, and dictionaries for fast lookups. +* **Algorithm Optimization:** Optimize algorithms to reduce time complexity. Consider using libraries like NumPy for numerical computations. +* **Caching:** Implement caching to store frequently accessed data and avoid redundant computations. Use libraries like `functools.lru_cache` or `cachetools`. +* **Just-In-Time (JIT) Compilation:** Explore using JIT compilers like Numba to accelerate numerical code. + +### 3.2 Memory Management + +* **Generators:** Use generators to process large datasets without loading them into memory all at once. +* **Context Managers:** Use context managers (`with` statement) to ensure that resources are properly released, preventing memory leaks. +* **Object Reuse:** Reuse objects where possible to reduce memory allocation overhead. +* **Avoid Circular References:** Circular references can prevent garbage collection. Use weak references (`weakref` module) to break cycles when needed. + +### 3.3 Bundle Size Optimization (Applicable for web applications using pyright in the backend) + +* **Code Minification:** Minify JavaScript and CSS code to reduce bundle size. +* **Tree Shaking:** Use tree shaking to remove unused code from your bundles. +* **Code Splitting:** Split your code into smaller chunks that can be loaded on demand. +* **Image Optimization:** Optimize images to reduce their file size. +* **Compression:** Enable compression (e.g., gzip or Brotli) on your web server. + +### 3.4 Lazy Loading + +* **Dynamic Imports:** Use dynamic imports (`import()`) to load modules or components on demand. +* **Conditional Imports:** Conditionally import modules based on certain conditions. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and Prevention + +* **SQL Injection:** Use parameterized queries or ORM libraries to prevent SQL injection attacks. +* **Cross-Site Scripting (XSS):** Sanitize user input to prevent XSS attacks. (Relevant if the backend code generates HTML output) +* **Cross-Site Request Forgery (CSRF):** Implement CSRF protection for web forms. +* **Authentication and Authorization Flaws:** Use secure authentication and authorization mechanisms. +* **Denial-of-Service (DoS):** Implement rate limiting and other measures to prevent DoS attacks. +* **Dependency Vulnerabilities:** Regularly scan your dependencies for known vulnerabilities using tools like `pip-audit` or `safety`. + +### 4.2 Input Validation + +* **Validate All Input:** Validate all user input, including data from forms, API requests, and command-line arguments. +* **Whitelisting:** Use whitelisting to specify allowed characters or patterns. +* **Data Type Validation:** Ensure that input data is of the expected type. +* **Length Validation:** Validate the length of input strings. +* **Range Validation:** Validate that numerical values are within the expected range. + +### 4.3 Authentication and Authorization + +* **Strong Passwords:** Enforce strong password policies. +* **Hashing:** Hash passwords securely using libraries like `bcrypt` or `argon2`. +* **Salt:** Use a unique salt for each password. +* **Two-Factor Authentication (2FA):** Implement 2FA for enhanced security. +* **Role-Based Access Control (RBAC):** Implement RBAC to control access to resources based on user roles. +* **JSON Web Tokens (JWT):** Use JWTs for secure authentication and authorization in APIs. + +### 4.4 Data Protection + +* **Encryption:** Encrypt sensitive data at rest and in transit. +* **Data Masking:** Mask sensitive data in logs and other non-production environments. +* **Access Control:** Implement strict access control policies to limit access to sensitive data. +* **Regular Backups:** Perform regular backups of your data. + +### 4.5 Secure API Communication + +* **HTTPS:** Use HTTPS for all API communication. +* **API Keys:** Protect API keys and store them securely. +* **Rate Limiting:** Implement rate limiting to prevent abuse. +* **Input Validation:** Validate all API requests. +* **Output Sanitization:** Sanitize API responses to prevent XSS attacks. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +* **`unittest` or `pytest`:** Use the `unittest` or `pytest` framework for unit testing. +* **Test Coverage:** Aim for high test coverage (e.g., 80% or higher). +* **Test-Driven Development (TDD):** Consider using TDD to write tests before writing code. +* **Arrange-Act-Assert:** Structure your unit tests using the Arrange-Act-Assert pattern. +* **Mocking and Stubbing:** Use mocking and stubbing to isolate units of code and test them in isolation. + +### 5.2 Integration Testing + +* **Test Interactions:** Test the interactions between different components or modules. +* **Database Testing:** Test the integration with databases. +* **API Testing:** Test the integration with external APIs. +* **Real Dependencies:** Use real dependencies (e.g., a real database) for integration tests when possible. + +### 5.3 End-to-End Testing + +* **Simulate User Flows:** Simulate real user flows to test the entire application from end to end. +* **UI Testing:** Use UI testing frameworks (e.g., Selenium, Playwright) to test the user interface. +* **Browser Automation:** Automate browser interactions to simulate user behavior. + +### 5.4 Test Organization + +* **Separate Test Directory:** Keep your tests in a separate `tests` directory. +* **Mirror Module Structure:** Mirror the module structure in your test directory. +* **`conftest.py`:** Use `conftest.py` to define fixtures and configuration for your tests. + +### 5.5 Mocking and Stubbing + +* **`unittest.mock` or `pytest-mock`:** Use the `unittest.mock` module or the `pytest-mock` plugin for mocking and stubbing. +* **Patching:** Use patching to replace objects or functions with mocks during testing. +* **Context Managers:** Use context managers to manage mock objects. +* **Side Effects:** Define side effects for mock objects to simulate different scenarios. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +* **Incorrect Type Annotations:** Using incorrect or incomplete type annotations. +* **Ignoring Pyright Errors:** Ignoring or silencing pyright errors without understanding the root cause. +* **Overuse of `Any`:** Overusing the `Any` type, which defeats the purpose of static typing. +* **Incorrect Configuration:** Incorrectly configuring pyright settings. +* **Not Updating Stubs:** Failing to update stubs (`.pyi` files) when the code changes. + +### 6.2 Edge Cases + +* **Dynamic Typing:** Dealing with dynamic typing features of Python. +* **Type Inference Limitations:** Understanding the limitations of pyright's type inference capabilities. +* **Complex Generics:** Handling complex generic types. +* **Meta-programming:** Dealing with meta-programming techniques. + +### 6.3 Version-Specific Issues + +* **Python Version Compatibility:** Ensuring compatibility with different Python versions. +* **Pyright Version Compatibility:** Being aware of changes and bug fixes in different pyright versions. + +### 6.4 Compatibility Concerns + +* **Third-Party Libraries:** Dealing with third-party libraries that may not have complete or accurate type annotations. +* **Integration with Other Tools:** Ensuring compatibility with other tools in your development workflow. + +### 6.5 Debugging Strategies + +* **Read Pyright Output Carefully:** Carefully examine pyright's error messages and warnings. +* **Use a Debugger:** Use a debugger (e.g., `pdb` or the debugger in your IDE) to step through your code and inspect variables. +* **Simplify the Code:** Simplify the code to isolate the source of the error. +* **Consult Documentation:** Consult the pyright documentation and community resources. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +* **Visual Studio Code:** VS Code with the Pylance extension (or the BasedPyright extension for BasedPyright features). +* **pyright CLI:** The pyright command-line tool for static analysis. +* **pre-commit:** pre-commit for automated code formatting and linting. +* **mypy:** Consider using mypy in conjunction with pyright, as some checks can be performed by mypy that pyright does not offer. + +### 7.2 Build Configuration + +* **`pyrightconfig.json` or `pyproject.toml`:** Configure pyright settings using a `pyrightconfig.json` or `pyproject.toml` file. +* **Include and Exclude Paths:** Specify include and exclude paths to control which files are analyzed by pyright. +* **Type Checking Mode:** Set the type checking mode to `basic`, `strict`, or `off`. +* **Python Version:** Specify the target Python version. +* **Enable Strict Rules:** Enable strict rules to enforce more stringent type checking. + +### 7.3 Linting and Formatting + +* **`flake8` or `ruff`:** Use linters like `flake8` or `ruff` to enforce code style guidelines. +* **`black`:** Use `black` for automatic code formatting. +* **pre-commit Hooks:** Integrate linters and formatters into your pre-commit workflow. + +### 7.4 Deployment Best Practices + +* **Containerization:** Use Docker to containerize your application. +* **Infrastructure as Code (IaC):** Use IaC tools (e.g., Terraform, Ansible) to manage your infrastructure. +* **Continuous Integration/Continuous Deployment (CI/CD):** Implement a CI/CD pipeline for automated testing and deployment. +* **Monitoring:** Monitor your application's performance and health in production. + +### 7.5 CI/CD Integration + +* **GitHub Actions or GitLab CI:** Use CI/CD platforms like GitHub Actions or GitLab CI. +* **Automated Testing:** Run automated tests in your CI/CD pipeline. +* **Static Analysis:** Run pyright and other static analysis tools in your CI/CD pipeline. +* **Deployment Automation:** Automate the deployment process in your CI/CD pipeline. + +## BasedPyright Specific Enhancements: + +* **reportUnreachable:** Reports errors on unreachable code, enhancing error detection in conditional blocks. +* **reportAny:** Bans the `Any` type completely, encouraging more precise type annotations and reducing potential runtime errors. +* **reportPrivateLocalImportUsage:** Restricts private imports within local code, promoting explicit re-exports and clearer module interfaces. +* **reportImplicitRelativeImport:** Flags incorrect relative imports, preventing module loading issues in various execution contexts. +* **reportInvalidCast:** Prevents non-overlapping casts, ensuring type safety during casting operations. +* **reportUnsafeMultipleInheritance:** Discourages multiple inheritance from classes with constructors, reducing the risk of unexpected behavior in complex class hierarchies. +* **Pylance Feature Re-implementations:** Enables features previously exclusive to Pylance such as import suggestions, semantic highlighting, and improved docstrings for compiled modules, offering more robust IDE support across different editors. +* **Inline TypedDict Support:** Reintroduces the support for defining TypedDicts inline, offering convenient type annotation syntax. +* **Better CI Integration:** Leverages GitHub Actions and GitLab CI for seamless error reporting in pull requests and merge requests. +* **Strict Defaults:** Defaults to a stricter type checking mode (all) and assumes code can run on any operating system (All), promoting more comprehensive type analysis. + +By following these best practices, you can leverage pyright and BasedPyright to improve the quality, maintainability, and security of your Python projects. \ No newline at end of file diff --git a/.cursor/rules/pyside.mdc b/.cursor/rules/pyside.mdc new file mode 100644 index 000000000..55b36e9d9 --- /dev/null +++ b/.cursor/rules/pyside.mdc @@ -0,0 +1,237 @@ +--- +description: This rule enforces best practices and coding standards for developing applications with the PySide library. It covers code organization, performance, security, testing, and common pitfalls to ensure maintainable and robust applications. +globs: *.py +--- +# PySide Best Practices and Coding Standards + +This document outlines best practices and coding standards for developing applications with the PySide library. Adhering to these guidelines will promote code readability, maintainability, and robustness. + +## 1. General Guidelines + +- **Follow PEP 8:** Adhere to the PEP 8 style guide for Python code. This includes: + - Consistent naming conventions (e.g., `snake_case` for functions and variables, `CamelCase` for classes). + - 4 spaces for indentation. + - Maximum line length of 79 characters (or 99 if agreed upon by the team). + - Blank lines separating top-level function and class definitions. +- **Qt and Python Naming Conventions:** Prefer Qt's `CamelCase` naming conventions for methods within classes inheriting from Qt widgets. This promotes consistency with the Qt framework. For example, use `loadItems()` instead of `load_items()`. +- **Avoid Wildcard Imports:** Import specific classes or modules instead of using wildcard imports (`from PySide2.QtWidgets import *`). This keeps the namespace clean and avoids potential conflicts. +- **Use Docstrings and Comments:** Document your methods and classes using docstrings (following PEP 257) to explain their purpose and usage. Use inline comments sparingly to clarify complex code, ensuring they are up-to-date. +- **Use Virtual Environments:** Always use virtual environments (e.g., `venv`, `conda`) to manage dependencies for each project. This isolates project dependencies and avoids conflicts between different projects. +- **Consistent Style:** Maintain a consistent coding style throughout the project. Use a linter and formatter (e.g., `flake8`, `pylint`, `black`, `autopep8`) to enforce the coding style. +- **Meaningful Variable Names:** Choose descriptive and meaningful names for variables, functions, and classes. + +## 2. Code Organization and Structure + +- **Directory Structure:** A well-defined directory structure is crucial for maintainability. Consider the following structure: + + + project_name/ + ├── src/ + │ ├── main.py # Entry point of the application + │ ├── gui/ + │ │ ├── __init__.py + │ │ ├── main_window.py # Main window class + │ │ ├── widgets/ + │ │ │ ├── __init__.py + │ │ │ ├── custom_widget.py # Custom widgets + │ │ ├── models/ + │ │ │ ├── __init__.py + │ │ │ ├── data_model.py # Data models + │ ├── core/ + │ │ ├── __init__.py + │ │ ├── logic.py # Core application logic + │ ├── resources/ + │ │ ├── __init__.py + │ │ ├── icons/ + │ │ │ ├── ... + │ │ ├── ui/ # Stores .ui files generated by Qt Designer + │ │ │ ├── main_window.ui + ├── tests/ + │ ├── __init__.py + │ ├── test_main_window.py # Unit tests for main window + ├── requirements.txt # Project dependencies + ├── README.md # Project documentation + ├── .gitignore # Git ignore file + + +- **File Naming Conventions:** + - Python files: `snake_case.py` (e.g., `main_window.py`, `custom_widget.py`). + - UI files: `snake_case.ui` (e.g., `main_window.ui`, `settings_dialog.ui`). + - Resource files: Descriptive names (e.g., `app_icons.qrc`). + +- **Module Organization:** Organize related classes and functions into modules. This improves code reusability and maintainability. + - Use packages (`__init__.py` files) to group related modules. + +- **Component Architecture:** Design the application with a component-based architecture. This promotes modularity and testability. + - Separate the UI (views) from the application logic (controllers and models). + - Use signals and slots for communication between components. + - Use dependency injection to decouple components. + +- **Code Splitting:** Split large files into smaller, more manageable files. This makes it easier to understand and maintain the code. + - Group related functions and classes into separate modules. + - Extract common code into utility functions or classes. + - Consider splitting UI logic into separate files (e.g., event handlers, data binding). + +## 3. Common Patterns and Anti-Patterns + +- **Model-View-Controller (MVC) or Model-View-Presenter (MVP):** Use these patterns to separate the UI from the application logic. + - **Model:** Represents the data and business logic. + - **View:** Displays the data to the user and handles user input. + - **Controller/Presenter:** Acts as an intermediary between the Model and the View, handling user actions and updating the View based on changes in the Model. + +- **Signals and Slots:** Leverage Qt's signals and slots mechanism for communication between objects. This is a powerful and flexible way to handle events and notifications. + - Use named signals and slots for better readability and maintainability. Avoid directly connecting signals and slots by string names if possible, prefer member functions. + - Disconnect signals when they are no longer needed to avoid memory leaks. + +- **Resource Management:** Properly manage resources (e.g., files, network connections, database connections) to avoid memory leaks and ensure the application runs smoothly. + - Use `with` statements or `try...finally` blocks to ensure resources are released even if exceptions occur. + - Consider using `QScopedPointer` for managing Qt objects. + +- **Data Binding:** Use data binding to automatically synchronize data between the Model and the View. This simplifies UI development and reduces boilerplate code. + - PySide supports data binding through `QDataWidgetMapper` and custom models. + +- **Anti-Patterns:** + - **God Objects:** Avoid creating classes that are too large and do too much. Split them into smaller, more focused classes. + - **Copy-Pasting Code:** Avoid duplicating code. Extract common code into reusable functions or classes. + - **Ignoring Exceptions:** Don't just catch exceptions and ignore them. Log the exception and handle it appropriately. + - **Tight Coupling:** Avoid tight coupling between components. Use dependency injection and interfaces to decouple components. + +- **State Management:** + - Use a central state management system (e.g., Redux-like pattern, or a simple state class) to manage the application's state. This is particularly helpful in large applications. + - Consider making the state immutable for easier debugging and predictability. + +- **Error Handling:** + - Use `try...except` blocks to handle exceptions. Catch specific exceptions whenever possible. Log exceptions using Python's `logging` module. + - Provide informative error messages to the user. Consider using message boxes or a status bar to display error messages. + +## 4. Performance Considerations + +- **Optimization Techniques:** + - **Avoid Blocking the Main Thread:** Long-running operations (e.g., network requests, file I/O) should be performed in a separate thread to avoid blocking the main UI thread. + - Use `QThread` or `QThreadPool` for multithreading. Use signals and slots to communicate between threads. + - **Efficient Data Structures:** Choose the appropriate data structures for the task. For example, use `QHash` instead of `QMap` if order doesn't matter and you need faster lookups. + - **Minimize Widget Creation:** Creating and destroying widgets can be expensive. Reuse widgets whenever possible. Consider using `QStackedWidget` to switch between different views without creating new widgets. + - **Optimize Painting:** Optimize custom painting operations by minimizing the amount of drawing and using caching techniques. + - **Profiling:** Use profiling tools to identify performance bottlenecks. Python's `cProfile` and `line_profiler` can be useful. Qt also offers profiling tools. + +- **Memory Management:** + - **Avoid Circular References:** Circular references can prevent objects from being garbage collected. Use `weakref` or manually break the references. + - **Use `QObject` Parenting:** Take advantage of `QObject`'s parenting mechanism for automatic memory management. When a parent object is destroyed, all its children are also destroyed. + - **Delete Objects Manually:** In some cases, you may need to manually delete objects to free memory. Use `del` to delete Python objects, and consider using `QScopedPointer` or manually calling `deleteLater()` on `QObject` instances when appropriate (especially when dealing with C++ objects exposed to Python). + +- **Rendering Optimization:** + - **Use Double Buffering:** Enable double buffering to reduce flickering during repainting. + - **Minimize Overlapping Updates:** Reduce the number of times the UI is updated by batching updates together. + - **Use Graphics Effects Sparingly:** Graphics effects can be expensive. Use them sparingly and optimize their settings. + - **Caching:** Cache frequently used resources (e.g., images, fonts) to avoid reloading them repeatedly. + +- **Bundle Size Optimization:** (Relevant for deployment using tools like PyInstaller) + - **Minimize Dependencies:** Only include the necessary dependencies in the project. Remove unused dependencies. + - **Use UPX Compression:** UPX can compress the executable file to reduce its size. + - **Exclude Unnecessary Files:** Exclude unnecessary files (e.g., documentation, tests) from the bundle. + +- **Lazy Loading:** Delay the loading of resources or components until they are needed. This can improve startup time and reduce memory usage. + - Load UI files on demand. + - Load images or data only when they are visible. + +## 5. Security Best Practices + +- **Common Vulnerabilities:** + - **Code Injection:** Prevent code injection by validating user input and avoiding the use of `eval()` or `exec()`. Be very cautious about using `pickle` to deserialize data, as it can be exploited for code execution. + - **Cross-Site Scripting (XSS):** If the application displays user-generated content, sanitize the input to prevent XSS attacks. Consider using Qt's built-in functions for HTML encoding. + - **SQL Injection:** If the application uses a database, use parameterized queries to prevent SQL injection attacks. Never concatenate user input directly into SQL queries. + - **Path Traversal:** Validate user-provided file paths to prevent path traversal attacks. Use `QDir::cleanPath()` to sanitize file paths. + - **Denial of Service (DoS):** Implement rate limiting and other measures to protect against DoS attacks. + +- **Input Validation:** + - **Validate all user input:** Check that the input is of the correct type, format, and range. + - **Use regular expressions for complex validation:** Regular expressions can be used to validate email addresses, phone numbers, and other complex data formats. + - **Sanitize input:** Remove or encode any potentially harmful characters from the input. + +- **Authentication and Authorization:** + - **Use strong passwords:** Enforce strong password policies and use a secure hashing algorithm (e.g., bcrypt, Argon2) to store passwords. + - **Implement proper authentication:** Verify the user's identity before granting access to protected resources. + - **Implement authorization:** Control which users have access to which resources. + - **Use secure communication protocols:** Use HTTPS to encrypt communication between the client and the server. + +- **Data Protection:** + - **Encrypt sensitive data:** Encrypt sensitive data (e.g., passwords, credit card numbers) before storing it. + - **Use secure storage:** Store sensitive data in a secure location, such as a hardware security module (HSM) or a secure enclave. + - **Protect against data leaks:** Be careful about logging sensitive data or storing it in temporary files. + +- **Secure API Communication:** + - **Use HTTPS:** Always use HTTPS for communication with APIs. + - **Validate API responses:** Verify that the API response is valid and that it hasn't been tampered with. + - **Use API keys or tokens:** Use API keys or tokens to authenticate API requests. + +## 6. Testing Approaches + +- **Unit Testing:** Test individual components (e.g., classes, functions) in isolation. + - Use a unit testing framework (e.g., `unittest`, `pytest`). + - Mock or stub dependencies to isolate the component being tested. + - Test all possible scenarios, including edge cases and error conditions. + +- **Integration Testing:** Test the interaction between different components. + - Verify that the components work together correctly. + - Use real dependencies or integration test environments. + - Test common use cases. + +- **End-to-End Testing:** Test the entire application from the user's perspective. + - Simulate user interactions. + - Verify that the application behaves as expected. + - Use an end-to-end testing framework (e.g., Selenium, Playwright). + +- **Test Organization:** + - Create a separate `tests` directory for test files. + - Organize test files to mirror the source code structure. + - Use descriptive test names. + - Run tests automatically using a CI/CD pipeline. + +- **Mocking and Stubbing:** + - Use mocking to replace dependencies with controlled objects. + - Use stubbing to provide canned responses for dependencies. + - Use a mocking framework (e.g., `unittest.mock`, `pytest-mock`). + - Be careful not to over-mock. Only mock the dependencies that are necessary to isolate the component being tested. + +## 7. Common Pitfalls and Gotchas + +- **QObject Ownership and Memory Management:** Incorrect object ownership can lead to memory leaks or crashes. Ensure that `QObject` instances have a clear parent-child relationship or are managed with `QScopedPointer`. +- **Event Loop Blocking:** Long-running tasks in the main thread will freeze the UI. Offload these tasks to separate threads using `QThread` or `QThreadPool`. +- **Signal-Slot Connections:** Incorrect signal-slot connections can lead to unexpected behavior. Always check that the signal and slot signatures match. +- **Thread Safety:** Be aware of thread safety issues when accessing shared resources from multiple threads. Use mutexes or other synchronization mechanisms to protect shared data. +- **UI Thread Access:** Only access UI elements from the main thread. Use `QMetaObject::invokeMethod()` to invoke methods on UI objects from other threads. +- **Resource File Paths:** Be careful with resource file paths. Use absolute paths or relative paths that are relative to the application's resource directory. Use `:/` to refer to Qt resources. +- **Unicode Handling:** Ensure that the application correctly handles Unicode characters. Use `QString` for storing and manipulating text. +- **Layout Management:** Understand and use layout managers (`QHBoxLayout`, `QVBoxLayout`, `QGridLayout`, etc.) effectively. Avoid hardcoding widget positions and sizes. +- **Qt Designer Limitations:** Be aware of the limitations of Qt Designer. Some UI elements or behaviors may require manual coding. +- **Event Filters:** Be cautious when using event filters, as they can impact performance and make it harder to debug the application. Only use event filters when necessary. +- **Context Menus and Actions:** When adding actions to context menus ensure you specify the parent to avoid a memory leak. + +## 8. Tooling and Environment + +- **Recommended Development Tools:** + - **Qt Creator:** An IDE specifically designed for Qt development. + - **PyCharm:** A popular Python IDE with excellent support for Qt and PySide. + - **VS Code:** A lightweight and versatile code editor with support for Python and Qt through extensions. + - **Qt Designer:** A visual tool for designing UI layouts. + +- **Build Configuration:** + - **Use `setuptools` or `poetry` for managing dependencies and building the application.** + - **Create a `setup.py` file for the project.** + - **Use a virtual environment to isolate project dependencies.** + +- **Linting and Formatting:** + - **Use `flake8`, `pylint`, or `ruff` for linting the code.** + - **Use `black` or `autopep8` for formatting the code.** + - **Configure the linter and formatter to follow the project's coding style.** + +- **Deployment:** + - **Use `PyInstaller`, `cx_Freeze`, or similar tools to create standalone executables.** + - **Bundle all necessary dependencies with the executable.** + - **Test the executable on different platforms.** + +- **CI/CD Integration:** + - **Use a CI/CD platform (e.g., Jenkins, GitHub Actions, GitLab CI) to automate the build, test, and deployment process.** + - **Run tests automatically on every commit.** + - **Create deployment packages automatically on every release.** + - **Automate code quality checks with linting and formatting tools.** \ No newline at end of file diff --git a/.cursor/rules/pytest.mdc b/.cursor/rules/pytest.mdc new file mode 100644 index 000000000..d5d787750 --- /dev/null +++ b/.cursor/rules/pytest.mdc @@ -0,0 +1,335 @@ +--- +description: This rule file outlines comprehensive best practices for using pytest in Python projects, covering code organization, testing strategies, performance optimization, security measures, and common pitfalls to avoid. +globs: *.py +--- +# Pytest Best Practices: A Comprehensive Guide + +This document provides a detailed guide to using pytest effectively in Python projects, covering various aspects from code organization to security considerations. It aims to provide actionable guidance for developers to improve their testing practices and build robust applications. + +## Library Information: +- Name: pytest +- Tags: development, testing, python + +## 1. Code Organization and Structure + +A well-organized codebase is crucial for maintainability and testability. Here are best practices for structuring your pytest projects: + +### 1.1. Directory Structure + +- **Separate `tests/` directory:** Keep your tests in a directory separate from your application code, typically named `tests/`. This promotes isolation and cleaner project structure. + + + my_project/ + ├── my_app/ + │ ├── __init__.py + │ ├── module1.py + │ └── module2.py + ├── tests/ + │ ├── __init__.py + │ ├── test_module1.py + │ └── test_module2.py + └── pyproject.toml + + +- **`src` layout (Recommended):** Consider using a `src` layout to further isolate application code from the project root. This prevents import conflicts and improves clarity. + + + my_project/ + ├── src/ + │ └── my_app/ + │ ├── __init__.py + │ ├── module1.py + │ └── module2.py + ├── tests/ + │ ├── __init__.py + │ ├── test_module1.py + │ └── test_module2.py + └── pyproject.toml + + +### 1.2. File Naming Conventions + +- **`test_*.py` or `*_test.py`:** pytest automatically discovers test files matching these patterns. +- **Descriptive names:** Use clear and descriptive names for your test files to indicate what they are testing (e.g., `test_user_authentication.py`). + +### 1.3. Module Organization + +- **Mirror application structure:** Structure your test modules to mirror the structure of your application code. This makes it easier to locate tests for specific modules. +- **`__init__.py`:** Include `__init__.py` files in your test directories to ensure they are treated as Python packages. + +### 1.4. Component Architecture + +- **Isolate components:** Design your application with well-defined components that can be tested independently. +- **Dependency injection:** Use dependency injection to provide components with their dependencies, making it easier to mock and stub external resources during testing. + +### 1.5. Code Splitting + +- **Small, focused functions:** Break down large functions into smaller, focused functions that are easier to test. +- **Modular design:** Organize your code into modules with clear responsibilities. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +- **Arrange-Act-Assert (AAA):** Structure your tests following the AAA pattern for clarity. + - **Arrange:** Set up the test environment and prepare any necessary data. + - **Act:** Execute the code being tested. + - **Assert:** Verify that the code behaved as expected. + + python + def test_example(): + # Arrange + data = ... + expected_result = ... + + # Act + result = function_under_test(data) + + # Assert + assert result == expected_result + + +- **Fixture factory:** Use fixture factories to create reusable test data. + + python + import pytest + + @pytest.fixture + def user_factory(): + def create_user(username, email): + return {"username": username, "email": email} + return create_user + + def test_create_user(user_factory): + user = user_factory("testuser", "test@example.com") + assert user["username"] == "testuser" + + +### 2.2. Recommended Approaches + +- **Use fixtures for setup and teardown:** Fixtures help manage test dependencies and ensure a clean test environment. +- **Parameterize tests:** Use `@pytest.mark.parametrize` to run the same test with different inputs and expected outputs, reducing code duplication. +- **Use descriptive names for tests and fixtures:** This makes it easier to understand the purpose of each test and fixture. +- **Single Assertion per Test:** A single assertion per test makes it easier to identify the specific failure point. + +### 2.3. Anti-patterns and Code Smells + +- **Over-reliance on fixtures:** Avoid creating too many fixtures, especially for simple data. Use direct data definition in the test if it's not reused. +- **Implicit dependencies:** Make dependencies explicit by passing them as arguments to your functions and tests. +- **Testing implementation details:** Focus on testing the behavior of your code, not the implementation details. This makes your tests more resilient to refactoring. +- **Skipping Tests Without a Reason:** Don't skip tests without a valid reason or comment explaining why. + +### 2.4. State Management + +- **Stateless tests:** Ensure your tests are stateless and independent to avoid unexpected side effects. Each test should set up its own data and clean up after itself. +- **Fixture scopes:** Use fixture scopes (`session`, `module`, `function`) to control the lifecycle of fixtures and manage state effectively. + +### 2.5. Error Handling + +- **Test exception handling:** Write tests to verify that your code handles exceptions correctly. + + python + import pytest + + def divide(a, b): + if b == 0: + raise ValueError("Cannot divide by zero") + return a / b + + def test_divide_by_zero(): + with pytest.raises(ValueError) as e: + divide(10, 0) + assert str(e.value) == "Cannot divide by zero" + + +- **Use `pytest.raises`:** Use `pytest.raises` to assert that a specific exception is raised. +- **Log errors:** Ensure your application logs errors appropriately, and consider writing tests to verify that errors are logged correctly. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +- **Profile slow tests:** Use the `--durations` option to identify slow tests and optimize them. +- **Parallel test execution:** Use `pytest-xdist` to run tests in parallel and reduce overall test execution time. `pip install pytest-xdist` then run `pytest -n auto`. The `auto` option utilizes all available CPU cores. +- **Caching:** Cache expensive computations to avoid redundant calculations during testing. + +### 3.2. Memory Management + +- **Resource cleanup:** Ensure your tests clean up any resources they allocate, such as temporary files or database connections. +- **Limit fixture scope:** Use the appropriate fixture scope to minimize the lifetime of fixtures and reduce memory consumption. + +### 3.3. Bundle Size Optimization + +- **N/A:** Pytest itself doesn't directly impact bundle sizes, but your application code should be optimized separately. + +### 3.4. Lazy Loading + +- **N/A:** Lazy loading is more relevant to application code than pytest itself, but can be used within fixtures if necessary to defer initialization. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities + +- **Injection attacks:** Prevent injection attacks by validating and sanitizing user inputs. +- **Cross-site scripting (XSS):** Protect against XSS vulnerabilities by escaping user-generated content. +- **Authentication and authorization flaws:** Implement secure authentication and authorization mechanisms to protect sensitive data. + +### 4.2. Input Validation + +- **Validate all inputs:** Validate all user inputs to ensure they conform to expected formats and ranges. +- **Use parameterized tests:** Use parameterized tests to test input validation logic with a variety of inputs, including edge cases and invalid values. + +### 4.3. Authentication and Authorization + +- **Test authentication:** Write tests to verify that your authentication mechanisms are working correctly. +- **Test authorization:** Write tests to verify that users only have access to the resources they are authorized to access. + +### 4.4. Data Protection + +- **Encrypt sensitive data:** Encrypt sensitive data at rest and in transit. +- **Use secure storage:** Store sensitive data in secure storage locations with appropriate access controls. + +### 4.5. Secure API Communication + +- **Use HTTPS:** Always use HTTPS for API communication to protect data in transit. +- **Validate API responses:** Validate API responses to ensure they are valid and haven't been tampered with. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +- **Test individual units:** Unit tests should focus on testing individual functions, methods, or classes in isolation. +- **Mock dependencies:** Use mocking to isolate units under test from their dependencies. + +### 5.2. Integration Testing + +- **Test interactions:** Integration tests should focus on testing the interactions between different components of your application. +- **Use real dependencies (where appropriate):** For integration tests, it's often appropriate to use real dependencies, such as databases or external APIs, to ensure that the different components work together correctly. Consider using test containers for database and service dependencies. + +### 5.3. End-to-End Testing + +- **Test complete workflows:** End-to-end tests should focus on testing complete user workflows, from start to finish. +- **Use browser automation:** Use browser automation tools like Selenium or Playwright to simulate user interactions with your application. + +### 5.4. Test Organization + +- **Organize tests by feature:** Group tests by the feature they are testing to improve organization and maintainability. +- **Use clear naming conventions:** Use clear naming conventions for your tests and test files to indicate what they are testing. + +### 5.5. Mocking and Stubbing + +- **Use `mocker` fixture:** Use the `mocker` fixture provided by the `pytest-mock` plugin for mocking and stubbing. +- **Mock external dependencies:** Mock external dependencies, such as databases or APIs, to isolate your tests and prevent them from relying on external resources. +- **Use `autospec=True`:** Use `autospec=True` when mocking to ensure that your mocks have the same API as the original objects. This helps prevent errors caused by incorrect mock implementations. + + python + def test_example(mocker): + mock_external_api = mocker.patch("module.external_api", autospec=True) + mock_external_api.return_value = {"data": "test data"} + + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +- **Not isolating tests:** Failing to isolate tests can lead to unpredictable results and make it difficult to debug failures. +- **Testing implementation details:** Testing implementation details makes your tests brittle and difficult to maintain. +- **Ignoring warnings:** Ignoring warnings from pytest can mask underlying problems in your tests. + +### 6.2. Edge Cases + +- **Empty inputs:** Test your code with empty inputs to ensure it handles them gracefully. +- **Invalid inputs:** Test your code with invalid inputs to ensure it handles them correctly and raises appropriate exceptions. +- **Boundary conditions:** Test your code with boundary conditions to ensure it handles them correctly. + +### 6.3. Version-Specific Issues + +- **Check release notes:** Check the release notes for each new version of pytest to be aware of any breaking changes or new features. +- **Pin dependencies:** Pin your pytest dependency to a specific version to avoid unexpected behavior caused by updates. + +### 6.4. Compatibility Concerns + +- **Check compatibility:** Check the compatibility of pytest with other technologies you are using, such as specific versions of Python or Django. + +### 6.5. Debugging Strategies + +- **Use `--pdb`:** Use the `--pdb` option to drop into the Python debugger when a test fails. +- **Use logging:** Use logging to add debugging information to your tests. +- **Simplify tests:** Simplify failing tests to isolate the cause of the failure. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +- **IDE:** Use a good IDE with pytest support, such as VS Code with the Python extension, PyCharm, or Sublime Text with the appropriate plugins. +- **pytest-watch:** Use `pytest-watch` for automatic test rerunning on file changes. `pip install pytest-watch`, then run `ptw`. + +### 7.2. Build Configuration + +- **Use `pyproject.toml`:** Use a `pyproject.toml` file to configure your pytest settings. + + toml + [tool.pytest.ini_options] + addopts = [ + "--cov=my_app", + "--cov-report term-missing", + "-v", + ] + testpaths = [ + "tests", + ] + + +### 7.3. Linting and Formatting + +- **Use `flake8-pytest-style`:** Use the `flake8-pytest-style` plugin to enforce pytest-specific coding standards. `pip install flake8 flake8-pytest-style` +- **Use `black` or `autopep8`:** Use a code formatter like `black` or `autopep8` to ensure consistent code formatting. `pip install black`, then run `black .` + +### 7.4. Deployment + +- **Include tests in your deployment pipeline:** Ensure your tests are run as part of your deployment pipeline to prevent regressions. +- **Use a dedicated test environment:** Use a dedicated test environment to avoid interfering with your production environment. + +### 7.5. CI/CD Integration + +- **Integrate with CI/CD:** Integrate pytest with your CI/CD system, such as GitHub Actions, GitLab CI, or Jenkins, to automatically run your tests on every commit. + + Example GitHub Actions workflow (`.github/workflows/test.yml`): + + + name: Test + on: + push: + branches: [ main ] + pull_request: + branches: [ main ] + jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + - name: Set up Python 3.10 + uses: actions/setup-python@v3 + with: + python-version: "3.10" + - name: Install dependencies + run: | + python -m pip install --upgrade pip + pip install pytest pytest-cov flake8 flake8-pytest-style black + pip install -e . # Install your project in editable mode + - name: Lint with flake8 + run: | + flake8 . + - name: Test with pytest + run: | + pytest --cov --cov-report xml + - name: Upload coverage to Codecov + uses: codecov/codecov-action@v3 + with: + token: ${{ secrets.CODECOV_TOKEN }} + flags: unittests + env_vars: OS,PYTHON + name: codecov-pytest + + +By following these best practices, you can write effective and maintainable tests with pytest, improving the quality and reliability of your Python applications. \ No newline at end of file diff --git a/.cursor/rules/python.mdc b/.cursor/rules/python.mdc new file mode 100644 index 000000000..dce325c2d --- /dev/null +++ b/.cursor/rules/python.mdc @@ -0,0 +1,347 @@ +--- +description: Comprehensive guidelines for Python development, covering code organization, performance, security, testing, and more. These rules promote maintainable, efficient, and secure Python codebases. +globs: *.py +--- +# Python Best Practices and Coding Standards + +This document outlines comprehensive best practices and coding standards for Python development, aiming to promote clean, efficient, maintainable, and secure code. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure Best Practices + +* **Flat is better than nested (but not always).** Start with a simple structure and refactor as needed. +* **Packages vs. Modules:** Use packages (directories with `__init__.py`) for logical grouping of modules. +* **src layout:** Consider using a `src` directory to separate application code from project-level files (setup.py, requirements.txt, etc.). This helps avoid import conflicts and clarifies the project's boundaries. +* **Typical Project Structure:** + + + project_name/ + ├── src/ + │ ├── package_name/ + │ │ ├── __init__.py + │ │ ├── module1.py + │ │ ├── module2.py + │ ├── main.py # Entry point + ├── tests/ + │ ├── __init__.py + │ ├── test_module1.py + │ ├── test_module2.py + ├── docs/ + │ ├── conf.py + │ ├── index.rst + ├── .gitignore + ├── pyproject.toml or setup.py + ├── README.md + ├── requirements.txt or requirements-dev.txt + + +### 1.2. File Naming Conventions + +* **Modules:** Lowercase, with underscores for readability (e.g., `my_module.py`). +* **Packages:** Lowercase (e.g., `my_package`). Avoid underscores unless necessary. +* **Tests:** Start with `test_` (e.g., `test_my_module.py`). + +### 1.3. Module Organization Best Practices + +* **Single Responsibility Principle:** Each module should have a well-defined purpose. +* **Imports:** + * Order: standard library, third-party, local. + * Absolute imports are generally preferred (e.g., `from my_package.module1 import function1`). + * Use explicit relative imports (`from . import sibling_module`) when dealing with complex package layouts where absolute imports are overly verbose or impractical. +* **Constants:** Define module-level constants in uppercase (e.g., `MAX_ITERATIONS = 100`). +* **Dunder names:** `__all__`, `__version__`, etc. should be after the module docstring but before any imports (except `from __future__`). Use `__all__` to explicitly define the public API. + +### 1.4. Component Architecture Recommendations + +* **Layered Architecture:** Suitable for larger applications, separating concerns into presentation, business logic, and data access layers. +* **Microservices:** For very large applications, consider breaking the system into smaller, independent services. +* **Hexagonal/Clean Architecture:** Emphasizes decoupling business logic from external dependencies like databases and frameworks. +* **Dependency Injection:** Use dependency injection to improve testability and reduce coupling. + +### 1.5. Code Splitting Strategies + +* **By Functionality:** Split code into modules based on distinct functionalities (e.g., user management, data processing). +* **By Layer:** Separate presentation, business logic, and data access code. +* **Lazy Loading:** Use `importlib.import_module()` to load modules on demand, improving startup time. +* **Conditional Imports:** Import modules only when needed, based on certain conditions. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +* **Singleton:** Restrict instantiation of a class to one object. +* **Factory:** Create objects without specifying the exact class to be created. +* **Observer:** Define a one-to-many dependency between objects so that when one object changes state, all its dependents are notified. +* **Strategy:** Define a family of algorithms, encapsulate each one, and make them interchangeable. +* **Decorator:** Add responsibilities to objects dynamically. +* **Context Manager:** Guarantees resources are properly cleaned up (e.g., files are closed). + +### 2.2. Recommended Approaches for Common Tasks + +* **Data Validation:** Use libraries like `pydantic` or `marshmallow` for data validation and serialization. +* **Configuration Management:** Use libraries like `python-decouple`, `dynaconf` or standard library's `configparser` to manage environment-specific settings. +* **Logging:** Use the `logging` module for structured logging. Configure log levels and handlers appropriately. +* **Command-Line Interfaces:** Use `argparse`, `click` or `typer` for creating command-line interfaces. +* **Asynchronous Programming:** Use `asyncio` for I/O-bound and concurrency problems. + +### 2.3. Anti-patterns and Code Smells + +* **God Class:** A class that does too much. Break it down into smaller, more focused classes. +* **Shotgun Surgery:** Making small changes to many different classes at once. Indicates poor cohesion. +* **Spaghetti Code:** Unstructured and difficult-to-follow code. Refactor using well-defined functions and classes. +* **Duplicate Code:** Extract common code into reusable functions or classes (DRY - Don't Repeat Yourself). +* **Magic Numbers/Strings:** Use named constants instead of hardcoded values. +* **Nested Callbacks:** Avoid excessive nesting of callbacks. Use `async/await` or promises for better readability. +* **Premature Optimization:** Don't optimize code before identifying bottlenecks. + +### 2.4. State Management Best Practices + +* **Stateless Functions:** Prefer stateless functions where possible. +* **Immutable Data:** Use immutable data structures to prevent accidental modification. +* **Explicit State:** Manage state explicitly using classes or data structures. Avoid relying on global variables. +* **Context Variables:** Use `contextvars` (Python 3.7+) for managing request-scoped state in asynchronous applications. +* **Redux-like patterns:** Consider redux-like patterns for managing client-side and complex application state. + +### 2.5. Error Handling Patterns + +* **Specific Exceptions:** Catch specific exceptions rather than broad `Exception` or `BaseException`. +* **`try...except...finally`:** Use `finally` to ensure cleanup code is always executed. +* **Context Managers:** Use context managers (`with open(...) as f:`) for resource management. +* **Logging Errors:** Log exceptions with complete traceback information. +* **Raising Exceptions:** Raise exceptions with informative error messages. +* **Custom Exceptions:** Create custom exception classes for specific error conditions. +* **Avoid using exceptions for control flow.** Exceptions should represent exceptional circumstances. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +* **Profiling:** Use `cProfile` to identify performance bottlenecks. +* **Efficient Data Structures:** Choose the right data structure for the task (e.g., `set` for membership testing, `dict` for lookups). +* **List Comprehensions and Generators:** Use list comprehensions and generator expressions for concise and efficient code. +* **Vectorization with NumPy:** Use NumPy for numerical computations, leveraging vectorized operations. +* **Just-In-Time Compilation (JIT):** Consider using JIT compilers like Numba for performance-critical code. +* **Caching:** Implement caching mechanisms using `functools.lru_cache` or external caching libraries like Redis or Memcached. +* **String Concatenation:** Use `''.join(iterable)` for efficient string concatenation. +* **Avoid Global Variables:** Accessing local variables is faster than accessing global variables. +* **Cython:** Use Cython to write C extensions for Python, improving performance. + +### 3.2. Memory Management Considerations + +* **Garbage Collection:** Understand Python's garbage collection mechanism. +* **Object References:** Be mindful of object references and circular dependencies, which can prevent garbage collection. +* **Memory Profiling:** Use `memory_profiler` to identify memory leaks. +* **Slots:** Use `__slots__` in classes to reduce memory footprint (disables `__dict__`). +* **Generators:** Use generators for processing large datasets without loading them into memory. +* **Data type sizing:** Use the most efficient data types possible to reduce memory use. + +### 3.3. Rendering Optimization + +* N/A for core Python libraries. Relevant for GUI frameworks (e.g., Tkinter, PyQt, Kivy). +* For web development with frameworks such as Django, Flask, or Pyramid, use efficient templating, caching and database query optimizations. + +### 3.4. Bundle Size Optimization + +* N/A for core Python libraries. Relevant for web applications or when creating executable bundles. +* Use tools like `PyInstaller` or `cx_Freeze` to create executable bundles. +* Minimize dependencies to reduce bundle size. +* Use code minification techniques. + +### 3.5. Lazy Loading Strategies + +* **Module Loading:** Use `importlib.import_module()` to load modules on demand. +* **Data Loading:** Load large datasets only when needed. +* **Deferred Execution:** Use generators or coroutines to defer execution of code. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities and Prevention + +* **SQL Injection:** Use parameterized queries or ORMs to prevent SQL injection attacks. +* **Cross-Site Scripting (XSS):** Sanitize user input and escape output to prevent XSS attacks. +* **Cross-Site Request Forgery (CSRF):** Use CSRF tokens to protect against CSRF attacks. +* **Command Injection:** Avoid executing arbitrary commands based on user input. If necessary, sanitize input carefully. +* **Path Traversal:** Validate file paths to prevent path traversal attacks. +* **Denial of Service (DoS):** Implement rate limiting and input validation to protect against DoS attacks. +* **Pickle Deserialization:** Avoid using `pickle` to deserialize untrusted data, as it can lead to arbitrary code execution. Use safer alternatives like JSON or Protocol Buffers. +* **Dependency Vulnerabilities:** Regularly audit and update dependencies to address security vulnerabilities. +* **Hardcoded Secrets:** Never hardcode secrets (passwords, API keys) in code. Use environment variables or secure configuration files. + +### 4.2. Input Validation Best Practices + +* **Whitelisting:** Validate input against a whitelist of allowed values. +* **Regular Expressions:** Use regular expressions to validate input formats. +* **Data Type Validation:** Ensure input data types are correct. +* **Length Validation:** Limit the length of input strings. +* **Sanitization:** Remove or escape potentially harmful characters from input. +* **Use libraries:** Use libraries like `cerberus` and `schematics` to assist with validating the input. + +### 4.3. Authentication and Authorization Patterns + +* **Authentication:** + * Use strong password hashing algorithms (e.g., bcrypt, Argon2). + * Implement multi-factor authentication (MFA). + * Use secure session management techniques. + * Consider using a dedicated authentication service (e.g., Auth0, Okta). +* **Authorization:** + * Implement role-based access control (RBAC) or attribute-based access control (ABAC). + * Use a permissions system to control access to resources. + * Enforce the principle of least privilege. + * Use access tokens (JWTs). + +### 4.4. Data Protection Strategies + +* **Encryption:** Encrypt sensitive data at rest and in transit. +* **Data Masking:** Mask sensitive data when displaying it to users. +* **Tokenization:** Replace sensitive data with non-sensitive tokens. +* **Data Loss Prevention (DLP):** Implement DLP measures to prevent sensitive data from leaving the organization. +* **Regular backups and disaster recovery plans.** + +### 4.5. Secure API Communication + +* **HTTPS:** Always use HTTPS for API communication. +* **API Keys:** Use API keys for authentication. +* **OAuth 2.0:** Use OAuth 2.0 for delegated authorization. +* **Input validation**: Validate all API requests before processing. +* **Rate Limiting:** Implement rate limiting to prevent abuse. +* **Web Application Firewall (WAF)** Implement WAF to provide centralized security layer. + +## 5. Testing Approaches + +### 5.1. Unit Testing Strategies + +* **Test Individual Units:** Test individual functions, classes, or modules in isolation. +* **Test-Driven Development (TDD):** Write tests before writing code. +* **Coverage:** Aim for high test coverage. +* **Assertion Styles:** Use appropriate assertion methods (e.g., `assertEqual`, `assertTrue`, `assertRaises`). +* **Boundary conditions:** Test boundary conditions and edge cases. +* **Error conditions:** Test that exceptions are raised when appropriate. + +### 5.2. Integration Testing Approaches + +* **Test Interactions:** Test the interactions between different modules or components. +* **Database Testing:** Test database interactions. +* **API Testing:** Test API endpoints. +* **Mock External Services:** Use mocks to simulate external services during integration tests. +* **Focus on key workflows.** Integration tests should exercise the most important user workflows. + +### 5.3. End-to-End Testing Recommendations + +* **Test Entire System:** Test the entire system from end to end. +* **User Perspective:** Write tests from the perspective of the user. +* **Browser Automation:** Use browser automation tools like Selenium or Playwright. +* **Real-World Scenarios:** Simulate real-world scenarios in end-to-end tests. +* **Focus on critical paths.** End-to-end tests are expensive to write and maintain, so focus on the most critical paths. + +### 5.4. Test Organization Best Practices + +* **Separate Test Directory:** Keep tests in a separate `tests` directory. +* **Mirror Source Structure:** Mirror the source code structure in the test directory. +* **Test Modules:** Create test modules for each source module. +* **Test Classes:** Use test classes to group related tests. +* **Use a test runner:** Use `pytest` or `unittest` test runners. +* **Use fixtures:** Utilize fixtures to setup and tear down resources for tests. + +### 5.5. Mocking and Stubbing Techniques + +* **`unittest.mock`:** Use the `unittest.mock` module for mocking and stubbing. +* **Patching:** Use `patch` to replace objects with mocks during tests. +* **Side Effects:** Define side effects for mocks to simulate different scenarios. +* **Mocking External Dependencies:** Mock external dependencies like databases, APIs, and file systems. +* **Use dependency injection for testability.** Dependency injection makes it easier to mock dependencies. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +* **Mutable Default Arguments:** Avoid using mutable default arguments in function definitions. +* **Scope of Variables:** Be aware of variable scope in nested functions. +* **`==` vs. `is`:** Use `==` for value comparison and `is` for object identity comparison. +* **`try...except` Blocks:** Placing too much code inside try blocks. Keep try blocks as small as possible. +* **Ignoring Exceptions:** Swallowing exceptions without handling or logging them. +* **Incorrect indentation.** Indentation errors are a common source of bugs. +* **Not using virtual environments.** Not using virtual environments can lead to dependency conflicts. + +### 6.2. Edge Cases + +* **Floating-Point Arithmetic:** Be aware of the limitations of floating-point arithmetic. +* **Unicode Handling:** Handle Unicode strings carefully. +* **File Encoding:** Specify file encoding when reading and writing files. +* **Time Zones:** Handle time zones correctly. +* **Resource limits:** Be aware of and handle system resource limits (e.g., file handles, memory). + +### 6.3. Version-Specific Issues + +* **Python 2 vs. Python 3:** Be aware of the differences between Python 2 and Python 3. +* **Syntax Changes:** Be aware of syntax changes in different Python versions. +* **Library Compatibility:** Ensure that libraries are compatible with the Python version being used. +* **Deprecated features.** Avoid using deprecated features. + +### 6.4. Compatibility Concerns + +* **Operating Systems:** Test code on different operating systems (Windows, macOS, Linux). +* **Python Implementations:** Consider compatibility with different Python implementations (CPython, PyPy, Jython). +* **Database Versions:** Ensure compatibility with different database versions. +* **External Libraries:** Be aware of compatibility issues with external libraries. + +### 6.5. Debugging Strategies + +* **`pdb`:** Use the `pdb` debugger for interactive debugging. +* **Logging:** Use logging to track program execution. +* **Print Statements:** Use print statements for simple debugging. +* **Assertions:** Use assertions to check for expected conditions. +* **Profiling:** Use profilers to identify performance bottlenecks. +* **Code Analysis Tools:** Use code analysis tools like pylint or flake8 to detect potential problems. +* **Remote debugging:** Use remote debugging tools when debugging code running on remote servers. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +* **IDEs:** PyCharm, VS Code (with Python extension), Sublime Text. +* **Virtual Environment Managers:** `venv` (built-in), `virtualenv`, `conda`, `pipenv`. +* **Package Managers:** `pip` (default), `conda`, `poetry`. +* **Debuggers:** `pdb`, IDE debuggers. +* **Profilers:** `cProfile`, `memory_profiler`. +* **Linters:** `pylint`, `flake8`. +* **Formatters:** `black`, `autopep8`, `YAPF`. +* **Static Analyzers:** `mypy`, `pytype`. +* **Notebook environments**: Jupyter Notebook, Jupyter Lab, Google Colab. + +### 7.2. Build Configuration Best Practices + +* **`pyproject.toml`:** Use `pyproject.toml` for build configuration (PEP 518, PEP 621). +* **`setup.py`:** Use `setup.py` for legacy projects (but prefer `pyproject.toml` for new projects). +* **Dependency Management:** Specify dependencies in `requirements.txt` or `pyproject.toml`. +* **Virtual Environments:** Use virtual environments to isolate project dependencies. +* **Reproducible builds:** Ensure reproducible builds by pinning dependencies. + +### 7.3. Linting and Formatting Recommendations + +* **PEP 8:** Adhere to PEP 8 style guidelines. +* **Linters:** Use linters to enforce code style and detect potential problems. +* **Formatters:** Use formatters to automatically format code according to PEP 8. +* **Pre-commit Hooks:** Use pre-commit hooks to run linters and formatters before committing code. +* **Consistent Style:** Maintain a consistent code style throughout the project. + +### 7.4. Deployment Best Practices + +* **Virtual Environments:** Deploy applications in virtual environments. +* **Dependency Management:** Install dependencies using `pip install -r requirements.txt` or `poetry install`. +* **Process Managers:** Use process managers like `systemd`, `Supervisor`, or `Docker` to manage application processes. +* **Web Servers:** Use web servers like Gunicorn or uWSGI to serve web applications. +* **Load Balancing:** Use load balancers to distribute traffic across multiple servers. +* **Containerization:** Use containerization technologies like Docker to package and deploy applications. +* **Infrastructure as Code (IaC)** Manage infrastructure using IaC tools like Terraform or CloudFormation. + +### 7.5. CI/CD Integration Strategies + +* **Continuous Integration (CI):** Automatically build and test code on every commit. +* **Continuous Delivery (CD):** Automatically deploy code to staging or production environments. +* **CI/CD Tools:** Use CI/CD tools like Jenkins, GitLab CI, GitHub Actions, CircleCI, or Travis CI. +* **Automated Testing:** Include automated tests in the CI/CD pipeline. +* **Code Analysis:** Integrate code analysis tools into the CI/CD pipeline. +* **Automated deployments.** Automate the deployment process to reduce manual effort and errors. + +By adhering to these best practices and coding standards, developers can create Python code that is more robust, maintainable, and secure. \ No newline at end of file diff --git a/.cursor/rules/pytorch.mdc b/.cursor/rules/pytorch.mdc new file mode 100644 index 000000000..32a846684 --- /dev/null +++ b/.cursor/rules/pytorch.mdc @@ -0,0 +1,381 @@ +--- +description: This rule provides comprehensive guidelines for PyTorch development, covering code organization, performance optimization, security, testing, and common pitfalls. It aims to ensure readable, maintainable, and efficient PyTorch code. +globs: *.py +--- +# PyTorch Best Practices and Coding Standards + +This document provides comprehensive guidelines for developing PyTorch projects, encompassing code organization, performance optimization, security, testing methodologies, and common pitfalls. Adhering to these best practices will result in more readable, maintainable, and efficient PyTorch code. + +**Library Information:** +- Name: PyTorch +- Category: ai_ml +- Subcategory: machine_learning + +## 1. Code Organization and Structure + +### 1.1 Directory Structure Best Practices + +A well-organized directory structure enhances code maintainability and collaboration. Here's a recommended structure for PyTorch projects: + + +project_root/ +├── data/ +│ ├── raw/ +│ ├── processed/ +│ └── ... +├── models/ +│ ├── layers.py +│ ├── networks.py +│ ├── losses.py +│ ├── ops.py +│ └── model_name.py +├── src/ +│ ├── data/ +│ │ ├── datasets.py +│ │ ├── dataloaders.py +│ │ └── transforms.py +│ ├── models/ +│ │ └── ... (model-related code) +│ ├── utils/ +│ │ └── ... (utility functions) +│ └── visualization/ +│ └── ... +├── notebooks/ +│ └── ... (Jupyter notebooks for experimentation) +├── tests/ +│ ├── unit/ +│ ├── integration/ +│ └── ... +├── scripts/ +│ └── train.py +│ └── eval.py +├── configs/ +│ └── ... (Configuration files, e.g., YAML) +├── README.md +├── requirements.txt +├── .gitignore +└── ... + + +- `data/`: Stores raw and processed datasets. +- `models/`: Contains PyTorch model definitions, layers, and custom loss functions. Separate network architectures, individual layers/blocks, and operations into different files. +- `src/`: Holds the main source code, including data loading, model definitions, utility functions, and visualization tools. It's common to split the `src/` folder further based on responsibilities. +- `notebooks/`: Jupyter notebooks for experimentation and exploration. Use notebooks for initial exploration and prototyping but transition finalized code to Python scripts. +- `tests/`: Unit, integration, and end-to-end tests. +- `scripts/`: Training, evaluation, and deployment scripts. The main training script should import model definitions. +- `configs/`: Configuration files for hyperparameter settings and other parameters. + +### 1.2 File Naming Conventions + +- Use descriptive and consistent file names. +- Python files: `lower_with_under.py` (e.g., `data_loader.py`, `model_utils.py`). +- Model files: `model_name.py` (e.g., `resnet.py`, `transformer.py`). +- Configuration files: `config_name.yaml` (e.g., `train_config.yaml`). + +### 1.3 Module Organization + +- Group related functions and classes into modules. +- Use clear and concise module names. +- Include a docstring at the beginning of each module to describe its purpose. +- Follow a consistent import style: + python + # Standard library imports + import os + import sys + + # Third-party library imports + import numpy as np + import torch + import torchvision + + # Local application/library imports + from src.data import data_loader + from src.models import resnet + from src.utils import helper_functions + + +### 1.4 Component Architecture + +- **`nn.Module`:** The fundamental building block for creating neural networks in PyTorch. Always subclass `nn.Module` for defining models, layers, and custom operations. +- **`forward()` method:** Implement the forward pass of your module within the `forward()` method. PyTorch uses the `__call__` method, which executes `forward()`, to pass data through the model. +- **Separation of Concerns:** Design models as compositions of smaller, reusable modules. This promotes modularity and simplifies debugging. + +Example: + +python +import torch.nn as nn + +class ConvBlock(nn.Module): + def __init__(self, in_channels, out_channels, kernel_size=3, stride=1, padding=1): + super().__init__() + self.conv = nn.Conv2d(in_channels, out_channels, kernel_size, stride, padding) + self.relu = nn.ReLU() + self.bn = nn.BatchNorm2d(out_channels) + + def forward(self, x): + x = self.conv(x) + x = self.relu(x) + x = self.bn(x) + return x + +class SimpleCNN(nn.Module): + def __init__(self, num_classes=10): + super().__init__() + self.conv1 = ConvBlock(3, 32) + self.conv2 = ConvBlock(32, 64) + self.pool = nn.MaxPool2d(2, 2) + self.fc = nn.Linear(64 * 8 * 8, num_classes) + + def forward(self, x): + x = self.conv1(x) + x = self.pool(x) + x = self.conv2(x) + x = self.pool(x) + x = x.view(x.size(0), -1) + x = self.fc(x) + return x + + +### 1.5 Code Splitting Strategies + +- **Vertical Splitting (Feature-based):** Divide code based on features or functionalities (e.g., data loading, model definition, training loop). This approach aligns well with module organization. +- **Horizontal Splitting (Layer-based):** Split code based on layers or components of a neural network. This is suitable for complex models with distinct layers (e.g., encoder, decoder). The `models/` directory structure supports this. +- **Microservices (for large projects):** For very large and complex machine learning deployments, consider breaking the workload into microservices - one for training, one for inference, another for data preprocessing, etc. This can improve scalability and fault tolerance. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns Specific to PyTorch + +- **`nn.Sequential`:** A container for chaining layers in a sequential manner. Suitable for simple, linear models. +- **Custom `nn.Module`:** Create custom modules for encapsulating reusable blocks of layers or operations. This promotes modularity and code reuse. +- **Hooks:** Utilize forward and backward hooks for inspecting and modifying activations and gradients during training. Useful for debugging and research. +- **DataParallel/DistributedDataParallel:** Wrap models with `DataParallel` or `DistributedDataParallel` to leverage multiple GPUs for training. `DistributedDataParallel` is generally preferred for multi-node training and offers better performance. +- **Transfer Learning:** Reuse pretrained models from `torchvision.models` as a starting point for new tasks. Fine-tune the pretrained weights or add custom layers. + +### 2.2 Recommended Approaches for Common Tasks + +- **Data Loading:** Use `torch.utils.data.Dataset` and `torch.utils.data.DataLoader` for efficient data loading and batching. Use custom Datasets to control data transformation. +- **Model Definition:** Define models as classes that inherit from `nn.Module`. Use the layers defined in `torch.nn` to build the model. +- **Training Loop:** Implement a structured training loop that iterates over epochs and batches. Include forward pass, loss calculation, backward pass, and optimization steps. +- **Validation:** Evaluate model performance on a validation set during training to monitor overfitting and tune hyperparameters. +- **Checkpointing:** Save model weights and optimizer state during training to resume training or load the best model. + +### 2.3 Anti-patterns and Code Smells to Avoid + +- **Hardcoding shapes:** Avoid hardcoding input or output shapes. Use dynamic shapes or calculate shapes based on input data. +- **Global Variables:** Minimize the use of global variables. Pass data and configurations as function arguments. +- **Magic Numbers:** Avoid using magic numbers without explanation. Define constants with descriptive names. +- **Deeply Nested Loops:** Optimize performance-critical sections of code. Consider using vectorized operations or optimized data structures to avoid nested loops where possible. +- **Ignoring Warnings:** Address all warnings. They often indicate potential problems or inefficiencies in your code. Use a debugger and logging to diagnose runtime behavior. +- **Over-commenting:** Avoid redundant comments that simply reiterate the code. Focus on explaining the *why* rather than the *what*. +- **Not using GPU when Available:** Always check if cuda is available and moves tensors and models to cuda for faster training and inference. + +### 2.4 State Management Best Practices + +- **Model Parameters:** Model parameters are automatically managed by PyTorch. Use `nn.Parameter` to register tensors as model parameters. +- **Buffers:** Use `nn.Buffer` to store non-trainable tensors that are part of the model state (e.g., running statistics in BatchNorm). +- **Optimizer State:** The optimizer maintains its own state, such as learning rate and momentum. Access and modify optimizer state through the `optimizer.state_dict()` and `optimizer.load_state_dict()` methods. +- **Random Seeds:** Set random seeds for reproducibility: + + python + import numpy as np + import torch + + def set_seed(seed): + np.random.seed(seed) + torch.manual_seed(seed) + if torch.cuda.is_available(): + torch.cuda.manual_seed(seed) + torch.cuda.manual_seed_all(seed) # if using multiple GPUs + torch.backends.cudnn.deterministic = True #Ensures that CUDA uses deterministic algorithms. + torch.backends.cudnn.benchmark = False #Prevents CUDA from benchmarking multiple convolution algorithms. + + +### 2.5 Error Handling Patterns + +- **Specific Exception Handling:** Catch specific exceptions rather than using bare `except` clauses. This allows you to handle different error scenarios appropriately. +- **Logging:** Use the `logging` module to log errors and warnings. Include relevant information, such as timestamps, file names, and line numbers. +- **Assertion Statements:** Use `assert` statements to check for unexpected conditions. This can help catch errors early in development. +- **Custom Exceptions:** Define custom exception classes for specific error conditions in your application. This improves code readability and maintainability. +- **Graceful Degradation:** Implement mechanisms to handle errors gracefully, such as providing default values or skipping problematic data points. Avoid crashing the application. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +- **GPU Utilization:** Use a GPU for training and inference. Move models and tensors to the GPU using `.to('cuda')`. +- **Batch Size:** Tune the batch size to maximize GPU utilization without running out of memory. Larger batch sizes generally lead to better performance. +- **Data Loading:** Optimize data loading by using multiple workers in the `DataLoader`. Consider using asynchronous data loading techniques. +- **Mixed Precision Training:** Use mixed precision training (`torch.cuda.amp`) to reduce memory usage and accelerate training on NVIDIA GPUs with Tensor Cores. +- **Gradient Accumulation:** Accumulate gradients over multiple batches to simulate a larger batch size when memory is limited. +- **Operator Fusion:** Utilize PyTorch's JIT compiler (`torch.jit.script` or `torch.jit.trace`) to fuse multiple operations into a single kernel, reducing overhead and improving performance. +- **Memory Usage:** When using `DataParallel`, ensure that the input data is evenly distributed across the available GPUs. Uneven data distribution can result in memory imbalances and slow down training. + +### 3.2 Memory Management + +- **Tensor Deletion:** Delete unnecessary tensors using `del` to free up memory. +- **`torch.no_grad()`:** Use `torch.no_grad()` during inference to disable gradient calculation and reduce memory usage. +- **In-place Operations:** Use in-place operations (e.g., `x.add_(1)`) to modify tensors directly without creating new tensors. Be cautious with in-place operations as they can sometimes cause issues with autograd. +- **Memory Profiling:** Use memory profiling tools to identify memory leaks and optimize memory usage. +- **Garbage Collection:** Explicitly call `gc.collect()` to force garbage collection. This can be useful in situations where memory is not being released promptly. + +### 3.3 Rendering Optimization (if applicable) + +- This section applies if PyTorch is used for rendering tasks (e.g., neural rendering). +- **Optimize Mesh Data Structures:** Use efficient mesh data structures to reduce memory usage and improve rendering performance. +- **Level of Detail (LOD):** Implement LOD techniques to reduce the complexity of rendered objects at a distance. +- **Caching:** Cache frequently accessed data to reduce rendering time. + +### 3.4 Bundle Size Optimization (if applicable) + +- This section applies if PyTorch models are deployed in web applications or other size-sensitive environments. +- **Model Quantization:** Quantize models to reduce their size. Use `torch.quantization` to quantize models to 8-bit integers. +- **Model Pruning:** Prune unimportant weights from models to reduce their size. Use `torch.nn.utils.prune` to prune models. +- **ONNX Export:** Convert PyTorch models to ONNX format for deployment on various platforms. + +### 3.5 Lazy Loading Strategies + +- **Lazy Initialization:** Defer the initialization of expensive resources until they are needed. +- **Data Streaming:** Load data in smaller chunks instead of loading the entire dataset into memory at once. +- **Just-In-Time (JIT) Compilation:** Use `torch.jit.script` or `torch.jit.trace` to compile models just before they are used. This can improve performance and reduce memory usage. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and How to Prevent Them + +- **Adversarial Attacks:** Be aware of adversarial attacks that can manipulate model predictions. Implement techniques like adversarial training and input sanitization. +- **Data Poisoning:** Protect against data poisoning attacks by validating input data and using robust training techniques. +- **Model Extraction:** Prevent model extraction by limiting access to model weights and predictions. +- **Integer Overflow:** Be aware of integer overflow issues in user-provided data. + +### 4.2 Input Validation + +- **Data Type Validation:** Ensure that input data has the correct data type. +- **Range Validation:** Check that input values are within a valid range. +- **Format Validation:** Validate the format of input strings (e.g., URLs, email addresses). +- **Sanitization:** Sanitize input data to remove potentially malicious characters or code. + +### 4.3 Authentication and Authorization Patterns + +- **API Keys:** Use API keys to authenticate clients accessing your models. +- **OAuth 2.0:** Implement OAuth 2.0 for secure authorization of user access. +- **Role-Based Access Control (RBAC):** Use RBAC to restrict access to specific models or functionalities based on user roles. + +### 4.4 Data Protection Strategies + +- **Encryption:** Encrypt sensitive data at rest and in transit. +- **Anonymization:** Anonymize data to protect user privacy. +- **Differential Privacy:** Use differential privacy techniques to protect the privacy of training data. +- **Access Control:** Implement strict access control policies to limit access to sensitive data. + +### 4.5 Secure API Communication + +- **HTTPS:** Use HTTPS to encrypt communication between clients and your API. +- **TLS/SSL:** Use TLS/SSL certificates to secure your API endpoints. +- **Rate Limiting:** Implement rate limiting to prevent denial-of-service attacks. +- **Input Validation:** Always validate and sanitize input data to prevent injection attacks. + +## 5. Testing Approaches + +### 5.1 Unit Testing Strategies + +- **Test Individual Modules:** Write unit tests for individual modules to ensure they function correctly. +- **Test Edge Cases:** Test edge cases and boundary conditions to identify potential errors. +- **Use Assertions:** Use assertions to verify expected outcomes. +- **Test Model Outputs:** Verify that model outputs have the correct shape and data type. + +### 5.2 Integration Testing + +- **Test Interactions Between Modules:** Write integration tests to ensure that different modules work together correctly. +- **Test Data Pipelines:** Test the entire data pipeline, from data loading to model output. +- **Test End-to-End Functionality:** Test the entire application to ensure that it meets the requirements. + +### 5.3 End-to-End Testing + +- **Simulate User Interactions:** Write end-to-end tests that simulate user interactions with the application. +- **Test the Entire System:** Test the entire system, including the user interface, API, and database. +- **Verify Expected Outcomes:** Verify that the application produces the expected outcomes. + +### 5.4 Test Organization + +- **Separate Test Files:** Create separate test files for each module or component. +- **Use a Test Runner:** Use a test runner (e.g., `pytest`, `unittest`) to discover and run tests. +- **Follow a Consistent Naming Convention:** Follow a consistent naming convention for test files and test functions. + +### 5.5 Mocking and Stubbing + +- **Use Mocks to Isolate Units:** Use mocks to isolate units of code from their dependencies. +- **Use Stubs to Provide Test Data:** Use stubs to provide test data to units of code. +- **Verify Interactions with Dependencies:** Verify that units of code interact with their dependencies as expected. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes Developers Make + +- **Incorrect Tensor Shapes:** Pay close attention to tensor shapes when performing operations. Use `torch.Size()` to check tensor shapes. +- **Gradient Issues:** Be mindful of gradient flow. Ensure that gradients are properly calculated and propagated through the network. Use `torch.autograd.set_detect_anomaly(True)` for debugging autograd issues. +- **Memory Leaks:** Be careful to deallocate tensors when they are no longer needed. Use memory profiling tools to identify memory leaks. +- **Data Type Mismatches:** Ensure that tensors have the correct data type (e.g., `torch.float32`, `torch.int64`). +- **Device Mismatches:** Ensure that tensors and models are on the same device (CPU or GPU). + +### 6.2 Edge Cases to Be Aware Of + +- **Empty Datasets:** Handle cases where datasets are empty or contain missing data. +- **Out-of-Memory Errors:** Handle out-of-memory errors gracefully by reducing batch size or using gradient accumulation. +- **Numerical Instability:** Be aware of numerical instability issues, such as vanishing or exploding gradients. Use techniques like gradient clipping and batch normalization. + +### 6.3 Version-Specific Issues + +- **Compatibility with Libraries:** Be aware of compatibility issues between different versions of PyTorch and other libraries. +- **API Changes:** Track API changes in PyTorch releases and update your code accordingly. + +### 6.4 Compatibility Concerns + +- **Hardware Compatibility:** Ensure that your code is compatible with different hardware configurations (e.g., different GPUs). +- **Operating System Compatibility:** Ensure that your code is compatible with different operating systems (e.g., Linux, Windows, macOS). + +### 6.5 Debugging Strategies + +- **Print Statements:** Use print statements to inspect the values of tensors and variables. +- **Debuggers:** Use a debugger (e.g., `pdb`) to step through your code and inspect the state of your application. +- **TensorBoard:** Use TensorBoard to visualize model training progress, including loss curves, metrics, and model architecture. +- **`torch.autograd.set_detect_anomaly(True)`:** A powerful tool for debugging autograd issues. It will raise an error when a NaN gradient is detected, helping you pinpoint the source of the problem. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +- **IDEs:** Visual Studio Code (with Python extension), PyCharm. +- **Virtual Environments:** `venv`, `conda`. +- **Debugging Tools:** `pdb`, `ipdb`. +- **Profiling Tools:** `torch.profiler`, `memory_profiler`. + +### 7.2 Build Configuration + +- **`requirements.txt`:** Specify project dependencies in a `requirements.txt` file. Use `pip freeze > requirements.txt` to generate the file. + + Example `requirements.txt`: + + torch==1.13.1 +torchvision==0.14.1 +numpy==1.24.1 + +- **`setup.py` (for libraries):** Use `setup.py` to define your library's metadata and dependencies. + +### 7.3 Linting and Formatting + +- **Linters:** `flake8`, `pylint`. +- **Formatters:** `black`, `autopep8`. +- **Pre-commit Hooks:** Use pre-commit hooks to automatically run linters and formatters before committing code. + +### 7.4 Deployment Best Practices + +- **Model Serialization:** Use `torch.save` to serialize models for deployment. Use `torch.load` to load models. +- **ONNX Export:** Convert PyTorch models to ONNX format for deployment on various platforms. +- **Serving Frameworks:** Use serving frameworks like TorchServe, FastAPI, or Flask to deploy PyTorch models as REST APIs. + +### 7.5 CI/CD Integration + +- **Continuous Integration (CI):** Use CI tools like Jenkins, GitHub Actions, or GitLab CI to automatically build and test your code. +- **Continuous Deployment (CD):** Use CD tools to automatically deploy your code to production environments. +- **Automated Testing:** Integrate automated testing into your CI/CD pipeline to ensure code quality and prevent regressions. Run unit, integration, and end-to-end tests. \ No newline at end of file diff --git a/.cursor/rules/qwik.mdc b/.cursor/rules/qwik.mdc new file mode 100644 index 000000000..1576024a8 --- /dev/null +++ b/.cursor/rules/qwik.mdc @@ -0,0 +1,204 @@ +--- +description: This rule provides comprehensive best practices for developing Qwik applications, covering code organization, performance, security, testing, and common pitfalls. It aims to guide developers in writing maintainable, efficient, and secure Qwik code. +globs: *.ts?(x) +--- +# Qwik Library Best Practices + +This document outlines the best practices for developing Qwik applications. Adhering to these guidelines will help you write maintainable, efficient, and secure code. + +## 1. Code Organization and Structure + +- **Directory Structure:** + - `src/`: Contains the source code of your application. + - `components/`: Reusable UI components. + - `routes/`: Defines the application's routes and pages. Use the file system routing for easy setup. Each file under `routes` will become a route based on the filename. + - `services/`: Contains business logic and data fetching services. + - `utils/`: Utility functions and helpers. + - `styles/`: Global styles and themes. + - `types/`: Type definitions used throughout the application. + - `public/`: Static assets like images, fonts, and favicons. + - `vite.config.ts`: Vite configuration file. + - `tsconfig.json`: TypeScript configuration file. + - `.eslintrc.js`: ESLint configuration file. + +- **File Naming Conventions:** + - Components: `ComponentName.tsx` (e.g., `MyButton.tsx`). + - Services: `serviceName.ts` (e.g., `userService.ts`). + - Utilities: `utilityName.ts` (e.g., `dateUtils.ts`). + - Styles: `componentName.css` or `componentName.module.css`. + - Routes: Filename dictates the route path (e.g., `index.tsx` for `/`, `about.tsx` for `/about`). + +- **Module Organization:** + - Group related components, services, and utilities into modules. + - Use `index.ts` files to re-export members from a module for cleaner imports. + - Keep modules focused and avoid creating large, monolithic modules. + +- **Component Architecture:** + - Favor small, reusable components. + - Use a component-based architecture for building UIs. + - Separate concerns: presentational components and container components. + - Presentational components: Focus on rendering UI elements based on props. + - Container components: Handle data fetching, state management, and business logic. + - Use Qwik's `component$` and `use*` APIs for creating and managing components. + +- **Code Splitting Strategies:** + - Use Qwik's automatic code splitting capabilities. + - Ensure routes are lazily loaded by their nature in Qwik. + - For larger components or modules, consider manually triggering lazy loading. + +## 2. Common Patterns and Anti-patterns + +- **Design Patterns:** + - **Composition:** Build complex components by composing smaller, reusable components. + - **Provider Pattern:** Use Context to share state or services across multiple components. + - **Hooks:** Use Qwik's `use*` hooks to manage state, effects, and other side effects. + - **Factories:** For complex object creation, consider using factory functions. + +- **Recommended Approaches:** + - Use Qwik's state management features (`useStore`, `useSignal`) for managing component state. + - Use Qwik City's routing capabilities for defining routes and navigation. + - Use Qwik's optimizer and server rendering for performance. + +- **Anti-patterns and Code Smells:** + - **Large Components:** Avoid creating overly large components. Break them down into smaller, more manageable pieces. + - **Tight Coupling:** Reduce dependencies between components and modules. + - **Mutating Props:** Never directly modify props passed to a component. + - **Global State Abuse:** Avoid using global state excessively. Prefer component-level state management. + - **Ignoring Server Rendering:** Ensure your components are server-renderable for initial page load performance. + +- **State Management Best Practices:** + - Use Qwik's `useStore` for reactive state management. + - Consider `useSignal` for simpler reactive values. + - Avoid direct DOM manipulation; rely on Qwik's component rendering. + +- **Error Handling Patterns:** + - Use `try...catch` blocks to handle errors gracefully. + - Display user-friendly error messages. + - Log errors for debugging and monitoring. + - Consider using error boundary components to prevent entire application crashes. + +## 3. Performance Considerations + +- **Optimization Techniques:** + - Use Qwik's optimizer to minimize bundle size and improve performance. + - Leverage server rendering to improve initial page load time. + - Optimize images and other assets. + - Use Qwik's built-in lazy loading features. + +- **Memory Management:** + - Avoid memory leaks by properly cleaning up resources. + - Unsubscribe from subscriptions and event listeners when components unmount. + +- **Rendering Optimization:** + - Use `track` property in `useStore` to selectively trigger updates. + - Memoize expensive computations using `useComputed$()` + - Avoid unnecessary re-renders by ensuring component props are stable. + +- **Bundle Size Optimization:** + - Use code splitting to reduce the initial bundle size. + - Remove unused code and dependencies. + - Use tree shaking to eliminate dead code. + +- **Lazy Loading Strategies:** + - Use Qwik's built-in lazy loading features for components and routes. + - Consider using dynamic imports for loading modules on demand. + +## 4. Security Best Practices + +- **Common Vulnerabilities:** + - **Cross-Site Scripting (XSS):** Prevent XSS attacks by sanitizing user inputs and using Qwik's built-in escaping mechanisms. + - **Cross-Site Request Forgery (CSRF):** Implement CSRF protection by using tokens and validating requests. + - **Injection Attacks:** Prevent SQL injection and other injection attacks by using parameterized queries and escaping user inputs. + +- **Input Validation:** + - Validate all user inputs on both the client and server sides. + - Use appropriate data types and formats. + - Sanitize user inputs to prevent XSS attacks. + +- **Authentication and Authorization Patterns:** + - Use a secure authentication mechanism, such as OAuth or JWT. + - Implement proper authorization checks to ensure users only have access to the resources they are authorized to access. + +- **Data Protection Strategies:** + - Encrypt sensitive data at rest and in transit. + - Use HTTPS to secure communication between the client and server. + - Implement proper access controls to protect data from unauthorized access. + +- **Secure API Communication:** + - Use HTTPS for all API requests. + - Validate API responses. + - Implement rate limiting to prevent abuse. + +## 5. Testing Approaches + +- **Unit Testing Strategies:** + - Write unit tests for individual components, services, and utilities. + - Use a testing framework like Jest or Vitest. + - Mock dependencies to isolate units under test. + +- **Integration Testing:** + - Write integration tests to verify interactions between different parts of the application. + - Test the integration of components, services, and routes. + +- **End-to-end Testing:** + - Write end-to-end tests to verify the entire application flow. + - Use a testing framework like Playwright or Cypress. + - Test user interactions and navigation. + +- **Test Organization:** + - Organize tests into separate directories based on the type of test. + - Use descriptive test names. + - Follow a consistent testing style. + +- **Mocking and Stubbing:** + - Use mocking and stubbing to isolate units under test. + - Mock external dependencies, such as APIs and databases. + +## 6. Common Pitfalls and Gotchas + +- **Frequent Mistakes:** + - Forgetting to use Qwik's `component$` for creating components. + - Not using `use*` hooks for managing state and effects. + - Over-reliance on `globalThis`. + +- **Edge Cases:** + - Handling errors during server rendering. + - Dealing with different browser environments. + - Managing application state across multiple pages. + +- **Version-Specific Issues:** + - Be aware of breaking changes between Qwik versions. + - Consult the Qwik changelog for migration instructions. + +- **Compatibility Concerns:** + - Ensure your application is compatible with different browsers and devices. + - Use polyfills to support older browsers. + +- **Debugging Strategies:** + - Use browser developer tools for debugging. + - Use Qwik's built-in debugging features. + - Log errors and warnings. + +## 7. Tooling and Environment + +- **Recommended Development Tools:** + - Visual Studio Code with the Qwik extension. + - Node.js and npm. + - A modern web browser. + +- **Build Configuration:** + - Use Vite for building and bundling Qwik applications. + - Configure Vite to optimize the build for production. + +- **Linting and Formatting:** + - Use ESLint and Prettier for linting and formatting code. + - Configure ESLint and Prettier to follow Qwik's coding style. + +- **Deployment Best Practices:** + - Deploy your application to a serverless environment, such as Netlify or Vercel. + - Configure your server to serve static assets efficiently. + - Use a CDN to cache static assets. + +- **CI/CD Integration:** + - Integrate your application with a CI/CD pipeline. + - Automate testing, building, and deployment. \ No newline at end of file diff --git a/.cursor/rules/railway.mdc b/.cursor/rules/railway.mdc new file mode 100644 index 000000000..b03c46946 --- /dev/null +++ b/.cursor/rules/railway.mdc @@ -0,0 +1,289 @@ +--- +description: This rule outlines the best practices and coding standards for developing and deploying applications on the Railway platform, covering aspects from code organization to security and performance. +globs: *.js,*.jsx,*.ts,*.tsx,*.py,*.go,*.java,*.rs,*.c,*.cpp,*.cs,*.html,*.css,*.yml,*.yaml,*.json,*.sh +--- +# Railway Best Practices and Coding Standards + +This document provides a comprehensive guide to best practices for developing and deploying applications on the Railway platform. Following these guidelines ensures optimized, secure, and maintainable deployments. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure + +* **Project Root:** Contains the `railway.toml` (or `railway.json`) file defining the project configuration, README, LICENSE, and other top-level files. +* **src/:** Holds the source code of the application. Further divided into modules and components. +* **config/:** Stores configuration files for different environments (development, staging, production). Use environment variables within Railway for sensitive data. +* **scripts/:** Contains utility scripts for build, deployment, and maintenance tasks. +* **tests/:** Includes unit, integration, and end-to-end tests. +* **docs/:** Holds project documentation, API specifications, and other relevant documentation. + +Example: + + +my-railway-app/ +├── railway.toml +├── README.md +├── LICENSE +├── src/ +│ ├── components/ +│ │ ├── Button.js +│ │ └── Input.js +│ ├── modules/ +│ │ ├── auth.js +│ │ └── data.js +│ ├── app.js +│ └── index.js +├── config/ +│ ├── development.js +│ └── production.js +├── scripts/ +│ ├── build.sh +│ └── deploy.sh +├── tests/ +│ ├── unit/ +│ ├── integration/ +│ └── e2e/ +└── docs/ + + +### 1.2. File Naming Conventions + +* **Source Code:** Use descriptive and consistent naming conventions (e.g., `ComponentName.js`, `moduleName.ts`). +* **Configuration Files:** Name configuration files according to their environment (e.g., `development.json`, `production.yml`). +* **Test Files:** Follow a naming convention that clearly identifies the tested component or module (e.g., `ComponentName.test.js`, `moduleName.spec.ts`). + +### 1.3. Module Organization + +* **Functional Grouping:** Organize modules based on their functionality (e.g., authentication, data fetching, UI components). +* **Loose Coupling:** Design modules to be loosely coupled, minimizing dependencies between them. +* **Clear Interfaces:** Define clear and well-documented interfaces for each module. +* **Avoid Circular Dependencies:** Prevent circular dependencies between modules to improve maintainability. + +### 1.4. Component Architecture + +* **Component-Based Approach:** Break down the application into reusable components. +* **Single Responsibility Principle:** Each component should have a single, well-defined responsibility. +* **Presentational and Container Components:** Separate presentational components (UI) from container components (logic and data fetching). +* **Props and State Management:** Use props for passing data down the component tree and manage state appropriately (using libraries like Redux, Zustand or React Context where necessary). + +### 1.5. Code Splitting Strategies + +* **Route-Based Splitting:** Split the application based on routes, loading only the necessary components and modules for each route. +* **Component-Based Splitting:** Split large components into smaller chunks, loading them on demand. +* **Dynamic Imports:** Use dynamic imports (`import()`) to load modules and components asynchronously. +* **Webpack/Parcel Configuration:** Configure your bundler (Webpack, Parcel, etc.) to optimize code splitting and chunking. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +* **Dependency Injection:** Use dependency injection to manage dependencies and improve testability. +* **Observer Pattern:** Implement the observer pattern for handling events and notifications. +* **Singleton Pattern:** Use the singleton pattern sparingly, only when a single instance of a class is truly required. +* **Factory Pattern:** Utilize factory patterns to abstract object creation. + +### 2.2. Recommended Approaches + +* **Environment Variables:** Utilize Railway's environment variables for configuration and secrets management. Access them in your application using appropriate methods for your language (e.g., `process.env` in Node.js, `os.environ` in Python). +* **Private Networking:** Use Railway's private networking to ensure secure and efficient communication between services within the same project and environment using the `RAILWAY_PRIVATE_DOMAIN` environment variable. +* **Reference Variables:** Use Railway's reference variables to dynamically reference other variables, either from a variable set on the current service or from another service in the same project using the `${{serviceName.VARIABLE_NAME}}` syntax, keeping your variable values in sync and avoiding hardcoding. +* **Config as Code:** Utilize Railway's config as code feature to maintain your Railway configuration in a JSON or TOML file, enabling you to keep track of changes just as you do with your source code. + +### 2.3. Anti-patterns + +* **Hardcoding Secrets:** Avoid hardcoding secrets or API keys in your code. Use environment variables instead. +* **Ignoring Errors:** Don't ignore errors or exceptions. Implement proper error handling and logging. +* **Over-Engineering:** Avoid over-engineering solutions. Keep the code simple and maintainable. +* **Tight Coupling:** Minimize dependencies between modules and components. +* **Long-Lived Branches:** Avoid creating long-lived branches. Use feature branches and merge them frequently. + +### 2.4. State Management + +* **Local State:** Use component-level state for simple UI-related state. +* **Context API (React):** Use the Context API for sharing state between components within a subtree. +* **State Management Libraries (Redux, Zustand, Recoil):** Use state management libraries for complex application state and predictable state updates. +* **Immutable Data Structures:** Use immutable data structures to prevent accidental state mutations. + +### 2.5. Error Handling + +* **Try-Catch Blocks:** Use `try-catch` blocks to handle exceptions gracefully. +* **Error Boundaries (React):** Use error boundaries to catch errors during rendering and prevent the entire application from crashing. +* **Centralized Error Handling:** Implement a centralized error handling mechanism for logging errors and displaying user-friendly messages. +* **Asynchronous Error Handling:** Handle errors in asynchronous operations (e.g., Promises) using `.catch()` or `async/await` with `try-catch`. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +* **Code Optimization:** Optimize code for performance (e.g., minimizing DOM manipulations, using efficient algorithms). +* **Caching:** Implement caching mechanisms to reduce server load and improve response times. Use Railway environment variables to configure cache TTLs. +* **Database Optimization:** Optimize database queries and indexing for faster data retrieval. Select database types that optimize well with Railway. Consider a database cluster for production workloads. +* **Image Optimization:** Optimize images for web delivery (e.g., compressing images, using appropriate formats). +* **Gzip Compression:** Enable Gzip compression to reduce the size of transferred data. + +### 3.2. Memory Management + +* **Memory Leaks:** Identify and fix memory leaks in your code. Use memory profiling tools to detect leaks. +* **Garbage Collection:** Understand how garbage collection works in your language and optimize memory usage accordingly. Ensure resources are properly released and terminated. +* **Large Data Structures:** Avoid loading large data structures into memory unnecessarily. Use streaming or pagination techniques. + +### 3.3. Rendering Optimization (If Applicable) + +* **Virtual DOM (React):** Take advantage of the virtual DOM in frameworks like React to minimize DOM updates. +* **Memoization:** Use memoization techniques to prevent unnecessary re-renders. +* **Code Splitting:** Split code into smaller chunks to reduce initial load time. + +### 3.4. Bundle Size Optimization + +* **Dead Code Elimination:** Eliminate dead code and unused dependencies using tools like Webpack's tree shaking. +* **Minification:** Minify code to reduce its size. +* **Dependency Optimization:** Optimize dependencies by using smaller alternatives or by importing only the necessary parts of a library. + +### 3.5. Lazy Loading + +* **Route-Based Lazy Loading:** Lazy load routes to improve initial load time. +* **Component-Based Lazy Loading:** Lazy load components that are not immediately visible or necessary. +* **Intersection Observer:** Use the Intersection Observer API to lazy load images and other resources when they come into view. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities + +* **SQL Injection:** Prevent SQL injection by using parameterized queries or ORM libraries. +* **Cross-Site Scripting (XSS):** Prevent XSS by sanitizing user inputs and escaping outputs. +* **Cross-Site Request Forgery (CSRF):** Prevent CSRF by using anti-CSRF tokens. +* **Authentication and Authorization Vulnerabilities:** Implement secure authentication and authorization mechanisms. +* **Dependency Vulnerabilities:** Keep dependencies up to date to address known vulnerabilities. + +### 4.2. Input Validation + +* **Server-Side Validation:** Validate all user inputs on the server-side. +* **Whitelisting:** Use whitelisting to allow only valid inputs. +* **Sanitization:** Sanitize user inputs to remove potentially harmful characters. +* **Regular Expressions:** Use regular expressions to validate input formats. + +### 4.3. Authentication and Authorization + +* **Secure Password Storage:** Use strong hashing algorithms (e.g., bcrypt, Argon2) to store passwords securely. +* **Multi-Factor Authentication (MFA):** Implement MFA for enhanced security. +* **Role-Based Access Control (RBAC):** Implement RBAC to control access to resources based on user roles. +* **JSON Web Tokens (JWT):** Use JWT for stateless authentication. + +### 4.4. Data Protection + +* **Encryption:** Encrypt sensitive data at rest and in transit. +* **Data Masking:** Mask sensitive data in logs and reports. +* **Regular Backups:** Create regular backups of your data to prevent data loss. Use Railway's volume and backup features. +* **Data Minimization:** Only store the data that is necessary. + +### 4.5. Secure API Communication + +* **HTTPS:** Use HTTPS for all API communication. +* **API Keys:** Protect API keys and other sensitive credentials. +* **Rate Limiting:** Implement rate limiting to prevent abuse. +* **Input Validation:** Validate all API requests. +* **Output Encoding:** Encode API responses to prevent injection attacks. +* **Private Networking:** Use private networking to protect your services from malicious threats by keeping them unexposed to the public network. Secure communication between services in the same project and environment. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +* **Test-Driven Development (TDD):** Write tests before writing code. +* **Component Isolation:** Isolate components during unit testing. +* **Mocking Dependencies:** Mock dependencies to control the behavior of external systems. +* **Code Coverage:** Aim for high code coverage. + +### 5.2. Integration Testing + +* **Integration with External Systems:** Test the integration between different components and external systems. +* **Database Integration:** Test database interactions. +* **API Integration:** Test API endpoints. + +### 5.3. End-to-End Testing + +* **User Flows:** Test complete user flows to ensure that the application works as expected. +* **Browser Automation:** Use browser automation tools (e.g., Selenium, Cypress) to simulate user interactions. +* **Environment Parity:** Run end-to-end tests in an environment that closely resembles production. + +### 5.4. Test Organization + +* **Test Suites:** Organize tests into logical test suites. +* **Test Naming Conventions:** Use consistent test naming conventions. +* **Test Documentation:** Document tests to explain their purpose and expected behavior. + +### 5.5. Mocking and Stubbing + +* **Mocking External Services:** Mock external services (e.g., APIs, databases) to control their behavior during testing. +* **Stubbing Functions:** Stub functions to replace their implementation with a predefined behavior. +* **Mocking Libraries:** Use mocking libraries (e.g., Jest, Mocha) to simplify the mocking process. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +* **Incorrect Environment Variable Configuration:** Ensure that environment variables are correctly configured and accessible in your application. +* **Failing to Configure Restart Policies:** Configure your services' restart policy to mitigate performance degradation and user impact when a service crashes. +* **Not Utilizing Private Networking:** Make sure to use private networking for faster networking along with no egress fees for service-to-service communication within a project. +* **Ignoring Asynchronous Operations:** Handle asynchronous operations correctly to prevent race conditions and unexpected behavior. +* **Overlooking Security Vulnerabilities:** Stay informed about common security vulnerabilities and take steps to mitigate them. + +### 6.2. Edge Cases + +* **Handling Edge Cases:** Consider and handle edge cases in your code. +* **Input Validation:** Validate all user inputs to prevent unexpected behavior. +* **Error Handling:** Handle errors gracefully to prevent application crashes. + +### 6.3. Version-Specific Issues + +* **Compatibility Issues:** Be aware of compatibility issues between different versions of Railway and its dependencies. +* **Breaking Changes:** Stay informed about breaking changes in Railway releases and update your code accordingly. + +### 6.4. Compatibility Concerns + +* **Dependency Conflicts:** Resolve dependency conflicts between Railway and other technologies. +* **Integration Issues:** Address integration issues between Railway and other services. + +### 6.5. Debugging Strategies + +* **Logging:** Use logging to track the execution flow and identify potential issues. +* **Debugging Tools:** Use debugging tools to step through your code and inspect variables. +* **Remote Debugging:** Use remote debugging to debug applications running on Railway servers. +* **Log Explorer:** Familiarize yourself with the Log Explorer so you can query logs across all of your services in one place. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +* **IDE (Integrated Development Environment):** Use a suitable IDE (e.g., VS Code, IntelliJ IDEA) with appropriate extensions for your language. +* **CLI (Command-Line Interface):** Use the Railway CLI for managing your Railway projects and services. +* **Version Control System (Git):** Use Git for version control. + +### 7.2. Build Configuration + +* **Build Scripts:** Define build scripts in your `package.json` (Node.js) or equivalent file. +* **Dependency Management:** Use a dependency management tool (e.g., npm, yarn, pip, maven) to manage dependencies. +* **Configuration Management:** Use configuration management tools to manage configuration files across different environments. + +### 7.3. Linting and Formatting + +* **Linters:** Use linters (e.g., ESLint, Pylint) to enforce code style and identify potential errors. +* **Formatters:** Use formatters (e.g., Prettier, Black) to automatically format your code. +* **Editor Integration:** Integrate linters and formatters with your editor to provide real-time feedback. + +### 7.4. Deployment Best Practices + +* **Automated Deployments:** Automate the deployment process using CI/CD pipelines. +* **Zero-Downtime Deployments:** Implement zero-downtime deployments to minimize downtime during deployments. (Consider using blue-green deployments or rolling updates). +* **Rollbacks:** Implement a rollback strategy to quickly revert to a previous deployment in case of issues. Be sure to check out the deployment rollback feature, in case you need to rollback to a previous deployment. +* **Check Suites:** Enable check suites to have Railway wait for your GitHub workflows to complete successfully before triggering a deployment. + +### 7.5. CI/CD Integration + +* **CI/CD Pipelines:** Set up CI/CD pipelines to automatically build, test, and deploy your application. +* **GitHub Actions:** Use GitHub Actions for CI/CD. +* **GitLab CI:** Use GitLab CI for CI/CD. +* **Environment Variables:** Use environment variables to configure the CI/CD pipeline. +* **Webhooks and Email Notifications:** Setup webhooks and email notifications to be alerted if the deployment status of your services change. + +By adhering to these best practices, you can develop and deploy robust, scalable, and secure applications on the Railway platform. \ No newline at end of file diff --git a/.cursor/rules/react-mobx.mdc b/.cursor/rules/react-mobx.mdc new file mode 100644 index 000000000..3bb5d50c7 --- /dev/null +++ b/.cursor/rules/react-mobx.mdc @@ -0,0 +1,203 @@ +--- +description: This rule provides comprehensive best practices for developing React applications with Mobx, covering code organization, performance, testing, and security considerations. It aims to guide developers in building robust and maintainable applications using React-Mobx. +globs: *.js,*.jsx,*.ts,*.tsx +--- +# React-Mobx Best Practices + +This document outlines best practices for developing React applications with Mobx. Following these guidelines will help you build robust, maintainable, and performant applications. + +## 1. Core Principles + +- **Model observable state:** Define the source of truth in your Mobx stores and make them observable. +- **Use derived state (computed properties):** Derive data from your observable state using `computed` properties to avoid unnecessary re-renders and ensure data consistency. +- **Embrace mutability:** Mobx encourages direct mutation of observable state within actions. +- **Use strict mode:** Enforce predictable state changes by using `use strict` or enabling strict mode in Mobx to ensure all state modifications happen within actions. +- **Keep actions in MobX stores:** Encapsulate state mutations within Mobx actions to ensure data integrity and simplify debugging. +- **Prefer class vs object syntax:** Use class syntax for creating stores as it offers better structure, inheritance, and maintainability. +- **Inject store rather than importing:** Use dependency injection (e.g., React Context) to provide stores to components instead of importing them directly to improve testability and reduce tight coupling. + +## 2. Code Organization and Structure + +- **Directory Structure:** + - `/src`: + - `/components`: React components (presentational and container components). + - `/stores`: Mobx stores containing observable state and actions. + - `/models`: Data models and types definitions. + - `/services`: API clients and other services. + - `/utils`: Utility functions. + - `/hooks`: Custom React hooks. + - `/constants`: Application-wide constants. + - `/test`: Unit, integration, and end-to-end tests. + +- **File Naming Conventions:** + - Components: `ComponentName.jsx` or `ComponentName.tsx` + - Stores: `StoreName.store.js` or `StoreName.store.ts` + - Models: `ModelName.model.js` or `ModelName.model.ts` + - Services: `ServiceName.service.js` or `ServiceName.service.ts` + - Hooks: `useHookName.js` or `useHookName.ts` + +- **Module Organization:** + - Group related components, stores, and models into modules. + - Use clear and descriptive module names. + - Avoid circular dependencies between modules. + +- **Component Architecture:** + - **Presentational Components:** Responsible for rendering UI and receiving data via props. + - **Container Components:** Connect presentational components to Mobx stores and manage state updates. + - Use the `observer` decorator/function from `mobx-react-lite` to make components react to observable changes. + +- **Code Splitting Strategies:** + - Use React's `lazy` and `Suspense` components for lazy loading routes or components. + - Implement route-based code splitting to load only necessary code for the current route. + - Consider using `dynamic imports` for on-demand loading of modules. + +## 3. Common Patterns and Anti-patterns + +- **Design Patterns:** + - **Observer Pattern:** Mobx uses the observer pattern to automatically update components when observable state changes. + - **Dependency Injection:** Use React Context to inject stores into components, making them more testable and reusable. + - **Provider Pattern:** Use a Provider component to make stores available to the component tree. + +- **Recommended Approaches:** + - **Form Handling:** Use Mobx to manage form state and validation. + - **Asynchronous Operations:** Use Mobx actions to handle asynchronous operations and update the state accordingly. + - **List Rendering:** Optimize list rendering using `key` props and `React.memo`. + +- **Anti-patterns and Code Smells:** + - **Modifying State Outside Actions:** Always modify observable state within Mobx actions to ensure data consistency. + - **Deeply Nested Observables:** Avoid deeply nested observable objects as they can impact performance. + - **Computing Data in Components:** Use `computed` properties in stores to derive data instead of computing it directly in components. + - **Over-Observing:** Only observe the specific parts of the store that a component needs. + +- **State Management Best Practices:** + - **Single Source of Truth:** Keep the application state in Mobx stores and avoid duplicating data. + - **Normalization:** Normalize data to reduce redundancy and improve data consistency. + - **Immutability (with Mutation):** While Mobx encourages mutation, treat observable state as immutable from the perspective of components (i.e., components shouldn't directly mutate the store). + +- **Error Handling Patterns:** + - Implement centralized error handling using a dedicated error store. + - Use try-catch blocks within actions to handle potential errors. + - Display user-friendly error messages to the user. + +## 4. Performance Considerations + +- **Optimization Techniques:** + - **`React.memo`:** Wrap components with `React.memo` to prevent unnecessary re-renders when props haven't changed. + - **`useMemo` and `useCallback`:** Use `useMemo` and `useCallback` hooks to memoize values and callbacks, preventing unnecessary re-renders of child components. + - **Computed Properties:** Use `computed` properties to derive data from observable state and cache the results. + - **Optimize List Rendering:** Use `key` props when rendering lists and consider using virtualized lists for large datasets. + +- **Memory Management:** + - **Dispose of Observers:** Dispose of observers when components unmount to prevent memory leaks. + - **Avoid Retaining Large Data Sets:** Remove or release references to large data sets when they are no longer needed. + +- **Rendering Optimization:** + - **Minimize Re-renders:** Optimize component rendering by using `React.memo` and avoiding unnecessary state updates. + - **Use ShouldComponentUpdate (if needed):** In class components, use `shouldComponentUpdate` lifecycle method to control when a component should re-render. This is less common with `observer`. Be careful, can lead to bugs. + +- **Bundle Size Optimization:** + - **Code Splitting:** Implement code splitting to reduce the initial bundle size. + - **Tree Shaking:** Use a bundler that supports tree shaking to remove unused code. + - **Minification:** Minify your code to reduce the bundle size. + +- **Lazy Loading Strategies:** + - **React.lazy and Suspense:** Use React's built-in lazy loading mechanism. + - **Dynamic Imports:** Use dynamic imports to load modules on demand. + +## 5. Security Best Practices + +- **Common Vulnerabilities:** + - **Cross-Site Scripting (XSS):** Sanitize user inputs to prevent XSS attacks. + - **Cross-Site Request Forgery (CSRF):** Implement CSRF protection mechanisms. + - **Injection Attacks:** Validate user inputs to prevent SQL injection and other injection attacks. + +- **Input Validation:** + - **Server-Side Validation:** Always validate user inputs on the server-side. + - **Client-Side Validation:** Implement client-side validation to provide immediate feedback to the user. + +- **Authentication and Authorization Patterns:** + - **JSON Web Tokens (JWT):** Use JWTs for authentication and authorization. + - **Role-Based Access Control (RBAC):** Implement RBAC to control access to different parts of the application. + +- **Data Protection Strategies:** + - **Encryption:** Encrypt sensitive data both in transit and at rest. + - **Data Masking:** Mask sensitive data to protect user privacy. + +- **Secure API Communication:** + - **HTTPS:** Use HTTPS for all API communication. + - **Rate Limiting:** Implement rate limiting to prevent abuse. + +## 6. Testing Approaches + +- **Unit Testing Strategies:** + - Test individual components and stores in isolation. + - Mock dependencies to isolate the unit under test. + - Use testing libraries like Jest and React Testing Library. + +- **Integration Testing:** + - Test the interaction between different components and stores. + - Use a testing environment that closely resembles the production environment. + +- **End-to-End Testing:** + - Test the entire application flow from the user's perspective. + - Use testing frameworks like Cypress or Selenium. + +- **Test Organization:** + - Organize tests into separate directories for unit, integration, and end-to-end tests. + - Use descriptive test names. + +- **Mocking and Stubbing:** + - Use mocking libraries to mock dependencies in unit tests. + - Use stubbing to replace complex dependencies with simpler implementations. + +## 7. Common Pitfalls and Gotchas + +- **Frequent Mistakes:** + - Forgetting to decorate components with `observer`. + - Modifying state outside of actions. + - Not disposing of observers properly. + - Over-observing state. + +- **Edge Cases:** + - Handling large datasets efficiently. + - Dealing with complex asynchronous operations. + - Managing nested observable objects. + +- **Version-Specific Issues:** + - Be aware of breaking changes in Mobx and React versions. + - Consult the Mobx and React documentation for migration guides. + +- **Compatibility Concerns:** + - Ensure compatibility with different browsers and devices. + - Test your application on different platforms. + +- **Debugging Strategies:** + - Use the Mobx devtools to inspect observable state and track changes. + - Use browser developer tools to debug React components. + - Add logging statements to track the flow of data and execution. + +## 8. Tooling and Environment + +- **Recommended Development Tools:** + - **VS Code:** A popular code editor with excellent support for React and Mobx. + - **Mobx Devtools:** A browser extension for inspecting Mobx state. + - **React Devtools:** A browser extension for inspecting React components. + +- **Build Configuration:** + - **Webpack:** A popular module bundler for React applications. + - **Parcel:** A zero-configuration bundler that's easy to use. + - **Rollup:** A bundler focused on creating libraries and smaller bundles. + +- **Linting and Formatting:** + - **ESLint:** A JavaScript linter for enforcing code style and preventing errors. + - **Prettier:** A code formatter for automatically formatting code. + - **Husky/lint-staged:** Tools to automatically lint code before commits + +- **Deployment Best Practices:** + - **Continuous Integration/Continuous Deployment (CI/CD):** Automate the build, test, and deployment process. + - **Caching:** Use caching to improve performance. + - **Content Delivery Network (CDN):** Use a CDN to serve static assets. + +- **CI/CD Integration:** + - Integrate your application with a CI/CD pipeline to automate the build, test, and deployment process. + - Use tools like Jenkins, Travis CI, or CircleCI. \ No newline at end of file diff --git a/.cursor/rules/react-native.mdc b/.cursor/rules/react-native.mdc new file mode 100644 index 000000000..52576a8c2 --- /dev/null +++ b/.cursor/rules/react-native.mdc @@ -0,0 +1,217 @@ +--- +description: This rule provides comprehensive best practices and coding standards for React Native development, covering code organization, performance, security, testing, and common pitfalls. +globs: *.js,*.jsx,*.ts,*.tsx +--- +- Use TypeScript for type safety and improved code maintainability. +- Prefer functional components with hooks over class components for simplicity and reusability. +- Maintain a clear and consistent project structure for scalability and maintainability. + +## 1. Code Organization and Structure: + +- **Directory Structure:** + - Adopt a feature-based or component-based structure. For example: + + src/ + ├── components/ + │ ├── Button/ + │ │ ├── Button.tsx + │ │ ├── Button.styles.ts + │ │ ├── Button.test.tsx + │ ├── Input/ + │ └── ... + ├── screens/ + │ ├── Home/ + │ │ ├── HomeScreen.tsx + │ │ ├── HomeScreen.styles.ts + │ │ ├── HomeScreen.test.tsx + │ ├── Profile/ + │ └── ... + ├── navigation/ + ├── services/ + ├── utils/ + ├── types/ + └── App.tsx + + - Separate concerns into distinct directories (e.g., components, screens, navigation, services, utils, types). + +- **File Naming Conventions:** + - Use descriptive names for files and components (e.g., `HomeScreen.tsx`, `useUserData.ts`). + - Follow a consistent naming convention (e.g., PascalCase for components, camelCase for functions and variables). + - Use `.styles.ts` for style files and `.test.tsx` for test files to keep components readable and organized. + +- **Module Organization:** + - Group related components and modules into logical units. + - Use absolute imports and module aliases to avoid long relative paths (e.g., `@components/Button` instead of `../../../components/Button`). + - Configure module aliases in `tsconfig.json` or `jsconfig.json`. + +- **Component Architecture:** + - Favor small, reusable components with a single responsibility (Single Responsibility Principle). + - Use composition over inheritance to create complex components. + - Consider using a UI library like React Native Paper or NativeBase for pre-built components. + +- **Code Splitting Strategies:** + - Implement lazy loading for screens or components that are not immediately needed. + - Use `React.lazy` and `Suspense` to load components on demand. + - Utilize dynamic imports for conditional loading of modules. + +## 2. Common Patterns and Anti-patterns: + +- **Design Patterns:** + - **Higher-Order Components (HOCs):** Use HOCs for cross-cutting concerns like authentication or logging. + - **Render Props:** Use render props to share code between React components. + - **Hooks:** Use custom hooks to encapsulate logic and stateful behavior. + +- **Recommended Approaches:** + - **State Management:** + - Use React Context for simple state management. + - Use Redux, Zustand, or Jotai for complex state management. + - Consider using Recoil for fine-grained state management. + - **API Calls:** + - Use `axios` or `fetch` for making API requests. + - Create a service layer to handle API calls and data transformations. + - **Navigation:** + - Use React Navigation for managing app navigation. + - Define navigation stacks and routes in a separate module. + +- **Anti-patterns and Code Smells:** + - **Long Component Files:** Break down large components into smaller, more manageable pieces. + - **Deeply Nested Components:** Avoid excessive nesting, which can impact performance. + - **Mutating State Directly:** Always use `setState` or a state management library to update state. + - **Unnecessary Re-renders:** Optimize components to prevent unnecessary re-renders. + - **Global Styles:** Avoid using global styles, as they can lead to conflicts and make it difficult to maintain the application. + +- **State Management Best Practices:** + - Choose a state management solution that fits the complexity of your application. + - Keep state minimal and derive values when possible. + - Use selectors to access state and memoize computed values. + +- **Error Handling Patterns:** + - Use `try...catch` blocks to handle errors gracefully. + - Implement a global error handler to catch unhandled exceptions. + - Log errors to a remote monitoring service. + +## 3. Performance Considerations: + +- **Optimization Techniques:** + - **Memoization:** Use `React.memo` to memoize components and prevent unnecessary re-renders. + - **Pure Components:** Extend `React.PureComponent` for components that only depend on props. + - **Debouncing and Throttling:** Use debouncing and throttling to limit the frequency of function calls. + - **Virtualization:** Use `FlatList` or `SectionList` for rendering large lists of data. + +- **Memory Management:** + - Avoid memory leaks by properly cleaning up event listeners and timers. + - Use `useCallback` and `useMemo` to prevent creating new functions and objects on every render. + +- **Rendering Optimization:** + - Minimize the number of re-renders by optimizing component updates. + - Use `shouldComponentUpdate` (for class components) or `React.memo` to control re-renders. + - Avoid using inline styles, as they are re-created on every render. + +- **Bundle Size Optimization:** + - Use code splitting to reduce the initial bundle size. + - Remove unused code and dependencies. + - Use a bundler like Metro or Webpack with tree shaking enabled. + - Compress images and other assets. + +- **Lazy Loading Strategies:** + - Implement lazy loading for images and other assets using `React.lazy` and `Suspense`. + - Use dynamic imports to load modules on demand. + +## 4. Security Best Practices: + +- **Common Vulnerabilities:** + - **Cross-Site Scripting (XSS):** Sanitize user input to prevent XSS attacks. + - **SQL Injection:** Use parameterized queries to prevent SQL injection attacks. + - **Cross-Site Request Forgery (CSRF):** Implement CSRF protection tokens. + - **Man-in-the-Middle (MITM) Attacks:** Use HTTPS to encrypt communication. + +- **Input Validation:** + - Validate user input on both the client and server sides. + - Use regular expressions or validation libraries to enforce input constraints. + +- **Authentication and Authorization Patterns:** + - Use a secure authentication protocol like OAuth 2.0 or JWT. + - Implement role-based access control (RBAC) to restrict access to sensitive resources. + +- **Data Protection Strategies:** + - Encrypt sensitive data at rest and in transit. + - Use secure storage mechanisms for storing API keys and other secrets. + +- **Secure API Communication:** + - Use HTTPS for all API communication. + - Implement API rate limiting to prevent abuse. + - Validate API responses to prevent data injection attacks. + +## 5. Testing Approaches: + +- **Unit Testing Strategies:** + - Write unit tests for individual components and modules. + - Use a testing framework like Jest or Mocha. + - Mock dependencies to isolate the component being tested. + +- **Integration Testing:** + - Write integration tests to verify the interaction between components and modules. + - Test the integration with external APIs and services. + +- **End-to-End Testing:** + - Write end-to-end tests to verify the entire application flow. + - Use a testing framework like Detox or Appium. + +- **Test Organization:** + - Organize tests into separate directories based on component or module. + - Use descriptive names for test files and test cases. + +- **Mocking and Stubbing:** + - Use mocking and stubbing to isolate components and control their behavior during testing. + - Use a mocking library like Jest or Sinon. + +## 6. Common Pitfalls and Gotchas: + +- **Frequent Mistakes:** + - **Directly Mutating State:** Always use `setState` or a state management library to update state. + - **Ignoring Platform Differences:** Test your application on both iOS and Android devices. + - **Over-Optimizing:** Optimize only when necessary, as premature optimization can lead to complex code. + - **Not Using a Debugger:** Utilize the React Native debugger for efficient debugging. + +- **Edge Cases:** + - **Handling Device Orientation Changes:** Implement logic to handle device orientation changes gracefully. + - **Handling Network Connectivity Issues:** Implement error handling for network connectivity issues. + - **Handling Different Screen Sizes and Densities:** Design your UI to adapt to different screen sizes and densities. + +- **Version-Specific Issues:** + - Be aware of breaking changes in React Native and its dependencies. + - Test your application with different versions of React Native. + +- **Compatibility Concerns:** + - Ensure that your application is compatible with the target operating systems and devices. + - Use polyfills to support older browsers and devices. + +- **Debugging Strategies:** + - Use the React Native debugger to inspect the component tree and state. + - Use the console to log messages and debug code. + - Use a remote debugging tool to debug on a physical device. + +## 7. Tooling and Environment: + +- **Recommended Development Tools:** + - **VS Code:** Use VS Code with the React Native Tools extension for debugging and code completion. + - **React Native CLI:** Use the React Native CLI for creating and managing React Native projects. + - **Expo CLI:** Use the Expo CLI for developing and testing React Native applications without native code. + - **Android Studio:** Use Android Studio for building and debugging Android applications. + - **Xcode:** Use Xcode for building and debugging iOS applications. + +- **Build Configuration:** + - Use a build system like Gradle (Android) or CocoaPods (iOS) to manage dependencies. + - Configure build variants for different environments (e.g., development, staging, production). + +- **Linting and Formatting:** + - Use ESLint and Prettier to enforce code style and catch potential errors. + - Configure ESLint and Prettier to automatically format code on save. + +- **Deployment Best Practices:** + - Use a continuous integration and continuous deployment (CI/CD) pipeline to automate the deployment process. + - Use a service like App Center or Bitrise for building and deploying React Native applications. + +- **CI/CD Integration:** + - Integrate your code repository with a CI/CD service like GitHub Actions or CircleCI. + - Configure the CI/CD pipeline to run tests and build the application on every commit. \ No newline at end of file diff --git a/.cursor/rules/react-query.mdc b/.cursor/rules/react-query.mdc new file mode 100644 index 000000000..760d1d255 --- /dev/null +++ b/.cursor/rules/react-query.mdc @@ -0,0 +1,267 @@ +--- +description: This rule enforces best practices for using react-query in React applications, covering code organization, performance, security, and testing. +globs: *.js,*.jsx,*.ts,*.tsx +--- +# react-query Best Practices + +This document outlines the best practices for using react-query in React applications, covering various aspects such as code organization, performance considerations, security, and testing. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure Best Practices + +* **Feature-based Organization:** Group react-query hooks and related components within feature-specific directories. This improves modularity and maintainability. + + src/ + ├── features/ + │ ├── users/ + │ │ ├── components/ + │ │ │ ├── UserList.tsx + │ │ │ └── UserDetails.tsx + │ │ ├── hooks/ + │ │ │ ├── useUsersQuery.ts + │ │ │ └── useCreateUserMutation.ts + │ │ ├── api/ + │ │ │ └── usersApi.ts + │ │ └── types/ + │ │ └── user.ts + │ ├── products/ + │ └── ... + ├── ... + + +* **Dedicated API Service Layer:** Abstract API interaction logic into separate modules. This allows for easier testing and decoupling of components from specific API implementations. Consider using a dedicated `api` directory within each feature. + +### 1.2 File Naming Conventions + +* **Consistent Naming:** Follow a consistent naming convention for react-query hooks. Prefix hooks with `use` and postfix with `Query` or `Mutation` to clearly indicate their purpose (e.g., `usePostsQuery`, `useUpdatePostMutation`). + +* **Descriptive Names:** Use descriptive names for files and variables to improve code readability. For example, `useFetchUsersQuery.ts` is more informative than `useUsers.ts`. + +### 1.3 Module Organization + +* **Custom Hooks for Reusability:** Encapsulate react-query logic within custom hooks to promote reusability and separation of concerns. + + typescript + // src/features/users/hooks/useUsersQuery.ts + import { useQuery } from '@tanstack/react-query'; + import { fetchUsers } from '../api/usersApi'; + + export const useUsersQuery = () => { + return useQuery('users', fetchUsers); + }; + + +* **Separate Query and Mutation Files:** Organize queries and mutations into separate files or modules. This enhances code readability and maintainability. + +### 1.4 Component Architecture + +* **Presentational and Container Components:** Separate presentational components (UI) from container components (data fetching and state management). This improves testability and reusability. + +* **Composition:** Use component composition to build complex UIs from smaller, reusable components. Leverage React Context for shared state when appropriate. + +### 1.5 Code Splitting Strategies + +* **Route-Based Splitting:** Split your application into separate bundles based on routes. This reduces the initial load time and improves perceived performance. React.lazy and React.Suspense can assist with this. + +* **Component-Based Splitting:** Split large components into smaller chunks that can be loaded on demand. This can improve the performance of individual pages or components. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns Specific to react-query + +* **Custom Hooks for Data Fetching:** As highlighted earlier, encapsulating react-query logic within custom hooks. This promotes reusability and separation of concerns. These hooks will typically return the result of a `useQuery` or `useMutation` call. + +* **Optimistic Updates:** Implement optimistic updates to improve perceived performance. This involves updating the UI before the API request completes, and then reverting the changes if the request fails. react-query provides utilities like `onMutate` to handle this. + +* **Pessimistic Updates:** Update the UI only after a successful response from the API. Simpler to implement but provides a less snappy user experience. + +### 2.2 Recommended Approaches for Common Tasks + +* **Prefetching Data:** Prefetch data for routes or components that the user is likely to visit next. This can significantly improve the user experience. Use `queryClient.prefetchQuery`. + +* **Pagination and Infinite Scrolling:** Implement pagination and infinite scrolling to handle large datasets efficiently. react-query provides hooks like `useInfiniteQuery` to simplify this. + +* **Dependent Queries:** Fetch data based on the result of a previous query. Use the `enabled` option in `useQuery` to conditionally execute queries. + + typescript + const { data: user } = useQuery(['user', userId], () => fetchUser(userId)); + + const { data: posts } = useQuery(['posts', user?.id], () => fetchPosts(user.id), { + enabled: !!user, + }); + + +### 2.3 Anti-patterns and Code Smells to Avoid + +* **Directly Calling API in Components:** Avoid making API calls directly within components. This makes testing difficult and tightly couples components to specific API implementations. + +* **Ignoring Error Handling:** Always handle errors properly. Display user-friendly error messages and provide options for retrying requests. + +* **Over-fetching Data:** Fetch only the data that is required by the component. Use GraphQL or API query parameters to reduce the amount of data transferred. + +* **Deeply Nested Queries:** Avoid deeply nesting queries, as this can lead to performance issues and make the code difficult to understand. Consider combining queries or using a different approach. + +### 2.4 State Management Best Practices + +* **Local vs. Global State:** Determine whether data should be stored in local component state or in a global state management solution. Use local state for component-specific data and global state for data that needs to be shared across multiple components. + +* **react-query as a State Manager:** Leverage react-query's built-in caching and state management capabilities. Avoid using external state management libraries for data that is already managed by react-query. + +### 2.5 Error Handling Patterns + +* **Centralized Error Handling:** Implement centralized error handling to provide consistent error messages and logging. + +* **Retry Logic:** Implement retry logic to automatically retry failed requests. react-query provides options for configuring retry behavior. + +* **Error Boundaries:** Use Error Boundaries to catch errors that occur during rendering. This prevents the entire application from crashing. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +* **Query Invalidation:** Invalidate queries when data changes. This ensures that the UI is always up-to-date. + +* **Stale-While-Revalidate:** Use the `staleTime` and `cacheTime` options to configure how long data should be considered fresh. `staleWhileRevalidate` allows the UI to display cached data while fetching fresh data in the background. + +* **Window Focus Refetching:** Configure refetching on window focus to keep data fresh when the user switches back to the application. + +* **Polling/Refetch Intervals:** Use `refetchInterval` to periodically refetch data. This is useful for data that changes frequently. + +### 3.2 Memory Management + +* **Query Cache Management:** Understand how react-query manages its cache. Configure the `cacheTime` option to control how long data is stored in the cache. + +* **Garbage Collection:** Ensure that unused queries are garbage collected properly. Use the `gcTime` option to configure how long inactive queries should be kept in the cache. + +### 3.3 Rendering Optimization + +* **Memoization:** Use `React.memo` to prevent unnecessary re-renders of components. This is especially important for components that display data fetched from react-query. + +* **Virtualization:** Use virtualization techniques (e.g., `react-window`, `react-virtualized`) to efficiently render large lists of data. + +### 3.4 Bundle Size Optimization + +* **Tree Shaking:** Ensure that your build process is configured for tree shaking. This removes unused code from the final bundle. + +* **Code Splitting:** As mentioned earlier, use code splitting to reduce the initial load time. + +### 3.5 Lazy Loading Strategies + +* **Lazy Load Components:** Use `React.lazy` to lazy load components that are not immediately needed. + +* **Lazy Load Data:** Fetch data only when it is needed. Use dependent queries to fetch data based on user interactions. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and How to Prevent Them + +* **Cross-Site Scripting (XSS):** Sanitize user input to prevent XSS attacks. Use a library like DOMPurify to sanitize HTML. + +* **Cross-Site Request Forgery (CSRF):** Implement CSRF protection to prevent attackers from performing actions on behalf of the user. Use a library or framework that provides CSRF protection. + +* **Injection Attacks:** Protect against injection attacks by validating user input and using parameterized queries. + +### 4.2 Input Validation + +* **Client-Side Validation:** Implement client-side validation to provide immediate feedback to the user. + +* **Server-Side Validation:** Always validate user input on the server-side to prevent malicious data from being stored in the database. + +### 4.3 Authentication and Authorization Patterns + +* **JSON Web Tokens (JWT):** Use JWTs for authentication. Store the JWT in a secure cookie or in local storage (with caution). Use `httpOnly` flag on cookies containing JWTs when possible to prevent client-side script access. + +* **Role-Based Access Control (RBAC):** Implement RBAC to control access to different parts of the application. Use middleware or custom hooks to check user roles. + +### 4.4 Data Protection Strategies + +* **Encryption:** Encrypt sensitive data at rest and in transit. Use HTTPS to encrypt data in transit. Encrypt sensitive data in the database. + +* **Data Masking:** Mask sensitive data in logs and reports. This prevents sensitive data from being exposed to unauthorized users. + +### 4.5 Secure API Communication + +* **HTTPS:** Use HTTPS for all API communication. This encrypts data in transit and prevents eavesdropping. + +* **API Rate Limiting:** Implement API rate limiting to prevent abuse. + +* **CORS:** Configure CORS properly to prevent cross-origin requests from unauthorized domains. + +## 5. Testing Approaches + +### 5.1 Unit Testing Strategies + +* **Test Custom Hooks:** Unit test custom react-query hooks to ensure they are working correctly. Mock the API calls using libraries like `msw` (Mock Service Worker). + +* **Test Components in Isolation:** Unit test components in isolation to ensure they render correctly and handle user interactions properly. Use libraries like `react-testing-library`. + +### 5.2 Integration Testing + +* **Test Data Flow:** Integration test the data flow between components and APIs. Verify that data is fetched correctly and displayed properly. + +* **Test Error Handling:** Integration test error handling scenarios to ensure that errors are handled properly. + +### 5.3 End-to-End Testing + +* **Simulate User Interactions:** Use end-to-end testing frameworks like Cypress or Playwright to simulate user interactions and verify that the application is working correctly from the user's perspective. + +* **Test Critical Paths:** Focus on testing critical user flows, such as login, registration, and checkout. + +### 5.4 Test Organization + +* **Colocate Tests with Components:** Colocate tests with the components they are testing. This makes it easier to find and maintain tests. + +* **Use Descriptive Test Names:** Use descriptive test names to clearly indicate what each test is verifying. + +### 5.5 Mocking and Stubbing + +* **Mock API Calls:** Use mocking libraries like `msw` to mock API calls during testing. This allows you to test components in isolation without relying on a real API. + +* **Stub External Dependencies:** Stub external dependencies to isolate components and make tests more predictable. + +## 6. Common Pitfalls and Gotchas + +* **Forgetting to Invalidate Queries:** Failing to invalidate queries when data changes can lead to stale data being displayed in the UI. + +* **Incorrect Cache Configuration:** Incorrectly configuring the `cacheTime` and `staleTime` options can lead to performance issues or stale data. + +* **Not Handling Errors Properly:** Not handling errors properly can lead to unexpected behavior and a poor user experience. + +* **Over-relying on Default Configuration:** Customizing react-query to match specific needs is essential. + +* **Ignoring Devtools:** The react-query devtools are invaluable for debugging and understanding what is happening under the hood. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +* **VS Code:** Use VS Code with extensions like ESLint, Prettier, and TypeScript to improve developer productivity. + +* **React Developer Tools:** Use the React Developer Tools browser extension to inspect React components and state. + +* **react-query Devtools:** Use the react-query Devtools to inspect the react-query cache and track query and mutation status. + +### 7.2 Build Configuration + +* **Webpack or Parcel:** Use a bundler like Webpack or Parcel to bundle your code for production. Configure the bundler for tree shaking and code splitting. + +* **Babel:** Use Babel to transpile your code to older versions of JavaScript. This ensures that your code is compatible with older browsers. + +### 7.3 Linting and Formatting + +* **ESLint:** Use ESLint to enforce coding standards and prevent errors. Configure ESLint to use a popular style guide like Airbnb or Google. + +* **Prettier:** Use Prettier to automatically format your code. This ensures that your code is consistently formatted and easy to read. + +### 7.4 Deployment Best Practices + +* **CDN:** Use a CDN to serve static assets. This improves performance and reduces the load on your server. + +* **Caching:** Configure caching properly on your server and CDN. This reduces the number of requests to your server and improves performance. + +### 7.5 CI/CD Integration + +* **Automated Testing:** Integrate automated testing into your CI/CD pipeline. This ensures that your code is tested automatically before it is deployed. + +* **Automated Deployment:** Automate the deployment process to reduce the risk of errors and improve efficiency. \ No newline at end of file diff --git a/.cursor/rules/react-redux.mdc b/.cursor/rules/react-redux.mdc new file mode 100644 index 000000000..4fa04be84 --- /dev/null +++ b/.cursor/rules/react-redux.mdc @@ -0,0 +1,283 @@ +--- +description: Enforces best practices for structuring and maintaining React-Redux applications, focusing on code organization, performance, and maintainability. This rule provides guidelines for developers to write efficient, scalable, and testable React-Redux code. +globs: *.js,*.jsx,*.ts,*.tsx +--- +# React-Redux Best Practices + +This document outlines best practices for developing React applications with Redux for state management. Following these guidelines will help you write maintainable, scalable, and performant code. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure + +Organize your code around features rather than technical concerns. A feature-based directory structure promotes modularity and makes it easier to understand the application's purpose. + + +src/ + ├── app/ + │ ├── components/ + │ │ └── App.tsx + │ └── App.css + ├── features/ + │ ├── featureName/ + │ │ ├── components/ + │ │ │ ├── FeatureComponent.tsx + │ │ │ └── FeatureComponent.css + │ │ ├── FeatureSlice.ts # Redux slice for the feature + │ │ ├── FeatureSelectors.ts # Selectors for accessing feature state + │ │ ├── FeatureActions.ts # Actions related to this feature + │ │ └── api.ts # API calls related to the feature + │ ├── anotherFeature/ + │ │ └── ... + ├── store.ts # Redux store configuration + ├── index.tsx # Entry point for the React application + └── ... + + +### 1.2. File Naming Conventions + +* **Components:** Use PascalCase (e.g., `MyComponent.tsx`). +* **Redux Slice Files:** Use PascalCase and end with `Slice.ts` or `Slice.js` (e.g., `UserSlice.ts`). +* **Selectors:** `selectors.ts` or `selectors.js` or `FeatureSelectors.ts` to include a feature name prefix +* **Actions:** `actions.ts` or `actions.js` or `FeatureActions.ts` to include a feature name prefix +* **Styles:** Use either `ComponentName.module.css` (for CSS Modules) or `ComponentName.css` (for global styles). +* **Utility Functions:** Use camelCase (e.g., `formatDate.ts`). + +### 1.3. Module Organization + +* **Encapsulation:** Each feature should be a self-contained module with its components, actions, reducers, and selectors. +* **Single Responsibility Principle:** Each module should have a single, well-defined purpose. +* **Clear Boundaries:** Define clear boundaries between modules to minimize dependencies and prevent tight coupling. + +### 1.4. Component Architecture + +* **Presentational and Container Components:** + * **Presentational (Dumb) Components:** Focus on rendering UI and receiving data and callbacks as props. They should be reusable and independent of Redux. + * **Container (Smart) Components:** Connect to the Redux store and pass data and actions to presentational components. Use `connect` or `useSelector` and `useDispatch` hooks. +* **Small and Reusable Components:** Break down complex UIs into smaller, reusable components to promote code reuse and maintainability. +* **Component Composition:** Build complex UIs by composing smaller components. + +### 1.5. Code Splitting Strategies + +* **Route-Based Splitting:** Split the application into chunks based on routes to reduce the initial load time. Use `React.lazy` and `Suspense` for lazy loading components. +* **Component-Based Splitting:** Split large components into smaller chunks that can be loaded on demand. Use dynamic imports for loading components asynchronously. +* **Library Splitting:** Extract commonly used libraries into separate chunks to allow browsers to cache them independently. + +## 2. Common Patterns and Anti-Patterns + +### 2.1. Design Patterns + +* **Selectors:** Use selectors to abstract the state shape and compute derived data. Selectors improve performance by memoizing results and prevent components from re-rendering unnecessarily. +* **Custom Hooks:** Create custom hooks for accessing Redux state and dispatching actions. This simplifies component logic and promotes code reuse. For example: + + typescript + import { useDispatch, useSelector } from 'react-redux'; + import { increment, decrement } from './counterSlice'; + import { RootState } from './store'; + + export const useCounter = () => { + const count = useSelector((state: RootState) => state.counter.value); + const dispatch = useDispatch(); + + const handleIncrement = () => { + dispatch(increment()); + }; + + const handleDecrement = () => { + dispatch(decrement()); + }; + + return { count, handleIncrement, handleDecrement }; + }; + +* **Redux Toolkit:** Utilize Redux Toolkit to simplify Redux setup and reduce boilerplate code. Redux Toolkit provides utilities for creating reducers, actions, and the store. + +### 2.2. Recommended Approaches + +* **Normalize State:** Structure your state as a normalized data structure, where each piece of data has a unique ID, and relationships between data are represented by IDs. This improves performance and simplifies data management. Use libraries like Normalizr to help with this. +* **Immutability:** Treat your state as immutable and use immutable update patterns. This ensures predictable state transitions and prevents unexpected side effects. +* **Middleware for Side Effects:** Use Redux middleware (e.g., Redux Thunk, Redux Saga) to handle asynchronous operations and side effects. Avoid performing side effects directly in reducers. + +### 2.3. Anti-Patterns and Code Smells + +* **Mutating State Directly:** Never mutate the state directly in reducers. Always create a new copy of the state with the desired changes. +* **Performing Side Effects in Reducers:** Reducers should be pure functions and should not perform any side effects (e.g., API calls, logging). +* **Storing UI State in Redux:** Avoid storing UI-specific state (e.g., component visibility, form input values) in the Redux store. Store UI state locally in component state. +* **Over-reliance on Global State:** Avoid putting everything in the Redux store. Only store data that needs to be accessed by multiple components or across the entire application. Consider React Context or local component state for component-specific data. + +### 2.4. State Management Best Practices + +* **Minimize Global State:** Only store data in the Redux store that needs to be shared across multiple components. For component-specific state, use local component state. +* **Use Selectors:** Use selectors to access and transform data from the Redux store. This allows you to abstract the state shape and prevent components from re-rendering unnecessarily. +* **Immutable Updates:** Always update state immutably to ensure predictable state transitions and prevent unexpected side effects. Use libraries like Immer to simplify immutable updates. + +### 2.5. Error Handling Patterns + +* **Centralized Error Handling:** Implement a centralized error handling mechanism to catch and handle errors consistently throughout the application. Use error boundary components to catch errors in the UI. +* **Action Creators for Errors:** Dispatch error actions to the Redux store when errors occur. This allows you to track errors and display error messages in the UI. +* **Logging Errors:** Log errors to a central logging service for debugging and monitoring purposes. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +* **Memoization:** Use memoization techniques (e.g., `React.memo`, `useMemo`, Reselect) to prevent components from re-rendering unnecessarily. +* **Batch Updates:** Batch multiple Redux dispatches into a single update to improve performance. Use `redux-batched-updates` or `unstable_batchedUpdates` from React DOM. +* **Virtualization:** Use virtualization techniques (e.g., `react-window`, `react-virtualized`) to efficiently render large lists and tables. +* **Code Splitting:** Split the application into smaller chunks to reduce the initial load time. + +### 3.2. Memory Management + +* **Avoid Memory Leaks:** Be mindful of memory leaks, especially in event listeners, timers, and subscriptions. Clean up resources properly when components unmount. +* **Use WeakRefs:** Use WeakRefs to hold references to objects without preventing them from being garbage collected. +* **Profile Memory Usage:** Use the browser's developer tools to profile memory usage and identify potential memory leaks. + +### 3.3. Rendering Optimization + +* **ShouldComponentUpdate / React.memo:** Use `shouldComponentUpdate` (for class components) or `React.memo` (for functional components) to prevent components from re-rendering unnecessarily. +* **Pure Components:** Use pure components to automatically implement `shouldComponentUpdate` based on prop comparisons. +* **Immutable Data Structures:** Use immutable data structures to make it easier to detect changes and prevent re-renders. + +### 3.4. Bundle Size Optimization + +* **Tree Shaking:** Use tree shaking to remove dead code from your bundles. Configure your bundler (e.g., Webpack, Parcel) to enable tree shaking. +* **Code Splitting:** Split the application into smaller chunks to reduce the initial load time. +* **Minification:** Minify your code to reduce the bundle size. +* **Compression:** Compress your bundles using gzip or Brotli. + +### 3.5. Lazy Loading Strategies + +* **Route-Based Lazy Loading:** Load components only when their corresponding routes are visited. +* **Component-Based Lazy Loading:** Load components on demand when they become visible in the UI. +* **Image Lazy Loading:** Load images only when they scroll into view. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities and Prevention + +* **Cross-Site Scripting (XSS):** Prevent XSS vulnerabilities by sanitizing user input and escaping output. +* **Cross-Site Request Forgery (CSRF):** Protect against CSRF attacks by using anti-CSRF tokens. +* **Injection Attacks:** Prevent injection attacks (e.g., SQL injection, command injection) by validating user input and using parameterized queries. +* **Authentication and Authorization:** Implement secure authentication and authorization mechanisms to protect sensitive data and resources. + +### 4.2. Input Validation + +* **Server-Side Validation:** Always validate user input on the server-side to prevent malicious data from being stored in the database. +* **Client-Side Validation:** Use client-side validation to provide immediate feedback to users and reduce the load on the server. +* **Whitelisting:** Use whitelisting to allow only specific characters or patterns in user input. + +### 4.3. Authentication and Authorization Patterns + +* **JSON Web Tokens (JWT):** Use JWTs for authentication and authorization. JWTs are stateless and can be easily verified on the server-side. +* **OAuth 2.0:** Use OAuth 2.0 for delegated authorization. OAuth 2.0 allows users to grant third-party applications access to their resources without sharing their credentials. +* **Role-Based Access Control (RBAC):** Implement RBAC to control access to resources based on user roles. + +### 4.4. Data Protection Strategies + +* **Encryption:** Encrypt sensitive data at rest and in transit. +* **Data Masking:** Mask sensitive data to prevent unauthorized access. +* **Data Anonymization:** Anonymize data to remove personally identifiable information (PII). + +### 4.5. Secure API Communication + +* **HTTPS:** Use HTTPS to encrypt communication between the client and the server. +* **API Keys:** Use API keys to authenticate API requests. +* **Rate Limiting:** Implement rate limiting to prevent abuse and denial-of-service attacks. + +## 5. Testing Approaches + +### 5.1. Unit Testing Strategies + +* **Test Reducers:** Test reducers to ensure that they handle actions correctly and update the state as expected. +* **Test Selectors:** Test selectors to ensure that they return the correct data from the state. +* **Test Actions:** Test action creators to ensure that they return the correct actions. +* **Test Components:** Test components to ensure that they render correctly and handle user interactions as expected. Use tools like React Testing Library. + +### 5.2. Integration Testing + +* **Test Component Interactions:** Test how components interact with each other and with the Redux store. +* **Test Data Flow:** Test the flow of data through the application, from the UI to the Redux store and back. +* **Test API Integrations:** Test how the application integrates with external APIs. + +### 5.3. End-to-End Testing + +* **Test User Flows:** Test complete user flows to ensure that the application functions correctly from end to end. Use tools like Cypress or Playwright. +* **Test Critical Functionality:** Test critical functionality (e.g., login, checkout) to ensure that it is working as expected. +* **Test Accessibility:** Test the application for accessibility to ensure that it is usable by people with disabilities. + +### 5.4. Test Organization + +* **Test Folder Structure:** Organize tests in a way that mirrors the application's directory structure. +* **Test Suites:** Group tests into logical suites based on functionality or component. +* **Test Naming Conventions:** Use clear and consistent naming conventions for tests. + +### 5.5. Mocking and Stubbing + +* **Mock API Calls:** Mock API calls to isolate components and reducers during testing. +* **Stub External Dependencies:** Stub external dependencies to control their behavior during testing. +* **Use Mock Store:** Use a mock Redux store to test components that are connected to the store. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +* **Forgetting to Connect Components:** Forgetting to connect components to the Redux store, resulting in components not receiving state updates. +* **Incorrectly Mapping State to Props:** Incorrectly mapping state to props, resulting in components not receiving the correct data. +* **Not Updating State Immutably:** Not updating state immutably, leading to unexpected side effects and re-renders. +* **Using `setState` with Redux:** Mixing `setState` with Redux can lead to inconsistencies and difficult-to-debug issues. Avoid using `setState` in components connected to Redux, unless for purely local, non-Redux-related state. + +### 6.2. Edge Cases + +* **Race Conditions:** Be aware of race conditions when handling asynchronous operations. +* **Memory Leaks:** Watch out for memory leaks in event listeners, timers, and subscriptions. +* **Performance Bottlenecks:** Identify and address performance bottlenecks, such as unnecessary re-renders or slow API calls. + +### 6.3. Version-Specific Issues + +* **Breaking Changes:** Be aware of breaking changes when upgrading React and Redux. +* **Deprecated Features:** Avoid using deprecated features and migrate to the recommended alternatives. + +### 6.4. Compatibility Concerns + +* **Browser Compatibility:** Test the application in different browsers to ensure compatibility. +* **Device Compatibility:** Test the application on different devices to ensure compatibility. + +### 6.5. Debugging Strategies + +* **Redux DevTools:** Use the Redux DevTools to inspect the Redux store, track actions, and time travel through state changes. +* **Console Logging:** Use console logging to debug code and track the flow of data. +* **Breakpoints:** Use breakpoints to pause execution and inspect variables in the debugger. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +* **Redux DevTools:** The Redux DevTools extension for Chrome and Firefox allows you to inspect the Redux store, track actions, and time travel through state changes. +* **React Developer Tools:** The React Developer Tools extension for Chrome and Firefox allows you to inspect the React component tree and profile component performance. +* **ESLint:** ESLint is a linter that helps you identify and fix code style issues and potential errors. +* **Prettier:** Prettier is a code formatter that automatically formats your code to ensure consistency. + +### 7.2. Build Configuration + +* **Webpack:** Webpack is a module bundler that bundles your code and dependencies into optimized bundles for production. +* **Parcel:** Parcel is a zero-configuration bundler that is easy to use and provides fast build times. +* **Create React App:** Create React App is a tool that sets up a React development environment with sensible defaults. + +### 7.3. Linting and Formatting + +* **ESLint:** Use ESLint to enforce code style rules and prevent potential errors. Configure ESLint to use a popular style guide, such as Airbnb or Standard. +* **Prettier:** Use Prettier to automatically format your code to ensure consistency. Integrate Prettier with ESLint to automatically fix code style issues. +* **Husky and Lint-Staged:** Use Husky and Lint-Staged to automatically run linters and formatters on staged files before committing code. + +### 7.4. Deployment Best Practices + +* **Environment Variables:** Use environment variables to configure the application for different environments (e.g., development, staging, production). +* **Continuous Integration:** Use a continuous integration (CI) system to automatically build and test the application whenever code is committed. +* **Continuous Deployment:** Use a continuous deployment (CD) system to automatically deploy the application to production whenever a new release is created. +* **Caching:** Configure caching on the server-side and client-side to improve performance. + +### 7.5. CI/CD Integration + +* **GitHub Actions:** Use GitHub Actions to automate the build, test, and deployment process. +* **CircleCI:** Use CircleCI to automate the build, test, and deployment process. +* **Jenkins:** Use Jenkins to automate the build, test, and deployment process. \ No newline at end of file diff --git a/.cursor/rules/react.mdc b/.cursor/rules/react.mdc new file mode 100644 index 000000000..a8802bc01 --- /dev/null +++ b/.cursor/rules/react.mdc @@ -0,0 +1,284 @@ +--- +description: Comprehensive guide to React best practices, covering code organization, performance, security, testing, and common pitfalls. Adhering to these guidelines helps developers build maintainable, scalable, and high-performing React applications. +globs: *.js,*.jsx,*.ts,*.tsx +--- +# React Best Practices: A Comprehensive Guide + +This document outlines the best practices for developing React applications, covering various aspects from code organization to security and testing. Following these guidelines leads to more maintainable, scalable, and performant applications. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure + +A well-defined directory structure is crucial for maintainability. Here's a recommended structure: + + +src/ + ├── components/ + │ ├── Button/ + │ │ ├── Button.jsx + │ │ ├── Button.module.css + │ │ └── Button.test.jsx + │ ├── Input/ + │ │ ├── Input.jsx + │ │ ├── Input.module.css + │ │ └── Input.test.jsx + │ └── ... + ├── contexts/ + │ ├── AuthContext.jsx + │ └── ThemeContext.jsx + ├── hooks/ + │ ├── useAuth.js + │ └── useTheme.js + ├── pages/ + │ ├── Home.jsx + │ ├── About.jsx + │ └── ... + ├── services/ + │ ├── api.js + │ └── auth.js + ├── utils/ + │ ├── helpers.js + │ └── validators.js + ├── App.jsx + ├── index.jsx + └── ... + + +- **`components/`**: Reusable UI components. + - Each component has its own directory containing the component file, associated styles (using CSS modules), and tests. +- **`contexts/`**: React context providers. +- **`hooks/`**: Custom React hooks. +- **`pages/`**: Top-level components representing different routes or views. +- **`services/`**: API interaction logic. +- **`utils/`**: Utility functions. + +### 1.2 File Naming Conventions + +- **Components**: Use PascalCase (e.g., `MyComponent.jsx`). +- **Hooks**: Use camelCase prefixed with `use` (e.g., `useMyHook.js`). +- **Contexts**: Use PascalCase suffixed with `Context` (e.g., `MyContext.jsx`). +- **Services/Utils**: Use camelCase (e.g., `apiService.js`, `stringUtils.js`). +- **CSS Modules**: Use `.module.css` or `.module.scss` (e.g., `Button.module.css`). + +### 1.3 Module Organization + +- **Co-location**: Keep related files (component, styles, tests) together in the same directory. +- **Single Responsibility**: Each module should have a clear and specific purpose. +- **Avoid Circular Dependencies**: Ensure modules don't depend on each other in a circular manner. + +### 1.4 Component Architecture + +- **Atomic Design**: Consider using Atomic Design principles (Atoms, Molecules, Organisms, Templates, Pages) to structure components. +- **Composition over Inheritance**: Favor component composition to reuse code and functionality. +- **Presentational and Container Components**: Separate UI rendering (presentational) from state management and logic (container). + +### 1.5 Code Splitting Strategies + +- **Route-Based Splitting**: Use `React.lazy` and `Suspense` to load components only when a specific route is accessed. This is very common and improves initial load time. +- **Component-Based Splitting**: Split large components into smaller chunks that can be loaded on demand. +- **Bundle Analyzer**: Use a tool like `webpack-bundle-analyzer` to identify large dependencies and optimize bundle size. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns + +- **Higher-Order Components (HOCs)**: Reusable logic that wraps components (use with caution; prefer hooks). +- **Render Props**: Sharing code using a prop whose value is a function. +- **Compound Components**: Components that work together implicitly (e.g., `Tabs`, `Tab`). +- **Hooks**: Reusable stateful logic that can be shared across functional components. + +### 2.2 Recommended Approaches + +- **Form Handling**: Use controlled components with local state or a form library like Formik or React Hook Form. +- **API Calls**: Use `useEffect` hook to make API calls and manage loading states. +- **Conditional Rendering**: Use short-circuit evaluation (`&&`) or ternary operators for simple conditions; use separate components for complex scenarios. +- **List Rendering**: Always provide a unique and stable `key` prop when rendering lists. + +### 2.3 Anti-patterns and Code Smells + +- **Direct DOM Manipulation**: Avoid directly manipulating the DOM; let React handle updates. +- **Mutating State Directly**: Always use `setState` or the state updater function to modify state. +- **Inline Styles**: Use CSS modules or styled-components for maintainable styles. +- **Over-Engineering**: Avoid using complex solutions for simple problems. +- **Prop Drilling**: Passing props through multiple levels of components without them being used. + +### 2.4 State Management Best Practices + +- **Local State**: Use `useState` for component-specific state. +- **Context API**: Use `useContext` for global state accessible to many components, but avoid for very frequently updated data. +- **Redux/Mobx**: Use these libraries for complex state management in large applications. +- **Recoil/Zustand**: Lightweight alternatives to Redux, often easier to set up and use. +- **Immutable Data**: Treat state as immutable to prevent unexpected side effects. + +### 2.5 Error Handling Patterns + +- **Error Boundaries**: Wrap components with error boundaries to catch errors during rendering and prevent crashes. +- **Try-Catch Blocks**: Use try-catch blocks for handling errors in asynchronous operations and event handlers. +- **Centralized Error Logging**: Implement a centralized error logging service to track errors and improve application stability. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +- **Memoization**: Use `React.memo`, `useMemo`, and `useCallback` to prevent unnecessary re-renders and recalculations. +- **Virtualization**: Use libraries like `react-window` or `react-virtualized` to efficiently render large lists or tables. +- **Debouncing/Throttling**: Limit the rate at which functions are executed (e.g., in input fields). +- **Code Splitting**: Load code on demand using `React.lazy` and `Suspense`. + +### 3.2 Memory Management + +- **Avoid Memory Leaks**: Clean up event listeners, timers, and subscriptions in `useEffect`'s cleanup function. +- **Release Unused Objects**: Avoid holding onto large objects in memory when they are no longer needed. +- **Garbage Collection**: Understand how JavaScript's garbage collection works and avoid creating unnecessary objects. + +### 3.3 Rendering Optimization + +- **Minimize State Updates**: Avoid unnecessary state updates that trigger re-renders. +- **Batch Updates**: Batch multiple state updates into a single update using `ReactDOM.unstable_batchedUpdates`. +- **Keys**: Ensure that keys are unique and consistent across renders. + +### 3.4 Bundle Size Optimization + +- **Tree Shaking**: Remove unused code during the build process. +- **Minification**: Reduce the size of JavaScript and CSS files. +- **Image Optimization**: Compress and optimize images to reduce file size. +- **Dependency Analysis**: Use tools like `webpack-bundle-analyzer` to identify large dependencies. + +### 3.5 Lazy Loading Strategies + +- **Route-Based Lazy Loading**: Load components when a user navigates to a specific route. +- **Component-Based Lazy Loading**: Load components when they are about to be rendered. +- **Intersection Observer**: Load components when they become visible in the viewport. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and Prevention + +- **Cross-Site Scripting (XSS)**: Sanitize user input to prevent malicious code injection. +- **Cross-Site Request Forgery (CSRF)**: Use anti-CSRF tokens to protect against unauthorized requests. +- **Denial of Service (DoS)**: Implement rate limiting and request validation to prevent abuse. +- **Injection Attacks**: Avoid directly embedding user input into database queries or system commands. + +### 4.2 Input Validation + +- **Client-Side Validation**: Validate user input in the browser to provide immediate feedback. +- **Server-Side Validation**: Always validate user input on the server to prevent malicious data. +- **Sanitize Input**: Sanitize user input to remove potentially harmful characters or code. + +### 4.3 Authentication and Authorization + +- **Secure Authentication**: Use secure authentication mechanisms like OAuth 2.0 or JWT. +- **Role-Based Access Control (RBAC)**: Implement RBAC to control access to resources based on user roles. +- **Multi-Factor Authentication (MFA)**: Enable MFA to add an extra layer of security. + +### 4.4 Data Protection Strategies + +- **Encryption**: Encrypt sensitive data at rest and in transit. +- **Data Masking**: Mask sensitive data in logs and UI displays. +- **Regular Backups**: Create regular backups of application data. + +### 4.5 Secure API Communication + +- **HTTPS**: Use HTTPS to encrypt communication between the client and the server. +- **API Keys**: Protect API keys and secrets. +- **CORS**: Configure Cross-Origin Resource Sharing (CORS) to prevent unauthorized access to APIs. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +- **Test Components**: Test individual components in isolation. +- **Testing Library**: Use React Testing Library for UI testing, focusing on user behavior. +- **Jest**: Use Jest as the test runner. + +### 5.2 Integration Testing + +- **Test Component Interactions**: Test how components interact with each other. +- **Mock API Calls**: Mock API calls to test component behavior in different scenarios. +- **React Testing Library**: Effective for testing integration points in components. + +### 5.3 End-to-End (E2E) Testing + +- **Test Full Application Flows**: Test complete user flows, such as login, registration, and checkout. +- **Cypress/Playwright**: Use tools like Cypress or Playwright for E2E testing. +- **Automated Browser Tests**: Automate browser tests to ensure application stability. + +### 5.4 Test Organization + +- **Co-locate Tests**: Keep test files close to the components they test (e.g., `Button.test.jsx` in the `Button` directory). +- **Descriptive Names**: Use descriptive names for test files and test cases. +- **Test Suites**: Organize tests into logical suites. + +### 5.5 Mocking and Stubbing + +- **Mock Modules**: Mock external modules or API calls to isolate components during testing. +- **Stub Functions**: Stub function implementations to control component behavior. +- **Jest Mocks**: Utilize Jest's mocking capabilities for effective unit testing. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +- **Ignoring Keys in Lists**: Forgetting to provide unique and stable `key` props when rendering lists. +- **Incorrect State Updates**: Mutating state directly instead of using `setState` or the state updater function. +- **Missing Dependencies in `useEffect`**: Not including all dependencies in the dependency array of the `useEffect` hook. +- **Over-Using State**: Storing derived data in state instead of calculating it on demand. + +### 6.2 Edge Cases + +- **Asynchronous State Updates**: Handling state updates in asynchronous operations. +- **Race Conditions**: Preventing race conditions when making multiple API calls. +- **Handling Errors in Event Handlers**: Properly handling errors in event handlers to prevent crashes. + +### 6.3 Version-Specific Issues + +- **React 16 vs. React 17/18**: Understanding differences in lifecycle methods, error handling, and concurrent mode. +- **Deprecated Features**: Being aware of deprecated features and using recommended alternatives. + +### 6.4 Compatibility Concerns + +- **Browser Compatibility**: Ensuring compatibility with different browsers and devices. +- **Library Compatibility**: Ensuring compatibility between React and other libraries. + +### 6.5 Debugging Strategies + +- **React DevTools**: Use React DevTools to inspect component hierarchies, props, and state. +- **Console Logging**: Use console logging to debug code and track variables. +- **Breakpoints**: Set breakpoints in the code to step through execution and inspect variables. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +- **VS Code**: A popular code editor with excellent React support. +- **Create React App**: A tool for quickly setting up a new React project. +- **React DevTools**: A browser extension for inspecting React components. +- **ESLint**: A linter for enforcing code style and preventing errors. +- **Prettier**: A code formatter for automatically formatting code. + +### 7.2 Build Configuration + +- **Webpack/Vite**: Configure Webpack or Vite to bundle and optimize code. +- **Babel**: Configure Babel to transpile JavaScript code to older versions. +- **Environment Variables**: Use environment variables to configure different environments. + +### 7.3 Linting and Formatting + +- **ESLint**: Configure ESLint with recommended React rules. +- **Prettier**: Configure Prettier to automatically format code. +- **Husky/lint-staged**: Use Husky and lint-staged to run linters and formatters before committing code. + +### 7.4 Deployment Best Practices + +- **Static Hosting**: Host static assets on a CDN. +- **Server-Side Rendering (SSR)**: Use SSR to improve SEO and initial load time. +- **Continuous Deployment**: Automate the deployment process using CI/CD. + +### 7.5 CI/CD Integration + +- **GitHub Actions/GitLab CI**: Use GitHub Actions or GitLab CI to automate testing, linting, and deployment. +- **Automated Testing**: Run automated tests on every commit or pull request. +- **Automated Deployment**: Automatically deploy code to production after successful tests. + +By following these best practices, React developers can build high-quality, maintainable, and scalable applications that meet the demands of modern web development. Continual education and adaptation to emerging trends in the React ecosystem are crucial for sustained success. \ No newline at end of file diff --git a/.cursor/rules/redis.mdc b/.cursor/rules/redis.mdc new file mode 100644 index 000000000..807cc3cbe --- /dev/null +++ b/.cursor/rules/redis.mdc @@ -0,0 +1,75 @@ +--- +description: This rule provides best practices for working with Redis, covering code organization, performance, security, testing, and common pitfalls to ensure efficient and reliable usage. It applies to any language file interacting with Redis. +globs: *.py,*.js,*.go,*.java,*.c,*.cpp,*.rb,*.php,*.ts,*.rs,*.kt,*.scala +--- +- **General Best Practices** + - Follow official Redis documentation and community best practices. + - Use the most recent stable version of the Redis client library for your language. + - Regularly update the Redis server and client libraries to benefit from bug fixes, performance improvements, and security patches. + +- **Connection Management** + - **Connection Pooling:** Use connection pooling to reduce the overhead of creating new connections for each operation. This significantly improves performance, especially in high-traffic scenarios. + - **Connection Timeout:** Configure appropriate connection timeouts to prevent indefinite blocking in case of network issues. + - **Retry Logic:** Implement retry logic with exponential backoff for transient connection errors. + - **Secure Connections:** When connecting to a remote Redis instance, always use TLS/SSL encryption (redis+ssl://) to protect data in transit, especially when dealing with sensitive information. Ensure proper certificate validation. + +- **Data Modeling and Serialization** + - **Key Naming:** Use consistent and meaningful key naming conventions. Consider using namespaces or prefixes to organize keys and avoid collisions. + - **Data Serialization:** Choose an efficient serialization format (e.g., JSON, Protocol Buffers, MessagePack) and use it consistently. Consider the trade-offs between human readability, storage space, and serialization/deserialization performance. + - **Smaller Values:** Redis works best with smaller values. Break larger data structures into smaller chunks and distribute them across multiple keys. This improves memory utilization and performance. + - **Data Types:** Leverage Redis's rich data types (strings, lists, sets, sorted sets, hashes, streams) effectively to optimize data storage and operations. Choose the data type that best suits the use case. + +- **Command Usage** + - **Transactions:** Use transactions (`MULTI`, `EXEC`, `DISCARD`, `WATCH`) to ensure atomicity when performing multiple operations. Be aware of the limitations of Redis transactions (no rollback on individual command failure). + - **Pipelines:** Use pipelines to batch multiple commands together and reduce network round-trip time. This dramatically improves performance for bulk operations. + - **Lua Scripting:** Use Lua scripting for complex operations that require atomicity and server-side processing. This reduces network traffic and improves performance compared to executing multiple individual commands. + - **Avoid Blocking Commands:** Avoid using blocking commands like `KEYS`, `FLUSHALL`, `FLUSHDB`, `SORT` without `LIMIT` in production environments. These commands can block the Redis server and degrade performance. + - **Use SCAN:** Instead of `KEYS`, use the `SCAN` command for iterating over keys in a non-blocking manner. This allows the Redis server to continue serving other requests while iterating. + - **Efficient Deletion:** Instead of `FLUSHALL` or `FLUSHDB`, use `SCAN` with `DEL` to delete keys in batches, minimizing disruption to the server. + - **TTL Management:** Set appropriate Time-To-Live (TTL) values for keys to automatically expire data that is no longer needed. This helps manage memory usage and prevent data from becoming stale. + +- **Memory Management** + - **Maxmemory:** Configure the `maxmemory` directive to limit the amount of memory Redis can use. When the limit is reached, Redis will evict keys based on the configured eviction policy. + - **Eviction Policies:** Choose an appropriate eviction policy (e.g., `LRU`, `LFU`, `volatile-ttl`) based on your application's needs. Understand the trade-offs between different eviction policies. + - **Memory Fragmentation:** Monitor memory fragmentation and consider restarting Redis periodically to defragment memory. The `INFO memory` command provides information about memory usage and fragmentation. + +- **Performance Monitoring and Tuning** + - **Redis Monitor:** Use the `MONITOR` command (with caution in production) to observe real-time commands being executed on the server. This can help identify performance bottlenecks. + - **Redis Slow Log:** Configure the Redis slow log to record commands that take longer than a specified amount of time to execute. Analyze the slow log to identify performance issues. + - **INFO Command:** Use the `INFO` command to gather information about the Redis server, including memory usage, CPU usage, and client connections. This information can be used to monitor performance and identify potential problems. + - **Latency Monitoring:** Monitor Redis latency using tools like `redis-cli --latency` or dedicated monitoring solutions. High latency can indicate performance issues. + +- **Security Considerations** + - **Authentication:** Enable authentication using the `requirepass` directive to protect the Redis server from unauthorized access. Use a strong password. + - **Access Control Lists (ACLs):** Use ACLs to restrict access to specific commands and keys for different users. This provides fine-grained control over access to Redis data. + - **Network Security:** Restrict network access to the Redis server using firewalls or other network security measures. Only allow connections from trusted sources. + - **Disable Unsafe Commands:** Disable or rename potentially dangerous commands like `FLUSHALL`, `FLUSHDB`, `KEYS`, `EVAL` using the `rename-command` directive. This reduces the risk of accidental or malicious misuse. + - **Regular Audits:** Conduct regular security audits of the Redis configuration and usage patterns to identify and address potential vulnerabilities. + - **Input Validation:** Always validate and sanitize any data being stored in Redis to prevent injection attacks. + +- **Testing Strategies** + - **Unit Tests:** Write unit tests to verify the functionality of your code that interacts with Redis. Use mocking or stubbing to isolate the Redis interactions from the rest of the code. + - **Integration Tests:** Write integration tests to verify the interaction between your code and the Redis server. Use a dedicated test Redis instance for integration tests. + - **End-to-End Tests:** Write end-to-end tests to verify the entire application flow, including the interaction with Redis. This ensures that the application works correctly in a realistic environment. + - **Data Population:** When testing, consider populating your redis database with a representative set of data. + - **Test Organization:** Organize tests logically, separating unit, integration, and end-to-end tests into different directories or modules. + - **Mocking and Stubbing:** Use mocking and stubbing frameworks to simulate Redis behavior during unit tests. + +- **Code Organization and Structure** + - **Dedicated Module:** Create a dedicated module or class to encapsulate all Redis-related operations. This promotes code reuse and maintainability. + - **Configuration Management:** Store Redis connection parameters (host, port, password) in a configuration file or environment variables. This makes it easy to change the Redis configuration without modifying code. + - **Abstraction Layer:** Consider creating an abstraction layer on top of the Redis client library to provide a higher-level API for your application. This can improve code readability and make it easier to switch to a different Redis client library in the future. + +- **Common Pitfalls and Gotchas** + - **N+1 Problem:** Avoid the N+1 problem when retrieving data from Redis. Instead of making multiple individual requests, use pipelining or Lua scripting to retrieve the data in a single request. + - **Race Conditions:** Be aware of potential race conditions when updating data in Redis. Use transactions or Lua scripting to ensure atomicity. + - **Large Values:** Avoid storing extremely large values in Redis. This can lead to performance issues and memory exhaustion. + - **Key Expiration:** Be careful when using key expiration (TTL). If keys expire unexpectedly, it can lead to data loss or inconsistent application behavior. + - **Blocking Operations in Event Loops:** Do not perform blocking Redis operations directly in event loops or GUI threads. Use asynchronous operations instead to avoid blocking the main thread. + +- **Tooling and Environment** + - **Redis CLI:** Use the Redis CLI (`redis-cli`) for interacting with the Redis server, executing commands, and monitoring performance. + - **Redis Desktop Manager:** Use a Redis desktop manager (e.g., RedisInsight, Medis) for visualizing data, managing keys, and monitoring the Redis server. + - **Linting and Formatting:** Configure linters and formatters to enforce consistent code style and best practices in your Redis-related code. + - **CI/CD Integration:** Integrate Redis testing and deployment into your CI/CD pipeline to automate the testing and deployment process. + - **Monitoring Tools:** Utilize monitoring tools like Prometheus, Grafana, or Datadog to monitor Redis performance and health in production environments. \ No newline at end of file diff --git a/.cursor/rules/redux.mdc b/.cursor/rules/redux.mdc new file mode 100644 index 000000000..2a5a03cc4 --- /dev/null +++ b/.cursor/rules/redux.mdc @@ -0,0 +1,196 @@ +--- +description: This rule provides comprehensive guidance on Redux best practices, covering code structure, performance optimization, testing strategies, and common pitfalls to ensure robust and maintainable Redux applications. +globs: *.js,*.jsx,*.ts,*.tsx +--- +- **Introduction:** This document outlines best practices for developing applications with Redux, covering various aspects from code organization to performance optimization. + +## 1. Code Organization and Structure: + +- **Directory Structure:** Organize your Redux-related files in a clear and maintainable structure. A feature-based approach is recommended. + - Example: + + src/ + ├── app/ + │ ├── store.js # Redux store configuration + ├── features/ + │ ├── counter/ # Feature module (e.g., 'counter') + │ │ ├── counterSlice.js # Redux slice (reducers + actions) + │ │ ├── Counter.jsx # React component + │ │ ├── counterAPI.js # API interaction logic + │ │ └── counterSelectors.js # Selectors + │ ├── todos/ + │ │ └── ... + ├── components/ # Reusable UI components + ├── utils/ # Utility functions + └── ... + + +- **File Naming Conventions:** Use consistent and descriptive names for your files. + - Example: + - `counterSlice.js` for Redux slice files + - `Counter.jsx` for React components + - `counterActions.js` or preferably included in slice + - `counterSelectors.js` for selectors + +- **Module Organization:** Group related logic into modules. Each feature should have its own directory containing its Redux slice, components, and selectors. + +- **Component Architecture:** Use a container/presentational component pattern to separate data fetching and state management logic from UI rendering. + - **Container Components:** Connect to the Redux store and pass data to presentational components. + - **Presentational Components:** Focus on UI rendering and receive data and callbacks as props. + +- **Code Splitting Strategies:** Implement code splitting to reduce the initial bundle size and improve loading times. + - **Route-based splitting:** Load different parts of the application based on the current route. + - **Component-based splitting:** Lazy-load components that are not immediately needed. + - Use `React.lazy` and `` for component-based splitting. + +## 2. Common Patterns and Anti-patterns: + +- **Design Patterns:** + - **Flux Standard Actions (FSA):** A convention for structuring Redux actions with a `type` and an optional `payload` and `error` property. + - **Selectors:** Use selectors (e.g., with Reselect) to derive data from the Redux store efficiently. + - **Thunks/Sagas:** Manage asynchronous side effects using Redux Thunk or Redux Saga. + +- **Recommended Approaches:** + - **Redux Toolkit:** Use Redux Toolkit to simplify Redux development and reduce boilerplate. + - **Immutability:** Treat state as immutable and use immutable data structures or techniques to update it. + - **Normalization:** Normalize your state to reduce data duplication and improve data consistency. + +- **Anti-patterns and Code Smells:** + - **Mutating State Directly:** Never mutate state directly. Always create new copies of objects and arrays. + - **Putting too much logic in components:** Keep components focused on rendering. Move complex logic to reducers, selectors, or middleware. + - **Large Reducers:** Split large reducers into smaller, more manageable reducers using `combineReducers`. + - **Over-fetching Data:** Fetch only the data that is needed by the component. Use selectors to derive specific data subsets from the state. + +- **State Management Best Practices:** + - **Single Source of Truth:** Keep all application state in the Redux store. + - **Predictable State Updates:** Ensure that state updates are predictable and deterministic. + - **Minimize Global State:** Only store data that is truly global in the Redux store. Local component state should be used for UI-specific data. + +- **Error Handling Patterns:** + - **Action-based Error Handling:** Dispatch actions to indicate errors and update the state accordingly. + - **Middleware Error Handling:** Use middleware to catch errors and log them to a central error reporting service. + +## 3. Performance Considerations: + +- **Optimization Techniques:** + - **Memoization:** Use memoization to avoid unnecessary re-renders. Use `React.memo` for functional components and `shouldComponentUpdate` for class components. + - **Selectors:** Use selectors with memoization (e.g., Reselect) to avoid recomputing derived data when the input state hasn't changed. + - **Batch Updates:** Batch multiple state updates into a single update to reduce the number of re-renders. + - **Debouncing/Throttling:** Use debouncing or throttling to limit the frequency of updates. + +- **Memory Management:** + - **Avoid Memory Leaks:** Clean up event listeners and timers when components unmount. + - **Optimize Data Structures:** Use efficient data structures to reduce memory usage. + +- **Rendering Optimization:** + - **Virtualization:** Use virtualization for large lists to only render the visible items. + - **Code Splitting:** Reduce the initial bundle size by splitting the code into smaller chunks. + +- **Bundle Size Optimization:** + - **Tree Shaking:** Remove unused code from the bundle using tree shaking. + - **Minification:** Minify the code to reduce the bundle size. + - **Compression:** Compress the bundle using Gzip or Brotli. + +- **Lazy Loading Strategies:** + - **Route-based Lazy Loading:** Load different routes on demand. + - **Component-based Lazy Loading:** Load components on demand using `React.lazy`. + +## 4. Security Best Practices: + +- **Common Vulnerabilities:** + - **Cross-Site Scripting (XSS):** Prevent XSS attacks by sanitizing user inputs and escaping outputs. + - **Cross-Site Request Forgery (CSRF):** Protect against CSRF attacks by using anti-CSRF tokens. + - **Data Injection:** Prevent data injection attacks by validating and sanitizing inputs. + +- **Input Validation:** + - **Client-Side Validation:** Validate inputs on the client-side to provide immediate feedback to the user. + - **Server-Side Validation:** Validate inputs on the server-side to ensure data integrity. + +- **Authentication and Authorization Patterns:** + - **JWT (JSON Web Tokens):** Use JWTs for authentication and authorization. + - **Role-Based Access Control (RBAC):** Implement RBAC to control access to different parts of the application. + +- **Data Protection Strategies:** + - **Encryption:** Encrypt sensitive data at rest and in transit. + - **Data Masking:** Mask sensitive data to protect it from unauthorized access. + +- **Secure API Communication:** + - **HTTPS:** Use HTTPS to encrypt communication between the client and the server. + - **API Rate Limiting:** Implement API rate limiting to prevent abuse. + +## 5. Testing Approaches: + +- **Unit Testing Strategies:** + - **Test Reducers:** Test reducers to ensure that they update the state correctly. + - **Test Actions:** Test actions to ensure that they dispatch the correct payloads. + - **Test Selectors:** Test selectors to ensure that they derive the correct data from the state. + +- **Integration Testing:** + - **Test Components:** Test components to ensure that they render correctly and interact with the Redux store. + - **Test Middleware:** Test middleware to ensure that they handle side effects correctly. + +- **End-to-end Testing:** + - **Test User Flows:** Test user flows to ensure that the application works as expected from the user's perspective. + +- **Test Organization:** + - **Test Directory Structure:** Organize tests in a clear and maintainable structure. + - Example: + + src/ + ├── features/ + │ ├── counter/ + │ │ ├── counterSlice.test.js + │ │ ├── Counter.test.jsx + + +- **Mocking and Stubbing:** + - **Mock API Calls:** Mock API calls to isolate components and reducers during testing. + - **Stub Dependencies:** Stub external dependencies to control their behavior during testing. + +## 6. Common Pitfalls and Gotchas: + +- **Frequent Mistakes:** + - **Forgetting to Return State:** Reducers must always return the new state. If no change, return the original state. + - **Incorrectly Updating Nested Objects:** Remember to create copies of all levels when updating nested objects. + - **Not Handling Asynchronous Actions Correctly:** Using thunks or sagas incorrectly can lead to unexpected behavior. + +- **Edge Cases:** + - **Initial State:** Ensure that the initial state is correctly defined. + - **Handling Errors:** Handle errors gracefully and provide informative error messages. + - **Empty States:** Handle empty states correctly to avoid unexpected behavior. + +- **Version-Specific Issues:** + - **React Redux v8 vs v7:** Be aware of the changes introduced in React Redux v8, such as the improved performance and the new `useSyncExternalStore` hook. + +- **Compatibility Concerns:** + - **Browser Compatibility:** Ensure that the application works correctly in all supported browsers. + - **Device Compatibility:** Ensure that the application works correctly on all supported devices. + +- **Debugging Strategies:** + - **Redux DevTools:** Use the Redux DevTools extension to inspect the state and track actions. + - **Console Logging:** Use console logging to debug code and track state changes. + - **Debugging Tools:** Use browser debugging tools to step through code and inspect variables. + +## 7. Tooling and Environment: + +- **Recommended Development Tools:** + - **Redux DevTools Extension:** Inspect Redux state and actions. + - **VS Code with Redux Extension:** Provides code completion, linting, and debugging support for Redux. + +- **Build Configuration:** + - **Webpack/Parcel/Rollup:** Use a module bundler to bundle the code and optimize it for production. + - **Babel:** Use Babel to transpile the code to older versions of JavaScript. + +- **Linting and Formatting:** + - **ESLint:** Use ESLint to enforce code style and prevent errors. + - **Prettier:** Use Prettier to automatically format the code. + +- **Deployment Best Practices:** + - **Environment Variables:** Use environment variables to configure the application for different environments. + - **CDN:** Use a CDN to serve static assets. + +- **CI/CD Integration:** + - **Automated Testing:** Run tests automatically as part of the CI/CD pipeline. + - **Automated Deployment:** Deploy the application automatically to different environments. + +This comprehensive guide should help you build robust, maintainable, and performant Redux applications. Remember to stay up-to-date with the latest best practices and tools in the Redux ecosystem. \ No newline at end of file diff --git a/.cursor/rules/remix.mdc b/.cursor/rules/remix.mdc new file mode 100644 index 000000000..ebec31841 --- /dev/null +++ b/.cursor/rules/remix.mdc @@ -0,0 +1,315 @@ +--- +description: This rule file provides comprehensive best practices for Remix development, covering code organization, performance, security, testing, and more. It aims to guide developers in building maintainable, scalable, and secure Remix applications. +globs: *.js,*.jsx,*.ts,*.tsx +--- +## Remix Best Practices and Coding Standards + +This document outlines the recommended best practices and coding standards for developing Remix applications. Following these guidelines will promote code consistency, maintainability, performance, and security. + +### 1. Code Organization and Structure + +#### 1.1. Directory Structure + +A well-structured directory is crucial for maintainability and scalability. Here's a recommended structure: + + +my-remix-app/ +├── app/ +│ ├── components/ # Reusable UI components +│ │ ├── Button.tsx +│ │ ├── Card.tsx +│ │ └── ... +│ ├── utils/ # Utility functions (e.g., date formatting, API helpers) +│ │ ├── date-utils.ts +│ │ ├── api.ts +│ │ └── ... +│ ├── services/ # Business logic and data access +│ │ ├── auth.server.ts # Authentication logic (server-only) +│ │ ├── user.server.ts # User data access (server-only) +│ │ └── ... +│ ├── routes/ # Remix route modules +│ │ ├── _index.tsx # Index route +│ │ ├── about.tsx # About page +│ │ ├── blog/ +│ │ │ ├── $slug.tsx # Dynamic blog post route +│ │ │ └── index.tsx # Blog index page +│ │ ├── api/ +│ │ │ ├── auth.ts # API routes for authentication +│ │ │ └── ... +│ │ └── ... +│ ├── styles/ # Global stylesheets +│ │ ├── global.css +│ │ └── ... +│ ├── entry.client.tsx # Client-side entry point +│ ├── entry.server.tsx # Server-side entry point +│ ├── root.tsx # Root component (HTML structure) +│ └── remix.env.d.ts +├── public/ # Static assets (images, fonts, etc.) +├── .gitignore +├── jsconfig.json +├── package-lock.json +├── package.json +├── remix.config.js +└── tsconfig.json + + +* `components`: Reusable UI elements. Separate presentational components from container components (smart vs. dumb components). +* `utils`: Helper functions that are not specific to any React component. This promotes reusability and testability. +* `services`: Business logic related files that handle server-side interactions. +* `routes`: Defines the application's routes. Each file represents a route segment. Use nested routes for complex layouts and data dependencies. +* `styles`: Global styles. +* `public`: Static assets. + +#### 1.2. File Naming Conventions + +* **Components:** PascalCase (e.g., `Button.tsx`). +* **Route Modules:** kebab-case (e.g., `about-us.tsx`). Use `$param` for dynamic route segments (e.g., `$postId.tsx`). +* **Utility Functions:** camelCase (e.g., `formatDate.ts`). +* **Stylesheets:** kebab-case (e.g., `global.css`). +* Server only utilities that do not include UI (e.g `auth.server.ts`) + +#### 1.3. Module Organization + +* Group related components, utilities, and services into modules. A module is a directory containing files that work together to provide a specific functionality. This improves code discoverability and reduces naming conflicts. +* Use index files (`index.ts` or `index.tsx`) to re-export members from a module, providing a single entry point. + + +components/ +├── Button.tsx +├── Input.tsx +└── index.ts # export { Button } from './Button'; export { Input } from './Input'; + + +#### 1.4. Component Architecture + +* **Presentational vs. Container Components:** Separate components that handle data fetching and state management (container components) from components that only render UI (presentational components). This promotes reusability and testability. +* **Composition:** Favor composition over inheritance. Use React's `children` prop or render props to create flexible and reusable components. +* **Controlled vs Uncontrolled Components:** Understand the difference between controlled and uncontrolled components. Controlled components manage their own state, while uncontrolled components rely on the DOM. + +#### 1.5. Code Splitting + +* Remix automatically handles route-based code splitting. Each route module is loaded independently, reducing the initial bundle size. +* For larger components or modules, consider using dynamic imports (`React.lazy`) to further split your code. This is particularly helpful for features that are not immediately needed on page load. +* Utilize Remix's built-in support for resource routes to handle data loading and background tasks separately, preventing them from blocking the main UI thread. + + +### 2. Common Patterns and Anti-patterns + +#### 2.1. Design Patterns + +* **Compound Components:** Useful for components that need to share state or logic implicitly (e.g., Tabs, Accordions). Uses React Context to provide communication between parent and child components. +* **Render Props/Function as Child:** Provides maximum flexibility by allowing the parent component to control the rendering of its children. +* **Hooks:** Extract reusable stateful logic into custom hooks. This promotes code reuse and makes components more readable. +* **Provider Pattern:** For managing global state or providing context to a subtree of components. + +#### 2.2. Recommended Approaches + +* **Data Loading:** Use Remix loaders for server-side data fetching. This ensures that data is available before the component renders, improving performance and SEO. +* **Data Mutations:** Use Remix actions for handling form submissions and data updates. This centralizes data mutations and simplifies state management. +* **Error Handling:** Implement error boundaries at the route level to catch errors and prevent the entire application from crashing. +* **Authentication:** Implement authentication using server-side sessions or cookies. Avoid storing sensitive data in client-side storage. +* **Authorization:** Implement authorization checks in loaders and actions to ensure that users only have access to authorized resources. + +#### 2.3. Anti-patterns + +* **Direct DOM Manipulation:** Avoid direct DOM manipulation using `document.querySelector` or `document.getElementById`. Use React's state management and rendering capabilities instead. +* **Over-reliance on Client-Side State:** Utilize Remix's server-side capabilities to minimize client-side state management. This improves performance and reduces the risk of state inconsistencies. +* **Ignoring Server-Side Rendering:** Take advantage of Remix's server-side rendering capabilities for improved performance and SEO. Don't perform all data fetching and rendering on the client-side. +* **Complex Conditional Rendering in JSX:** Avoid deeply nested conditional rendering within JSX. Extract complex logic into separate functions or components. + +#### 2.4. State Management + +* Remix encourages server-side data fetching and mutations, reducing the need for complex client-side state management. +* For simple component-level state, use React's `useState` hook. +* For more complex application-level state, consider using Context API with `useReducer` or a state management library like Zustand or Jotai. +* If needed, integrate third party state management libraries like Redux with caution, considering the benefits of Remix's built in data handling. + +#### 2.5. Error Handling + +* Utilize Remix's ErrorBoundary component to create dedicated error screens for routes. This provides a better user experience when errors occur. +* Handle errors gracefully in loaders and actions. Return error responses or throw exceptions to trigger the error boundary. +* Implement logging to track errors and diagnose issues. +* Avoid try-catch blocks within components and rely on ErrorBoundaries for global exception handling. + + +### 3. Performance Considerations + +#### 3.1. Optimization Techniques + +* **Minimize Bundle Size:** Remove unused code, optimize images, and use code splitting to reduce the initial bundle size. +* **Optimize Data Fetching:** Fetch only the data that is needed for a specific route. Avoid over-fetching data. +* **Cache Data:** Use HTTP caching or server-side caching to reduce the number of requests to the server. +* **Memoization:** Use `React.memo` or `useMemo` to prevent unnecessary re-renders of components. +* **Debouncing and Throttling:** Use debouncing and throttling to limit the frequency of event handlers, improving performance for user input and animations. + +#### 3.2. Memory Management + +* Avoid memory leaks by properly cleaning up event listeners and subscriptions. +* Use the `useEffect` hook with a cleanup function to unsubscribe from subscriptions when a component unmounts. +* Avoid storing large amounts of data in component state. Consider using a server-side data store or a more efficient data structure. + +#### 3.3. Rendering Optimization + +* Use the `shouldComponentUpdate` lifecycle method (or `React.memo`) to prevent unnecessary re-renders of components. Carefully analyze component re-renders with the React Profiler. +* Virtualize long lists or tables to improve rendering performance. +* Optimize CSS and avoid complex selectors that can slow down rendering. + +#### 3.4. Bundle Size Optimization + +* Use tools like `webpack-bundle-analyzer` or `rollup-plugin-visualizer` to analyze your bundle size and identify areas for optimization. +* Remove unused dependencies and use tree shaking to eliminate dead code. +* Use code splitting to load only the code that is needed for a specific route or component. + +#### 3.5. Lazy Loading + +* Use `React.lazy` to lazily load components that are not immediately needed on page load. This improves the initial load time. +* Use Intersection Observer API to load images or other resources when they are visible in the viewport. + + +### 4. Security Best Practices + +#### 4.1. Common Vulnerabilities + +* **Cross-Site Scripting (XSS):** Prevent XSS attacks by sanitizing user input and escaping HTML entities. +* **Cross-Site Request Forgery (CSRF):** Protect against CSRF attacks by using anti-CSRF tokens in forms and API requests. +* **SQL Injection:** Prevent SQL injection attacks by using parameterized queries or ORMs. +* **Authentication and Authorization Issues:** Implement strong authentication and authorization mechanisms to protect sensitive data and resources. + +#### 4.2. Input Validation + +* Validate all user input on both the client-side and server-side. +* Use a validation library like Zod or Yup to define schemas for your data. +* Sanitize user input to remove potentially malicious characters or code. + +#### 4.3. Authentication and Authorization + +* Use a secure authentication protocol like OAuth 2.0 or OpenID Connect. +* Store user credentials securely using hashing and salting. +* Implement role-based access control (RBAC) to restrict access to sensitive resources. +* Use server-side sessions or cookies for authentication. Avoid storing sensitive data in client-side storage. + +#### 4.4. Data Protection + +* Encrypt sensitive data at rest and in transit. +* Use HTTPS to secure communication between the client and server. +* Protect against data breaches by implementing strong access controls and monitoring for suspicious activity. +* Implement data masking or anonymization to protect sensitive data in development and testing environments. + +#### 4.5. Secure API Communication + +* Use HTTPS to encrypt API communication. +* Implement API rate limiting to prevent abuse. +* Validate API requests and responses to prevent data injection attacks. +* Use authentication tokens to authorize API requests. + + +### 5. Testing Approaches + +#### 5.1. Unit Testing + +* Write unit tests for individual components, utilities, and services. +* Use a testing framework like Jest or Mocha. +* Mock dependencies to isolate the unit under test. +* Test component rendering, state updates, and event handlers. + +#### 5.2. Integration Testing + +* Write integration tests to verify the interaction between different parts of the application. +* Test data flow between components, loaders, and actions. +* Use a testing library like React Testing Library to simulate user interactions. +* Mock external APIs and services to ensure that integration tests are reliable. + +#### 5.3. End-to-End Testing + +* Write end-to-end tests to verify the entire application flow from the user's perspective. +* Use a testing framework like Cypress or Playwright. +* Test user authentication, data input, and navigation. +* Run end-to-end tests in a continuous integration environment to ensure that the application is working as expected. + +#### 5.4. Test Organization + +* Organize tests in a directory structure that mirrors the application code. +* Create separate test files for each component, utility, or service. +* Use descriptive test names to clearly communicate the purpose of each test. +* Keep tests small and focused to improve readability and maintainability. + +#### 5.5. Mocking and Stubbing + +* Use mocking and stubbing to isolate units under test and control their dependencies. +* Use mocking libraries like Jest's `jest.fn()` or Mock Service Worker (MSW) to mock API responses and external services. +* Avoid over-mocking, which can lead to tests that are not representative of the real application. + + +### 6. Common Pitfalls and Gotchas + +#### 6.1. Frequent Mistakes + +* **Incorrectly Using Loaders and Actions:** Understanding the lifecycle and purpose of loaders and actions is crucial. Incorrect use can lead to performance issues and data inconsistencies. +* **Ignoring Server-Side Rendering:** Failing to leverage Remix's server-side rendering capabilities can result in poor SEO and performance. +* **Over-Complicating State Management:** Using complex state management libraries for simple applications can add unnecessary overhead. +* **Not Validating User Input:** Failing to validate user input can lead to security vulnerabilities and data corruption. +* **Using Browser Specific APIs in Server Code**: Only use web standard API that are available in Node.js in server code. + +#### 6.2. Edge Cases + +* **Handling Empty Data Sets:** Properly handle cases where loaders return empty data sets. Display appropriate messages to the user. +* **Dealing with Network Errors:** Implement robust error handling to gracefully handle network errors and API failures. +* **Managing User Sessions:** Implement secure session management to protect user data and prevent unauthorized access. +* **Handling Concurrent Requests:** Be aware of potential race conditions when handling concurrent requests to the server. + +#### 6.3. Version-Specific Issues + +* Stay up-to-date with the latest Remix version and be aware of any breaking changes or bug fixes. +* Consult the Remix documentation and release notes for information about version-specific issues. +* Test your application thoroughly after upgrading to a new version of Remix. + +#### 6.4. Compatibility Concerns + +* Be aware of compatibility issues between Remix and other technologies, such as third-party libraries or server-side environments. +* Test your application thoroughly in different environments to ensure that it is working as expected. +* Use polyfills or shims to address compatibility issues when necessary. + +#### 6.5. Debugging Strategies + +* Use the browser's developer tools to debug client-side code. +* Use server-side logging to track requests, responses, and errors. +* Use a debugger to step through code and inspect variables. +* Use profiling tools to identify performance bottlenecks. + + +### 7. Tooling and Environment + +#### 7.1. Recommended Tools + +* **Code Editor:** VS Code, Sublime Text, or Atom with appropriate extensions for JavaScript/TypeScript, React, and Remix. +* **Browser:** Chrome or Firefox with developer tools for debugging and performance analysis. +* **Testing Framework:** Jest or Mocha. +* **Testing Library:** React Testing Library or Enzyme. +* **Linting:** ESLint with recommended Remix and React rules. +* **Formatting:** Prettier. + +#### 7.2. Build Configuration + +* Use Remix's built-in build configuration for optimal performance. +* Customize the build configuration as needed to optimize bundle size and performance. +* Use environment variables to configure the application for different environments. + +#### 7.3. Linting and Formatting + +* Use ESLint to enforce code style and prevent errors. +* Use Prettier to automatically format code for consistency. +* Configure ESLint and Prettier to work together seamlessly. +* Use a pre-commit hook to run ESLint and Prettier before committing code. + +#### 7.4. Deployment + +* Deploy Remix applications to a serverless environment like Vercel or Netlify for optimal performance and scalability. +* Use a containerization platform like Docker to package and deploy the application. +* Use a CDN to cache static assets and improve delivery speed. + +#### 7.5. CI/CD Integration + +* Use a continuous integration/continuous deployment (CI/CD) pipeline to automate the build, test, and deployment process. +* Use a CI/CD platform like GitHub Actions or GitLab CI. +* Automate code linting, formatting, and testing in the CI/CD pipeline. +* Automate deployment to different environments in the CI/CD pipeline. \ No newline at end of file diff --git a/.cursor/rules/requests.mdc b/.cursor/rules/requests.mdc new file mode 100644 index 000000000..75a910d26 --- /dev/null +++ b/.cursor/rules/requests.mdc @@ -0,0 +1,267 @@ +--- +description: This rule file outlines best practices for using the Python requests library, covering performance, security, code organization, and testing. +globs: *.py +--- +--- + +# Requests Library Best Practices + +This document provides comprehensive guidelines for using the Python `requests` library effectively. It covers various aspects, including code organization, common patterns, performance considerations, security best practices, testing approaches, common pitfalls, and tooling. + +## Library Information: + +- Name: requests +- Tags: web, python, http-client + +## 1. Code Organization and Structure + +### 1.1 Directory Structure Best Practices: + +* **Simple Scripts:** For simple, single-file scripts, the structure is less critical. Keep related resources (e.g., configuration files, data files) in the same directory. +* **Larger Projects:** For larger projects: + + my_project/ + ├── src/ + │ ├── __init__.py + │ ├── api_client.py # Contains requests-related functions + │ ├── models.py # Data models for API responses + │ ├── utils.py # Utility functions + │ └── config.py # Configuration settings + ├── tests/ + │ ├── __init__.py + │ ├── test_api_client.py + │ └── conftest.py # pytest configuration + ├── README.md + ├── requirements.txt + └── .gitignore + + * `src/`: Contains the main application code. + * `api_client.py`: Houses the `requests` calls, session management, and error handling. + * `models.py`: Defines data classes/namedtuples to represent the structure of API responses, aiding type hinting and validation. + * `tests/`: Contains unit and integration tests. + * `requirements.txt`: Lists project dependencies. + +### 1.2 File Naming Conventions: + +* Use descriptive names for files related to `requests`, such as `api_client.py`, `http_utils.py`, or `_client.py`. +* Follow PEP 8 guidelines: lowercase with underscores (e.g., `get_data.py`). + +### 1.3 Module Organization Best Practices: + +* **Grouping by Functionality:** Organize code into modules based on functionality. For example, a module for authentication, another for data retrieval, and another for error handling. +* **Avoiding Circular Dependencies:** Design modules to minimize dependencies between them and prevent circular imports. + +### 1.4 Component Architecture Recommendations: + +* **Layered Architecture:** Use a layered architecture to separate concerns: + * **Presentation Layer:** (If applicable) Handles user input and displays data. + * **Business Logic Layer:** Orchestrates the application logic and uses the data access layer. + * **Data Access Layer:** Contains the `requests` calls and interacts with external APIs. This layer should abstract away the details of the HTTP client. +* **Dependency Injection:** Use dependency injection to provide the `requests` session or client to components that need it, allowing for easier testing and configuration. + +### 1.5 Code Splitting Strategies: + +* **By API Endpoint:** If your application interacts with multiple API endpoints, consider creating separate modules or classes for each endpoint to improve maintainability. +* **Functional Decomposition:** Split complex tasks into smaller, manageable functions. This promotes reusability and testability. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns Specific to `requests`: + +* **Singleton (for Session):** Use a singleton pattern to manage a single `requests.Session` instance, ensuring connection pooling is used effectively across the application. Be mindful of potential thread-safety issues with singletons in multithreaded environments. +* **Adapter Pattern:** Create an adapter class around the `requests` library to abstract away the underlying HTTP client. This makes it easier to switch to a different library in the future or add custom logic. +* **Retry Pattern:** Implement retry logic with exponential backoff to handle transient network errors and API rate limits. + +### 2.2 Recommended Approaches for Common Tasks: + +* **Fetching Data:** Use `requests.get()` for retrieving data. Handle potential errors using `response.raise_for_status()` and appropriate exception handling. +* **Sending Data:** Use `requests.post()`, `requests.put()`, or `requests.patch()` for sending data. Set the `Content-Type` header appropriately (e.g., `application/json`). +* **Authentication:** Utilize the `auth` parameter for authentication. Use appropriate authentication schemes such as HTTP Basic Auth, Bearer tokens, or OAuth. +* **File Uploads:** Use the `files` parameter to upload files. Provide a dictionary where the keys are the field names and the values are file-like objects or tuples containing the filename and file-like object. + +### 2.3 Anti-patterns and Code Smells to Avoid: + +* **Ignoring Errors:** Not handling exceptions raised by `requests` can lead to unexpected behavior and application crashes. +* **Hardcoding URLs:** Hardcoding URLs makes the code less flexible and harder to maintain. Store URLs in configuration files or environment variables. +* **Not Using Sessions:** Failing to use `requests.Session()` for multiple requests to the same host can result in performance degradation due to repeated connection setups. +* **Disabling SSL Verification Unnecessarily:** Disabling SSL verification (`verify=False`) should only be done when absolutely necessary and with caution, as it can expose the application to man-in-the-middle attacks. Investigate the root cause of SSL verification failures instead of simply disabling it. +* **Excessive Retries without Backoff:** Retrying requests excessively without an exponential backoff strategy can overwhelm the server and worsen the situation. +* **Storing Sensitive Information in Code:** Avoid storing API keys, passwords, and other sensitive information directly in the code. Use environment variables or a secure configuration management system. +* **Not setting timeouts:** Not setting timeouts on requests can lead to your application hanging indefinitely if a server is unresponsive. + +### 2.4 State Management Best Practices: + +* **Stateless API Clients:** Design API clients to be stateless whenever possible. Avoid storing request-specific data within the client object. Pass all necessary data as arguments to the request methods. +* **Session Management:** Use `requests.Session()` to maintain state (e.g., cookies, authentication) across multiple requests. + +### 2.5 Error Handling Patterns: + +* **Catching Exceptions:** Catch `requests.exceptions.RequestException` and its subclasses (e.g., `requests.exceptions.HTTPError`, `requests.exceptions.ConnectionError`, `requests.exceptions.Timeout`) to handle different types of errors. +* **Using `raise_for_status()`:** Call `response.raise_for_status()` to raise an exception for HTTP error codes (4xx and 5xx). +* **Logging Errors:** Log detailed error messages, including the URL, status code, and response content, to aid in debugging. +* **Returning Meaningful Error Responses:** When creating your own APIs that use the `requests` library internally, return informative error responses to the client. +* **Retry Logic:** Implement retry logic for transient errors (e.g., network timeouts, temporary server errors) with exponential backoff. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques: + +* **Using Sessions:** Utilize `requests.Session()` to reuse connections and reduce overhead. +* **Connection Pooling:** The `requests.Session()` object automatically handles connection pooling, which improves performance by reusing existing connections. +* **Streaming Responses:** For large responses, use `stream=True` to process data incrementally and avoid loading the entire response into memory at once. Remember to close the response after processing. +* **Caching Responses:** Consider caching responses to reduce redundant API calls. Use libraries like `requests_cache` to store responses temporarily. +* **Using HTTP/2:** If the server supports HTTP/2, use the `HTTPX` library, which provides both synchronous and asynchronous support for HTTP/2. +* **Compression:** Ensure the server is using compression (e.g., gzip) and that the `Accept-Encoding` header is set appropriately in the request. + +### 3.2 Memory Management Considerations: + +* **Streaming Large Responses:** Use `stream=True` to avoid loading the entire response into memory. +* **Closing Responses:** Close the response object after processing the data to release resources. Use a `try...finally` block or a context manager to ensure the response is always closed. +* **Iterating over Chunks:** When streaming, iterate over the response content in chunks using `response.iter_content()` or `response.iter_lines()` to process data in smaller pieces. + +### 3.3 Bundle Size Optimization: + +* **Minimize Dependencies:** Only include the dependencies that are strictly necessary. + +### 3.4 Lazy Loading Strategies: + +* **Lazy Initialization of Clients:** Initialize the `requests` session or client only when it is first needed, rather than at application startup. This can improve startup time if the API client is not immediately required. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and How to Prevent Them: + +* **Man-in-the-Middle Attacks:** Always use HTTPS to encrypt communication and prevent eavesdropping. Verify SSL certificates unless absolutely necessary to disable verification. +* **Cross-Site Scripting (XSS):** If displaying data from API responses in a web application, sanitize the data to prevent XSS attacks. +* **Server-Side Request Forgery (SSRF):** Avoid constructing URLs based on user input without proper validation. This can prevent attackers from making requests to internal resources. +* **Exposure of Sensitive Information:** Store API keys, passwords, and other sensitive information securely using environment variables or a secrets management system. +* **Denial of Service (DoS):** Implement timeouts and rate limiting to prevent attackers from overwhelming the server with requests. + +### 4.2 Input Validation Best Practices: + +* **Validating Input Data:** Validate all input data before sending it to the API. This can prevent injection attacks and other security vulnerabilities. +* **Sanitizing Input Data:** Sanitize input data to remove potentially malicious characters or code. +* **Using Prepared Statements:** When constructing database queries with data from API responses, use prepared statements to prevent SQL injection attacks. + +### 4.3 Authentication and Authorization Patterns: + +* **Using Secure Authentication Schemes:** Use strong authentication schemes such as OAuth 2.0 or JWT (JSON Web Tokens) to protect API endpoints. +* **Storing Credentials Securely:** Store API keys, passwords, and other credentials securely using environment variables, a secrets management system, or a hardware security module (HSM). +* **Implementing Authorization:** Implement authorization to control which users or applications have access to specific API endpoints. +* **Using HTTPS:** Always use HTTPS to encrypt communication and protect credentials during transmission. + +### 4.4 Data Protection Strategies: + +* **Encrypting Sensitive Data:** Encrypt sensitive data at rest and in transit. +* **Masking Sensitive Data:** Mask sensitive data in logs and error messages. +* **Complying with Data Privacy Regulations:** Ensure compliance with relevant data privacy regulations such as GDPR and CCPA. + +### 4.5 Secure API Communication: + +* **HTTPS:** Always use HTTPS for all API communication. +* **TLS Versions:** Ensure that the server and client support the latest TLS versions (TLS 1.2 or 1.3) and disable older, insecure versions (SSLv3, TLS 1.0, TLS 1.1). +* **Cipher Suites:** Configure the server to use strong cipher suites that provide forward secrecy and authenticated encryption. +* **Certificate Pinning:** Consider using certificate pinning to prevent man-in-the-middle attacks by verifying the server's SSL certificate against a known good certificate. + +## 5. Testing Approaches + +### 5.1 Unit Testing Strategies: + +* **Testing Individual Functions:** Write unit tests for individual functions that make `requests` calls. +* **Mocking `requests`:** Use mocking libraries like `unittest.mock` or `pytest-mock` to mock the `requests` library and isolate the code being tested. +* **Testing Error Handling:** Write unit tests to verify that the code handles different types of errors correctly. +* **Parametrizing Tests:** Use parametrization to test the same function with different inputs and expected outputs. +* **Test Data:** Create a set of test data (e.g., JSON files) to simulate API responses. + +### 5.2 Integration Testing Approaches: + +* **Testing API Client Integrations:** Write integration tests to verify that the API client integrates correctly with other components of the application. +* **Using a Test API Server:** Set up a test API server (e.g., using Flask or FastAPI) to simulate the real API and control the responses. +* **Verifying Data Integrity:** Verify that the data returned by the API is processed correctly and stored in the database or other data store. + +### 5.3 End-to-End Testing Recommendations: + +* **Testing Complete Workflows:** Write end-to-end tests to verify that complete workflows involving the API client work correctly. +* **Using Automation Tools:** Use automation tools like Selenium or Playwright to automate end-to-end tests. + +### 5.4 Test Organization Best Practices: + +* **Separating Tests:** Separate unit tests, integration tests, and end-to-end tests into different directories or files. +* **Using a Test Runner:** Use a test runner like pytest to discover and run tests automatically. +* **Following Test Naming Conventions:** Follow consistent test naming conventions to make it easier to understand the purpose of each test. + +### 5.5 Mocking and Stubbing Techniques: + +* **Mocking the `requests` Library:** Use mocking to replace the `requests` library with a mock object that returns predefined responses. +* **Using Mock Objects:** Create mock objects to simulate the behavior of external APIs. +* **Stubbing Functions:** Use stubbing to replace functions with simplified versions that return predefined values. +* **Using Context Managers for Mocking:** Use context managers to temporarily replace objects with mock objects during tests. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes Developers Make: + +* **Not Handling Exceptions:** Ignoring exceptions raised by `requests` can lead to unexpected behavior and application crashes. +* **Not Using Sessions:** Failing to use `requests.Session()` for multiple requests to the same host can result in performance degradation. +* **Not Setting Timeouts:** Not setting timeouts on requests can lead to your application hanging indefinitely if a server is unresponsive. +* **Disabling SSL Verification Unnecessarily:** Disabling SSL verification (`verify=False`) should only be done when absolutely necessary and with caution. +* **Not Handling Rate Limits:** Not handling API rate limits can lead to the application being blocked by the API provider. +* **Incorrectly Handling Character Encoding:** Failing to properly handle character encoding when dealing with non-ASCII characters in request or response bodies. + +### 6.2 Edge Cases to Be Aware Of: + +* **Handling Redirects:** Be aware of how `requests` handles redirects by default (following them). If you need to control redirect behavior, use the `allow_redirects` parameter. +* **Dealing with Large Payloads:** When sending or receiving large payloads, use streaming to avoid memory issues. +* **Handling Keep-Alive Connections:** Be aware that `requests` uses keep-alive connections by default, which can lead to issues if the server closes the connection unexpectedly. +* **Proxy Configuration:** Properly configure proxies if your application needs to access the internet through a proxy server. + +### 6.3 Version-Specific Issues: + +* **Changes in API:** Be aware of changes in the `requests` API between different versions. Consult the release notes for each version to identify any breaking changes. +* **Dependency Conflicts:** Be aware of potential dependency conflicts between `requests` and other libraries in your project. Use a virtual environment to isolate dependencies. + +### 6.4 Compatibility Concerns: + +* **Python Versions:** Ensure that the version of `requests` you are using is compatible with the version of Python you are using. +* **Operating Systems:** Be aware of potential compatibility issues between `requests` and different operating systems (e.g., Windows, Linux, macOS). + +### 6.5 Debugging Strategies: + +* **Logging Requests and Responses:** Log detailed information about requests and responses, including URLs, headers, and bodies, to aid in debugging. +* **Using Debugging Tools:** Use debugging tools such as `pdb` or `ipdb` to step through the code and inspect variables. +* **Capturing Network Traffic:** Use network traffic analysis tools like Wireshark or tcpdump to capture and analyze network traffic. +* **Using `httpbin.org` for Testing:** Use the `httpbin.org` service to test different types of requests and responses. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools: + +* **Virtual Environments:** Use virtual environments (e.g., `venv`, `virtualenv`) to isolate project dependencies. +* **pip:** Use `pip` to install and manage dependencies. +* **IDEs:** Use an IDE such as Visual Studio Code, PyCharm, or Sublime Text to write and debug code. +* **Debugging Tools:** Use debugging tools such as `pdb` or `ipdb` to step through the code and inspect variables. + +### 7.2 Build Configuration Best Practices: + +* **Using `requirements.txt`:** Use a `requirements.txt` file to specify project dependencies. +* **Using `setup.py`:** Use a `setup.py` file to define the project metadata and package the code for distribution. +* **Using a Build System:** Consider using a build system such as Make or Poetry to automate the build process. + +### 7.3 Linting and Formatting Recommendations: + +* **Using a Linter:** Use a linter such as `flake8` or `pylint` to identify potential code quality issues. +* **Using a Formatter:** Use a formatter such as `black` or `autopep8` to automatically format the code according to PEP 8 guidelines. +* **Configuring the IDE:** Configure the IDE to use the linter and formatter automatically. + +### 7.4 Deployment Best Practices: + +* **Using a Production Environment:** Deploy the application to a production environment that is separate from the development environment. +* **Using a Process Manager:** Use a process manager such as Supervisor or systemd to manage the application process. +* **Using a Load Balancer:** Use a load balancer to distribute traffic across multiple instances of the application. +* **Monitoring the Application:** Monitor the application for errors and performance issues. + +### 7.5 CI/CD Integration Strategies: + +* **Using a CI/CD System:** Use a CI/CD system such as Jenkins, GitLab CI, GitHub Actions, or CircleCI to automate the build, test, and deployment processes. +* **Running Tests Automatically:** Configure the CI/CD system to run tests automatically on every commit. +* **Deploying Automatically:** Configure the CI/CD system to deploy the application automatically to the production environment after the tests have passed. \ No newline at end of file diff --git a/.cursor/rules/rich.mdc b/.cursor/rules/rich.mdc new file mode 100644 index 000000000..e2ba5be54 --- /dev/null +++ b/.cursor/rules/rich.mdc @@ -0,0 +1,160 @@ +--- +description: Comprehensive best practices and coding standards for the Rich library, focusing on code quality, performance, and maintainability within Python terminal applications. +globs: *.py +--- +- **General Guidelines** + - Adhere to PEP 8 coding style guidelines for Python code, emphasizing readability and consistency. + - Limit lines to a maximum of 79 characters to enhance readability across different environments. + - Prioritize code clarity and simplicity, making it easy to understand and maintain. + - Use UTF-8 encoding for source files to ensure compatibility with a wide range of characters. + +- **Installation and Environment** + - Use `uv` for installing dependencies to improve speed and reliability, if appropriate. + - Specify Python 3.12 or later to leverage the latest language features and performance improvements. (If applicable for the specific features of rich you are using) + +- **Code Organization and Structure** + + - **Directory Structure:** Follow a logical directory structure. + + project_root/ + ├── src/ + │ ├── main.py # Entry point of the application + │ ├── utils/ + │ │ ├── __init__.py + │ │ ├── helper_functions.py + │ ├── modules/ + │ │ ├── __init__.py + │ │ ├── module_a.py # Rich-related components + │ │ ├── module_b.py + ├── tests/ + │ ├── __init__.py + │ ├── test_main.py + │ ├── test_module_a.py + ├── README.md + ├── pyproject.toml # Or requirements.txt + + - **File Naming:** Use descriptive lowercase names with underscores (e.g., `console_output.py`). + - **Module Organization:** Group related functionalities into separate modules (e.g., `rich_display.py`, `data_formatting.py`). + - **Component Architecture:** Design modular components with clear interfaces for reusability and maintainability. + - **Code Splitting:** Break down large files into smaller, more manageable pieces based on functionality. Consider lazy loading of less-frequently used modules. + +- **Coding Style and Best Practices** + + - **Indentation:** Use 4 spaces for indentation. + - **Blank Lines:** Separate top-level functions and classes with two blank lines, and methods within a class with one blank line. + - **Imports:** + - Group imports in the following order: + 1. Standard library imports + 2. Third-party library imports (including Rich) + 3. Local application imports + - Use absolute imports for clarity, unless relative imports significantly improve readability within a package. + - Avoid wildcard imports (`from module import *`). + - **Naming Conventions:** + - Use lowercase with underscores for function and variable names (`my_variable`, `my_function`). + - Use CapWords for class names (`MyClass`). + - Use UPPER_CASE_WITH_UNDERSCORES for constants (`MAX_VALUE`). + - **String Quotes:** Use double quotes consistently for strings, especially for docstrings. + +- **Rich Library Specific Best Practices** + + - **Console Instantiation:** Create a single `Console` instance for your application and reuse it throughout. + python + from rich.console import Console + + console = Console() + + def my_function(): + console.print("Hello, [bold red]World![/bold red]") + + - **Styling:** Use Rich's markup system for styling text. Refer to the Rich documentation for available styles and their usage. + python + console.print("[link=https://example.com]Click here[/link] to visit example.com") + + - **Tables:** Utilize the `Table` class for structured data display. Configure columns and rows appropriately. + python + from rich.table import Table + + table = Table(title="My Data") + table.add_column("Name", style="cyan", no_wrap=True) + table.add_column("Age", style="magenta") + table.add_row("Alice", "30") + table.add_row("Bob", "25") + console.print(table) + + - **Progress Bars:** Employ the `Progress` class for tracking long-running tasks. Configure the progress bar to accurately reflect the task's progress. + python + from rich.progress import Progress + import time + + with Progress() as progress: + task1 = progress.add_task("[red]Downloading...", total=1000) + task2 = progress.add_task("[green]Processing...", total=100) + + while not progress.finished: + progress.update(task1, advance=0.5) + progress.update(task2, advance=0.1) + time.sleep(0.01) #Simulate some work + + - **Inspect:** Use `console.inspect()` for debugging and understanding objects. This is invaluable for exploring Rich's own classes and objects, or any object in your application. + python + from rich.panel import Panel + + panel = Panel("Hello, World!") + console.inspect(panel, methods=True) + + +- **Common Patterns and Anti-patterns** + - **Design Patterns:** Employ appropriate design patterns such as Factory Pattern for creating Rich objects or Strategy Pattern for different output styles. + - **Recommended Approaches:** Use Rich's features for displaying data structures (lists, dictionaries) in a human-readable format. + - **Anti-patterns:** Avoid directly printing to the console using `print()` when you should be using `console.print()` to take advantage of Rich's features. Avoid excessive nesting of Rich markup, which can reduce readability. + - **State Management:** Manage the state of Rich components appropriately, especially when creating dynamic displays. Avoid mutating objects directly without updating the console output. + - **Error Handling:** Handle exceptions gracefully and use Rich's features to display error messages to the user in a clear and informative way. + python + try: + result = 1 / 0 + except Exception as e: + console.print_exception(show_locals=True) + + +- **Performance Considerations** + - **Optimization Techniques:** Use Rich's built-in caching mechanisms to avoid re-rendering the same content repeatedly. Minimize the use of computationally expensive Rich features when performance is critical. + - **Memory Management:** Be mindful of memory usage when displaying large amounts of data with Rich. Consider using generators or iterators to process data in chunks. + - **Rendering Optimization:** If applicable, profile your Rich-based application to identify rendering bottlenecks. Optimize the rendering of complex Rich elements by simplifying the markup or reducing the number of elements. + - **Bundle Size Optimization:** Not directly applicable since rich is primarily server-side for terminal apps but for web integrated terminals, ensure only necessary Rich dependencies are bundled. + - **Lazy Loading:** Not directly applicable, but relevant for other parts of your application that might interact with Rich. + +- **Security Best Practices** + - **Vulnerabilities:** Be aware of potential vulnerabilities related to untrusted input. Sanitize any user-provided text before displaying it with Rich to prevent markup injection attacks. + - **Input Validation:** Validate any input before using it in Rich's markup to prevent unexpected behavior or security issues. + - **Authentication and Authorization:** Not directly applicable to Rich, but ensure proper authentication and authorization mechanisms are in place for any data displayed by Rich. + - **Data Protection:** Protect sensitive data by masking or redacting it before displaying it with Rich. Utilize Rich's styling options to emphasize sensitive data that requires special attention. + +- **Testing Approaches** + - **Unit Testing:** Write unit tests for individual Rich components to ensure they function correctly. Mock the `Console` object to isolate the component under test. + - **Integration Testing:** Test the integration of Rich components with other parts of your application to ensure they work together seamlessly. + - **End-to-end Testing:** Verify the overall behavior of your Rich-based application by simulating user interactions and validating the output displayed on the console. + - **Test Organization:** Organize your tests into logical modules that correspond to the structure of your application. + - **Mocking and Stubbing:** Use mocking and stubbing techniques to isolate components and simulate dependencies during testing. The `unittest.mock` module provides tools for creating mock objects. + +- **Common Pitfalls and Gotchas** + - **Frequent Mistakes:** Forgetting to import the `Console` class or using `print()` instead of `console.print()`. Overcomplicating Rich markup, leading to unreadable code. + - **Edge Cases:** Handling Unicode characters correctly. Dealing with terminals that have limited color support. Handling very large data sets, which can cause memory issues or performance problems. + - **Version-Specific Issues:** Being aware of breaking changes or new features in different versions of Rich. Consult the Rich changelog for details. + - **Compatibility Concerns:** Ensuring compatibility between Rich and other terminal libraries or frameworks you are using. + - **Debugging:** Using `console.inspect()` to explore objects and identify issues. Setting breakpoints and stepping through the code to understand the flow of execution. + +- **Tooling and Environment** + - **Recommended Tools:** VS Code with the Python extension, PyCharm, or any other IDE that supports Python development. Consider using a Rich-specific plugin if available. + - **Build Configuration:** Use `pyproject.toml` (preferred) or `requirements.txt` to manage project dependencies, including Rich. + - **Linting and Formatting:** Use Pylint, Flake8, and Black to enforce code style guidelines and catch potential errors. + - **Deployment:** Ensure the target environment has Python and Rich installed. Consider using a virtual environment to isolate dependencies. + - **CI/CD Integration:** Integrate Rich-based applications into your CI/CD pipeline to automate testing and deployment. Use tools like Jenkins, GitLab CI, or GitHub Actions. + +- **Additional Notes** + - Consult the official Rich documentation (https://rich.readthedocs.io) for the most up-to-date information and examples. + - Explore Rich's examples and demonstrations to learn more about its capabilities. + - Contribute to the Rich community by reporting bugs, suggesting features, or submitting pull requests. + +- **References** + - PEP 8: Style Guide for Python Code (https://peps.python.org/pep-0008/) + - Rich Documentation: (https://rich.readthedocs.io) \ No newline at end of file diff --git a/.cursor/rules/riverpod.mdc b/.cursor/rules/riverpod.mdc new file mode 100644 index 000000000..26350438b --- /dev/null +++ b/.cursor/rules/riverpod.mdc @@ -0,0 +1,150 @@ +--- +description: Enforces Riverpod library best practices for Flutter applications. This rule provides guidance on code organization, performance, testing, and common pitfalls when using Riverpod. +globs: *.dart +--- +- **Core Principles:** + - **Provider Initialization:** Avoid initializing providers within widgets. Providers should initialize themselves, managing their own state and dependencies independently. This ensures proper lifecycle management and prevents unexpected behavior during widget rebuilds. Specifically, do not initialize providers inside `build()` methods or widget constructors. + - **Provider Usage:** Primarily use `Provider` as the foundation for your state management. Avoid legacy solutions like `ChangeNotifierProvider`, `StateNotifierProvider`, and `StateProvider` unless there are compelling reasons to use them, such as compatibility with existing code. Prefer `StateProvider` or `NotifierProvider` (or `AsyncNotifierProvider`) for mutable state. + - **Immutability:** Strive for immutability in your state management where possible. Immutable data structures simplify debugging and prevent unexpected side effects. Use `freezed` or similar libraries to help enforce immutability. + +- **Code Organization and Structure:** + - **Directory Structure:** Adopt a feature-based directory structure. Group related files (providers, widgets, models, services) into dedicated directories representing specific features or modules. For example: + + + lib/ + ├── features/ + │ ├── authentication/ + │ │ ├── data/ + │ │ │ ├── auth_repository.dart + │ │ │ └── models/ + │ │ │ └── user.dart + │ │ ├── presentation/ + │ │ │ ├── login_screen.dart + │ │ │ ├── login_controller.dart + │ │ │ └── providers.dart + │ ├── home/ + │ │ ├── data/ + │ │ ├── presentation/ + │ │ └── providers.dart + ├── core/ + │ ├── providers.dart # Global providers + │ └── services/ + │ └── api_service.dart + └── main.dart + + - **File Naming Conventions:** Use descriptive and consistent file names. For example: + - `feature_name_screen.dart` (for widgets/screens) + - `feature_name_service.dart` (for services) + - `feature_name_repository.dart` (for repositories) + - `feature_name_provider.dart` (for provider definitions) + - **Module Organization:** Break down your application into loosely coupled modules. Each module should have a clear responsibility and a well-defined interface. Use dependency injection to manage dependencies between modules, leveraging Riverpod's provider system. + - **Component Architecture:** Design your UI with reusable components. Extract common UI elements into separate widgets to promote code reuse and maintainability. Favor composition over inheritance. + - **Code Splitting Strategies:** Implement code splitting to reduce the initial load time of your application. Use Flutter's lazy loading capabilities and consider splitting your application into smaller modules that can be loaded on demand. Leverage `flutter_modular` or similar packages for advanced module management. + +- **Common Patterns and Anti-patterns:** + - **Design Patterns:** + - **Repository Pattern:** Abstract data access logic behind repositories. Repositories handle data retrieval and persistence, isolating the rest of the application from the specifics of data sources (e.g., local storage, API). + - **Service Pattern:** Encapsulate business logic into services. Services perform specific tasks or operations, such as authentication, data processing, or API integration. Providers can depend on services to provide data or perform actions. + - **Provider as State Holder:** Utilize Riverpod's providers to hold and manage application state. Choose the appropriate provider type based on the state's characteristics (e.g., `StateProvider` for simple mutable state, `NotifierProvider` for complex state with side effects, `FutureProvider` or `StreamProvider` for asynchronous data). + - **Recommended Approaches:** + - **Asynchronous Data Handling:** Use `FutureProvider` and `StreamProvider` to handle asynchronous data. Display loading indicators while data is being fetched and handle errors gracefully. + - **State Updates:** Use `state = newState` for `StateProvider` and `NotifierProvider`. Use `AsyncValue` to represent the different states of an asynchronous operation (loading, data, error). Use `.when()` to handle different states in your UI. + - **Side Effects:** Isolate side effects (e.g., API calls, database updates) in dedicated services or repositories. Avoid performing side effects directly within widgets or providers. + - **Anti-patterns and Code Smells:** + - **Over-reliance on `ChangeNotifier`:** In most cases, `StateProvider` or `NotifierProvider` offer more straightforward and type-safe solutions than `ChangeNotifier`. + - **Initializing Providers in Widgets:** This leads to provider recreation on every widget rebuild, impacting performance and potentially causing unexpected behavior. + - **Directly Mutating State:** Avoid directly mutating state managed by providers. Instead, use the `state` setter for `StateProvider` or update the state within the `build` method of a `Notifier` / `AsyncNotifier`. + - **Complex Logic in Widgets:** Keep widgets focused on UI rendering. Move complex logic to services, repositories, or providers. + - **State Management Best Practices:** + - **Single Source of Truth:** Ensure that each piece of state has a single, authoritative source. Avoid duplicating state across multiple providers or widgets. + - **Scoped Providers:** Use `providerScope` to scope providers to specific parts of the widget tree, avoiding unnecessary rebuilds. This also helps to manage the lifecycle of providers and prevent memory leaks. + - **Error Handling Patterns:** + - **Use `AsyncValue`'s `.when`:** Handle different states of asynchronous operations, using `AsyncValue.when` to display loading indicators, data, and error messages. + - **Global Error Handling:** Implement a global error handling mechanism to catch and log unexpected errors. Consider using a `ProviderListener` to listen for errors and display appropriate error messages. + +- **Performance Considerations:** + - **Optimization Techniques:** + - **`select`:** Use `.select` to only rebuild widgets when specific parts of a provider's state change. This can significantly reduce unnecessary rebuilds. + - **`Consumer` Widget:** Use `Consumer` widgets to rebuild only the parts of the widget tree that depend on a specific provider. + - **`useProvider` Hook:** Use the `useProvider` hook in functional components to access provider values efficiently. + - **Memory Management:** + - **Dispose Providers:** Ensure that providers are properly disposed of when they are no longer needed. Use `ScopedProvider` to automatically dispose of providers when their scope is destroyed. + - **Avoid Memory Leaks:** Be mindful of potential memory leaks, especially when using streams or listeners. Cancel subscriptions and dispose of resources when they are no longer needed. + - **Rendering Optimization:** + - **Minimize Widget Rebuilds:** Use techniques like `const` constructors, `shouldRepaint`, and `useMemoized` to minimize unnecessary widget rebuilds. + - **Optimize Image Loading:** Use cached network images and optimize image sizes to improve rendering performance. + - **Bundle Size Optimization:** + - **Tree Shaking:** Enable tree shaking to remove unused code from your application's bundle. Use `flutter build apk --split-debug-info` for Android. + - **Code Splitting:** Implement code splitting to reduce the initial load time of your application. + - **Lazy Loading Strategies:** Load resources on demand to improve startup time and reduce memory consumption. Use `FutureProvider` and `StreamProvider` to load data lazily. + +- **Security Best Practices:** + - **Common Vulnerabilities:** + - **Data Injection:** Sanitize user input to prevent data injection attacks. + - **Authentication Bypass:** Implement robust authentication and authorization mechanisms to prevent unauthorized access. + - **Insecure Data Storage:** Protect sensitive data by encrypting it and storing it securely. + - **Input Validation:** + - **Validate User Input:** Validate all user input on both the client-side and the server-side to prevent invalid data from being processed. + - **Use Strong Data Types:** Use strong data types to enforce data integrity and prevent type-related errors. + - **Authentication and Authorization Patterns:** + - **Secure Authentication:** Use secure authentication protocols like OAuth 2.0 or JWT to authenticate users. + - **Role-Based Authorization:** Implement role-based authorization to control access to resources based on user roles. + - **Data Protection Strategies:** + - **Encryption:** Encrypt sensitive data both in transit and at rest. + - **Data Masking:** Mask sensitive data in logs and error messages. + - **Secure API Communication:** + - **HTTPS:** Use HTTPS to encrypt communication between the client and the server. + - **API Keys:** Protect API keys and store them securely. + +- **Testing Approaches:** + - **Unit Testing Strategies:** + - **Test Provider Logic:** Write unit tests to verify the logic within your providers. + - **Mock Dependencies:** Use mocking frameworks like `mockito` to isolate your providers from external dependencies. + - **Integration Testing:** + - **Test Provider Interactions:** Write integration tests to verify how providers interact with each other and with other parts of the application. + - **Test Data Flow:** Test the data flow through the application to ensure that data is being processed correctly. + - **End-to-End Testing:** + - **Test User Flows:** Write end-to-end tests to simulate user interactions and verify that the application behaves as expected. + - **Automated UI Tests:** Use tools like `flutter_driver` or `patrol` to automate UI tests. + - **Test Organization:** + - **Organize Tests by Feature:** Organize your tests into directories that correspond to the features they test. + - **Use Descriptive Test Names:** Use descriptive test names that clearly explain what each test is verifying. + - **Mocking and Stubbing:** + - **Use Mocking Frameworks:** Use mocking frameworks like `mockito` to create mock objects for testing purposes. + - **Create Test Doubles:** Create test doubles to replace real dependencies with simplified versions that are easier to test. + +- **Common Pitfalls and Gotchas:** + - **Frequent Mistakes:** + - **Forgetting to dispose of providers:** Leads to memory leaks. + - **Incorrect provider scope:** Leads to unexpected widget rebuilds. + - **Not handling asynchronous errors:** Leads to unhandled exceptions. + - **Edge Cases:** + - **Provider lifecycle management:** Understand when providers are created and disposed of. + - **Concurrency issues:** Be aware of potential concurrency issues when working with asynchronous data. + - **Version-Specific Issues:** + - **Breaking changes:** Be aware of breaking changes in new versions of Riverpod. + - **Deprecated features:** Avoid using deprecated features. + - **Compatibility Concerns:** + - **Flutter version compatibility:** Ensure that your version of Riverpod is compatible with your version of Flutter. + - **Package dependencies:** Resolve version conflicts between Riverpod and other packages. + - **Debugging Strategies:** + - **Use the Riverpod DevTools:** Use the Riverpod DevTools to inspect provider state and dependencies. + - **Logging:** Add logging statements to your providers to track their behavior. + +- **Tooling and Environment:** + - **Recommended Development Tools:** + - **VS Code or IntelliJ IDEA:** Use a powerful IDE with Flutter and Dart support. + - **Riverpod DevTools:** Install the Riverpod DevTools extension for your IDE to inspect provider state and dependencies. + - **Build Configuration:** + - **Use `flutter build`:** Use the `flutter build` command to build your application. + - **Configure Build Flavors:** Use build flavors to create different versions of your application for different environments (e.g., development, staging, production). + - **Linting and Formatting:** + - **Use `flutter analyze`:** Use the `flutter analyze` command to analyze your code for potential problems. + - **Use `dart format`:** Use the `dart format` command to format your code according to the Dart style guide. + - **Configure your linter**: Configure your linter to enforce Riverpod best practices. + - **Deployment Best Practices:** + - **Use CI/CD:** Use CI/CD pipelines to automate the build, test, and deployment process. + - **Configure your environments**: Configure your different environments (development, staging, production) and their secrets properly. Using `flutter_dotenv` or similar. + - **CI/CD Integration:** + - **Integrate with GitHub Actions or GitLab CI:** Integrate your CI/CD pipeline with GitHub Actions or GitLab CI. + - **Automate Testing and Deployment:** Automate the testing and deployment process to ensure that your application is always up-to-date. \ No newline at end of file diff --git a/.cursor/rules/rocket.mdc b/.cursor/rules/rocket.mdc new file mode 100644 index 000000000..8177c4c6b --- /dev/null +++ b/.cursor/rules/rocket.mdc @@ -0,0 +1,113 @@ +--- +description: Comprehensive guidelines for developing robust and maintainable web applications with the Rocket web framework, covering code organization, security, performance, testing, and more. +globs: *.rs +--- +- **Code Organization and Structure:** + - **Directory Structure:** Organize your project with a clear and maintainable directory structure. A common pattern is: + - `src/`: Contains the main source code. + - `main.rs`: Entry point of the application. + - `lib.rs`: Defines library components. + - `routes/`: Contains route handlers. + - `mod.rs`: Defines and exports route modules. + - `resource_name.rs`: Individual route files. + - `models/`: Defines data structures and database interactions. + - `mod.rs`: Defines and exports model modules. + - `data_model.rs`: Individual model definitions. + - `utils/`: Utility functions and modules. + - `mod.rs`: Defines and exports utility modules. + - `helper_functions.rs`: Individual utility function files. + - `config/`: Configuration files and settings. + - `static/`: Static assets like CSS, JavaScript, and images. + - `templates/`: Templates for rendering dynamic content (using templating engines like Handlebars or Tera). + - `tests/`: Unit and integration tests. + - `migrations/`: Database migrations (if applicable). + - **File Naming Conventions:** Use descriptive and consistent file names. Follow Rust's conventions, such as `snake_case` for file and module names. For example, `user_routes.rs`, `database_connection.rs`. + - **Module Organization:** Break down your code into logical modules. Use `mod.rs` files to define and re-export modules within a directory. + - **Component Architecture:** Design your application using a component-based architecture. This promotes reusability and maintainability. Consider using traits to define interfaces between components. + - **Code Splitting Strategies:** For larger applications, split your code into smaller, manageable crates to improve compilation times and code organization. + +- **Common Patterns and Anti-patterns:** + - **Design Patterns:** + - **Repository Pattern:** Abstract data access logic behind a repository interface. + - **Service Layer Pattern:** Decouple route handlers from business logic using a service layer. + - **State Pattern:** Manage complex state transitions within route handlers or components. + - **Recommended Approaches for Common Tasks:** + - **Database Interactions:** Use an ORM like Diesel or SeaORM to interact with databases. Use connection pooling to efficiently manage database connections. + - **Authentication and Authorization:** Implement authentication using JWT (JSON Web Tokens) or sessions. Use role-based access control (RBAC) for authorization. + - **Input Validation:** Thoroughly validate user input to prevent security vulnerabilities and data corruption. + - **Anti-patterns and Code Smells:** + - **Global Variables:** Avoid using global variables, as they can lead to unexpected side effects and make code harder to reason about. Use dependency injection instead. + - **Tight Coupling:** Minimize dependencies between components. Use interfaces and abstractions to decouple components. + - **Long Functions:** Break down long functions into smaller, more manageable functions. + - **State Management:** Use Rocket's managed state feature (`#[rocket::State]`) to manage application-level state. + - **Error Handling:** Use Rust's `Result` type for error handling. Provide informative error messages to the user. + +- **Performance Considerations:** + - **Optimization Techniques:** + - **Asynchronous Programming:** Use asynchronous programming (`async/await`) to handle concurrent requests efficiently. + - **Caching:** Implement caching to reduce database load and improve response times. + - **Connection Pooling:** Use connection pooling to reuse database connections. + - **Memory Management:** Be mindful of memory allocation and deallocation. Use Rust's ownership and borrowing system to prevent memory leaks and data races. + - **Rendering Optimization:** Optimize your templates to reduce rendering time. Use template caching. + - **Bundle Size Optimization:** Minimize the size of your application's binary. Use release mode compilation (`cargo build --release`). Strip debug symbols. + - **Lazy Loading Strategies:** Load resources on demand to improve initial page load time. + +- **Security Best Practices:** + - **Common Vulnerabilities:** + - **SQL Injection:** Prevent SQL injection by using parameterized queries or an ORM. + - **Cross-Site Scripting (XSS):** Sanitize user input to prevent XSS attacks. + - **Cross-Site Request Forgery (CSRF):** Implement CSRF protection using tokens. + - **Input Validation:** Validate all user input on both the client-side and server-side. + - **Authentication and Authorization:** + - Use strong password hashing algorithms (e.g., bcrypt or Argon2). + - Implement two-factor authentication (2FA). + - Use role-based access control (RBAC) to restrict access to resources. + - **Data Protection:** + - Encrypt sensitive data at rest and in transit. + - Use HTTPS to secure communication between the client and server. + - Store passwords securely using a strong hashing algorithm. + - **Secure API Communication:** + - Use HTTPS for all API communication. + - Implement rate limiting to prevent abuse. + - Use API keys or JWT for authentication. + +- **Testing Approaches:** + - **Unit Testing:** Write unit tests to verify the functionality of individual functions and modules. Use the `#[cfg(test)]` attribute to define test modules. + - **Integration Testing:** Write integration tests to verify the interaction between different parts of your application. Test routes and database interactions. + - **End-to-End Testing:** Write end-to-end tests to verify the complete functionality of your application. Use a testing framework like Selenium or Cypress. + - **Test Organization:** Organize your tests in a clear and maintainable way. Create separate test modules for each component or module. + - **Mocking and Stubbing:** Use mocking and stubbing to isolate units of code during testing. Use a mocking framework like Mockall. + +- **Common Pitfalls and Gotchas:** + - **Frequent Mistakes:** + - **Forgetting to handle errors:** Always handle errors properly using `Result` and provide informative error messages. + - **Not validating user input:** Always validate user input to prevent security vulnerabilities. + - **Using global state:** Avoid using global state, as it can lead to unexpected side effects. + - **Edge Cases:** + - **Handling concurrent requests:** Use asynchronous programming to handle concurrent requests efficiently. + - **Dealing with large files:** Use streaming to process large files without loading them into memory. + - **Handling database connection errors:** Implement retry logic to handle database connection errors. + - **Version-Specific Issues:** Be aware of version-specific issues and compatibility concerns. Consult the Rocket documentation and release notes. + - **Compatibility Concerns:** Ensure that your application is compatible with the target platform and browser versions. + - **Debugging Strategies:** + - Use the `println!` macro for debugging. + - Use a debugger like GDB or LLDB. + - Use logging to track the execution of your application. + +- **Tooling and Environment:** + - **Recommended Development Tools:** + - **Rust toolchain:** Install the Rust toolchain using `rustup`. + - **IDE:** Use an IDE like Visual Studio Code with the Rust Analyzer extension. + - **Database client:** Use a database client like DBeaver or pgAdmin. + - **Build Configuration:** Configure your build using `Cargo.toml`. Specify dependencies, build scripts, and other build settings. + - **Linting and Formatting:** + - Use `rustfmt` for code formatting. + - Use `clippy` for linting. + - Configure your IDE to automatically format and lint your code. + - **Deployment Best Practices:** + - Use a process manager like systemd or PM2 to manage your application. + - Use a reverse proxy like Nginx or Apache to handle incoming requests. + - Use a load balancer to distribute traffic across multiple instances of your application. + - **CI/CD Integration:** + - Use a CI/CD platform like GitHub Actions or GitLab CI to automate your build, test, and deployment process. + - Write tests to ensure that your application is working correctly before deploying. \ No newline at end of file diff --git a/.cursor/rules/ros.mdc b/.cursor/rules/ros.mdc new file mode 100644 index 000000000..11363551d --- /dev/null +++ b/.cursor/rules/ros.mdc @@ -0,0 +1,277 @@ +--- +description: This rule provides comprehensive best practices and coding standards for ROS (Robot Operating System) development, covering code organization, common patterns, performance, security, testing, and tooling. It aims to enhance code quality, maintainability, and interoperability in ROS projects. +globs: *.cpp,*.h,*.py,*.launch,*.msg,*.srv,*.action,*.yaml +--- +# ROS Best Practices and Coding Standards + +This document outlines best practices and coding standards for developing robust, maintainable, and interoperable ROS (Robot Operating System) code. It covers various aspects of ROS development, from code organization to security considerations, based on community best practices and official guidelines. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure Best Practices + +A standard ROS package directory structure helps maintainability and discoverability. Here's a recommended structure: + + +my_package/ +├── CMakeLists.txt # CMake build file +├── package.xml # Package manifest +├── include/ # Header files (C++) +│ └── my_package/ +│ └── my_node.h +├── src/ # Source code (C++ and Python) +│ ├── my_node.cpp +│ └── my_python_node.py +├── scripts/ # Executable scripts (e.g., Python) +│ └── my_script.py +├── msg/ # Message definitions (.msg files) +│ └── MyMessage.msg +├── srv/ # Service definitions (.srv files) +│ └── MyService.srv +├── action/ # Action definitions (.action files) +│ └── MyAction.action +├── launch/ # Launch files (.launch or .launch.py) +│ └── my_launch.launch.py +├── config/ # Configuration files (YAML) +│ └── my_config.yaml +├── urdf/ # URDF files for robot models (.urdf or .xacro) +│ └── my_robot.urdf.xacro +├── rviz/ # RViz configuration files (.rviz) +│ └── my_rviz_config.rviz +├── test/ # Test files +│ ├── my_node_test.cpp +│ └── my_python_node_test.py +└── doc/ # Documentation + └── README.md + + +### 1.2 File Naming Conventions + +- **Packages:** Use underscore-separated lowercase names (e.g., `my_package`). +- **Source Files:** Use underscore-separated lowercase names with `.cpp` or `.py` extension (e.g., `my_node.cpp`, `my_python_node.py`). +- **Header Files:** Use underscore-separated lowercase names with `.h` extension, typically matching the class name (e.g., `my_class.h`). +- **Message/Service/Action Definitions:** Use CamelCase for names (e.g., `MyMessage.msg`, `MyService.srv`, `MyAction.action`). +- **Launch Files:** Use underscore-separated lowercase names with `.launch.py` or `.launch` extension (e.g., `my_launch.launch.py`). +- **Configuration Files:** Use underscore-separated lowercase names with `.yaml` extension (e.g., `my_config.yaml`). + +### 1.3 Module Organization + +- **Logical Grouping:** Group related functionalities into separate modules or classes. +- **Namespaces (C++):** Use namespaces to avoid naming conflicts and organize code (e.g., `namespace my_package`). +- **Python Modules:** Organize code into Python modules within the `src` directory. + +### 1.4 Component Architecture + +- **Node Design:** Design nodes to be modular and focused on specific tasks. Avoid monolithic nodes. +- **Communication:** Use ROS topics, services, and actions appropriately for communication between nodes. Consider using ROS 2 actions over services when a task could take a long time to perform. Using actionlib, nodes can execute goals asynchronously, provide feedback, and be cancelled. +- **Composition:** Use ROS 2 component composition to combine multiple nodes into a single process for improved performance. + +### 1.5 Code Splitting + +- **Libraries:** Extract reusable code into libraries within the package (e.g., in a `lib` subdirectory). +- **Configuration Files:** Use configuration files (YAML) to parameterize node behavior without modifying code. +- **Launch Files:** Separate node execution and configuration into launch files for easy deployment and modification. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns + +- **Publisher-Subscriber:** Use topics for asynchronous, one-to-many communication. +- **Request-Reply:** Use services for synchronous, one-to-one communication requiring a response. +- **Actionlib:** Use actions for long-running tasks with feedback and cancellation capabilities. +- **State Machine:** Implement state machines for complex node behavior. +- **Parameterization:** Externalize configurable parameters to YAML files loaded at runtime. +- **Lifecycle Management:** Implement node lifecycle management for controlled startup and shutdown (ROS 2). +- **Sensor Drivers:** Create dedicated nodes that abstract the hardware and provide a unified interface to the sensor. This also provides a single point of failure and restart for sensor data. + +### 2.2 Recommended Approaches + +- **Configuration:** Prefer using ROS parameters and dynamic reconfigure for node configuration. +- **Logging:** Use `ROS_INFO`, `ROS_WARN`, `ROS_ERROR`, and `ROS_DEBUG` macros (C++) or `rospy.loginfo`, `rospy.logwarn`, `rospy.logerr`, and `rospy.logdebug` (Python) for logging messages. +- **Error Handling:** Use exceptions (C++) or handle errors gracefully in Python with `try...except` blocks. +- **Time Handling:** Use `ros::Time::now()` (C++) or `rospy.Time.now()` (Python) for getting the current ROS time. +- **Transformations:** Use the `tf2` library for managing coordinate transformations. +- **Unit Testing:** Create comprehensive unit tests for individual components. + +### 2.3 Anti-patterns and Code Smells + +- **God Nodes:** Avoid creating nodes that perform too many unrelated tasks. +- **Hardcoded Values:** Avoid hardcoding values in the code; use parameters instead. +- **Spinning Too Much:** Avoid busy-waiting loops; use `ros::Rate` or `rospy.Rate` to control loop frequency. +- **Ignoring Errors:** Handle potential errors and exceptions properly. +- **Global Variables:** Avoid using global variables; use class members or local variables instead. +- **Magic Numbers:** Define constants for important values instead of using magic numbers. + +### 2.4 State Management + +- **Internal State:** Encapsulate node state within class members or local variables. +- **External State:** Use ROS parameters to store and manage persistent state. +- **State Machines:** Employ state machines for complex state transitions and behavior. + +### 2.5 Error Handling + +- **Exceptions (C++):** Use exceptions to signal errors and handle them appropriately. +- **Try...Except (Python):** Use `try...except` blocks to catch and handle exceptions. +- **Return Codes:** In cases where exceptions are not appropriate, use return codes to indicate success or failure. +- **Logging:** Log error messages using `ROS_ERROR` or `rospy.logerr`. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +- **Minimize Data Copying:** Use references or pointers to avoid unnecessary data copying. +- **Efficient Data Structures:** Use appropriate data structures (e.g., `std::vector`, `std::map`) for efficient data storage and retrieval. +- **Pre-allocation:** Pre-allocate memory for frequently used data structures. +- **Node Composition (ROS 2):** Compose multiple nodes into a single process for improved inter-node communication. +- **ZeroMQ (ROS 2):** Use the ZeroMQ transport for low-latency communication. + +### 3.2 Memory Management + +- **Smart Pointers (C++):** Use smart pointers (`std::shared_ptr`, `std::unique_ptr`) to manage memory automatically and prevent memory leaks. +- **Resource Acquisition Is Initialization (RAII):** Use RAII to ensure that resources are released when they are no longer needed. +- **Memory Pools:** Use memory pools for allocating and deallocating memory efficiently. + +### 3.3 Rendering Optimization (If Applicable) + +- **Minimize Polygons:** Use fewer polygons in 3D models. +- **Texture Optimization:** Optimize textures for size and resolution. +- **Culling:** Implement view frustum culling to avoid rendering objects that are not visible. + +### 3.4 Bundle Size Optimization + +- **Minimal Dependencies:** Only include the necessary dependencies in the package manifest. +- **Code Stripping:** Strip unnecessary symbols and debug information from binaries. + +### 3.5 Lazy Loading + +- **On-Demand Loading:** Load resources (e.g., 3D models, textures) only when they are needed. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities + +- **Unvalidated Input:** Protect against injection attacks by validating all input data. +- **Buffer Overflows:** Prevent buffer overflows by using safe string manipulation functions. +- **Denial of Service (DoS):** Protect against DoS attacks by limiting resource consumption. +- **Man-in-the-Middle (MitM):** Use secure communication protocols (e.g., TLS/SSL) to prevent MitM attacks. +- **Remote Code Execution (RCE):** Avoid RCE vulnerabilities by carefully sanitizing user input and preventing the execution of arbitrary code. + +### 4.2 Input Validation + +- **Data Type Validation:** Ensure that input data matches the expected data type. +- **Range Checking:** Verify that input values are within the allowed range. +- **String Sanitization:** Sanitize string input to prevent injection attacks. + +### 4.3 Authentication and Authorization + +- **ROS 2 Security Features:** Utilize the built-in security features of ROS 2, such as authentication and access control. +- **Secure Communication:** Use secure communication protocols (e.g., TLS/SSL) for sensitive data. +- **Access Control:** Implement access control mechanisms to restrict access to sensitive resources. + +### 4.4 Data Protection + +- **Encryption:** Encrypt sensitive data at rest and in transit. +- **Data Masking:** Mask sensitive data to protect it from unauthorized access. +- **Data Auditing:** Audit data access and modifications to detect potential security breaches. + +### 4.5 Secure API Communication + +- **API Keys:** Use API keys for authenticating API requests. +- **Rate Limiting:** Implement rate limiting to prevent abuse of APIs. +- **Input Validation:** Validate all input data to prevent injection attacks. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +- **Individual Components:** Test individual components (e.g., classes, functions) in isolation. +- **Test Frameworks:** Use testing frameworks like Google Test (C++) or `unittest` (Python). +- **Test Coverage:** Aim for high test coverage to ensure that all code paths are tested. + +### 5.2 Integration Testing + +- **Multiple Components:** Test the interaction between multiple components or nodes. +- **ROS Integration Tests:** Use ROS-specific testing tools (e.g., `rostest`) to test ROS nodes and their communication. + +### 5.3 End-to-End Testing + +- **Complete System:** Test the entire ROS system from end to end. +- **Simulation:** Use simulation environments (e.g., Gazebo) for testing in a realistic setting. + +### 5.4 Test Organization + +- **Test Directory:** Create a dedicated `test` directory in the package. +- **Test Naming:** Use descriptive names for test files and functions. +- **Test Suites:** Group related tests into test suites. + +### 5.5 Mocking and Stubbing + +- **Mock Objects:** Use mock objects to simulate the behavior of dependencies. +- **Stub Functions:** Use stub functions to replace complex or external functions with simplified versions. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +- **Incorrect Topic Names:** Double-check topic names for typos or inconsistencies. +- **Missing Dependencies:** Ensure that all dependencies are declared in the package manifest. +- **Incorrect Time Handling:** Be aware of the difference between real-world time and ROS time. +- **Ignoring TF2:** Don't forget to use TF2 for managing coordinate transformations. + +### 6.2 Edge Cases + +- **Network Latency:** Consider the impact of network latency on ROS communication. +- **Message Loss:** Handle potential message loss in unreliable networks. +- **Concurrency Issues:** Be aware of concurrency issues when using multiple threads. + +### 6.3 Version-Specific Issues + +- **ROS 1 vs. ROS 2:** Be aware of the differences between ROS 1 and ROS 2. +- **Package Compatibility:** Ensure that packages are compatible with the ROS distribution being used. + +### 6.4 Compatibility Concerns + +- **Operating Systems:** Test ROS code on different operating systems (e.g., Linux, macOS, Windows). +- **Hardware Platforms:** Ensure that ROS code is compatible with different hardware platforms. + +### 6.5 Debugging Strategies + +- **ROS Logging:** Use `ROS_INFO`, `ROS_WARN`, `ROS_ERROR`, and `ROS_DEBUG` (C++) or `rospy.loginfo`, `rospy.logwarn`, `rospy.logerr`, and `rospy.logdebug` (Python) to log messages for debugging. +- **RViz:** Use RViz for visualizing ROS data and debugging visual issues. +- **Rosbag:** Use Rosbag for recording and playing back ROS messages. +- **GDB (C++):** Use GDB for debugging C++ code. +- **pdb (Python):** Use pdb for debugging Python code. + +## 7. Tooling and Environment + +### 7.1 Recommended Tools + +- **IDE:** VS Code with ROS extension, Eclipse, or other IDEs. +- **Build System:** CMake (ROS 1 and ROS 2). +- **Package Manager:** `apt` (Ubuntu), `pip` (Python). +- **Version Control:** Git. +- **Containerization:** Docker. + +### 7.2 Build Configuration + +- **CMakeLists.txt:** Configure the build process using `CMakeLists.txt`. +- **package.xml:** Declare package dependencies and metadata in `package.xml`. + +### 7.3 Linting and Formatting + +- **C++:** Use `clang-format` for code formatting and `cpplint` for code style checking. +- **Python:** Use `flake8` for code style checking and `black` for code formatting. +- **ROS Lint:** Use `roslint` for checking ROS-specific code style issues. + +### 7.4 Deployment + +- **Debian Packages:** Create Debian packages for easy deployment on Ubuntu systems. +- **Docker Containers:** Package ROS applications into Docker containers for portability. + +### 7.5 CI/CD Integration + +- **GitHub Actions:** Use GitHub Actions for continuous integration and continuous deployment. +- **Jenkins:** Use Jenkins for automated testing and deployment. +- **ROS Buildfarm:** Leverage the ROS buildfarm for automated builds and testing. + +This document provides a comprehensive guide to ROS best practices. Adhering to these guidelines will improve the quality, maintainability, and interoperability of ROS projects. \ No newline at end of file diff --git a/.cursor/rules/ruby.mdc b/.cursor/rules/ruby.mdc new file mode 100644 index 000000000..cd9dc03a2 --- /dev/null +++ b/.cursor/rules/ruby.mdc @@ -0,0 +1,478 @@ +--- +description: Comprehensive best practices for Ruby development, covering code organization, common patterns, performance, security, testing, and tooling. This guide offers actionable advice to improve Ruby code quality, maintainability, and efficiency. +globs: *.rb +--- +# Ruby Best Practices + +This document outlines best practices for Ruby development to ensure code quality, maintainability, performance, and security. It covers various aspects of Ruby projects, from code structure to testing and deployment. + +## 1. Code Organization and Structure + +### Directory Structure Best Practices + +Following a consistent and well-defined directory structure makes Ruby projects easier to navigate and maintain. Here's a recommended structure, especially for Rails applications but adaptable for other Ruby projects: + + +project_root/ +├── app/ +│ ├── models/ +│ ├── controllers/ +│ ├── views/ +│ ├── helpers/ +│ ├── mailers/ +│ ├── assets/ +│ │ ├── stylesheets/ +│ │ ├── javascripts/ +│ │ └── images/ +│ └── jobs/ +├── config/ +│ ├── routes.rb +│ ├── database.yml +│ ├── environments/ +│ │ ├── development.rb +│ │ ├── test.rb +│ │ └── production.rb +│ └── application.rb +├── db/ +│ ├── migrate/ +│ └── seeds.rb +├── lib/ +│ ├── tasks/ +│ └── modules/ +├── log/ +├── public/ +├── test/ +│ ├── models/ +│ ├── controllers/ +│ ├── integration/ +│ ├── fixtures/ +│ └── support/ +├── vendor/ +│ └── cache/ +├── Gemfile +├── Gemfile.lock +├── Rakefile +└── README.md + + +- **app/:** Contains the core application code, organized into models, controllers, views, helpers, mailers, assets, and jobs. +- **config/:** Holds configuration files, including routes, database settings, environment-specific configurations, and the main application configuration. +- **db/:** Contains database-related files, such as migration scripts and seed data. +- **lib/:** Used for custom modules, utility classes, and reusable code that doesn't fit into the app/ directory. +- **log/:** Stores application logs. +- **public/:** Contains static assets like HTML files, images, and compiled JavaScript/CSS. +- **test/:** Holds test files organized in a structure mirroring the app/ directory. +- **vendor/:** Stores third-party code, gems, or libraries. +- **Gemfile:** Specifies the gems (Ruby packages) required by the project. +- **Rakefile:** Defines Rake tasks for automating common development tasks. + +### File Naming Conventions + +Consistent file naming improves readability and maintainability. Follow these conventions: + +- **Models:** Use singular names (e.g., `user.rb`, `product.rb`). +- **Controllers:** Use plural names (e.g., `users_controller.rb`, `products_controller.rb`). +- **Views:** Use corresponding controller and action names (e.g., `users/index.html.erb`, `products/show.html.erb`). +- **Helpers:** Use corresponding controller names with `_helper` suffix (e.g., `users_helper.rb`, `products_helper.rb`). +- **Migrations:** Use descriptive names with timestamps (e.g., `20240101000000_create_users.rb`). +- **Jobs:** Use descriptive names with `_job` suffix (e.g., `send_email_job.rb`). + +### Module Organization + +Modules provide a way to organize code into logical groups and avoid namespace collisions. Use modules to encapsulate related classes and methods: + +ruby +module MyModule + class MyClass + def my_method + # ... + end + end +end + + +- Organize modules in the `lib/modules/` directory. +- Use namespaces to avoid naming conflicts. +- Consider extracting complex logic into separate modules. + +### Component Architecture Recommendations + +For larger applications, consider a component-based architecture. This involves breaking down the application into independent, reusable components: + +- **Define Components:** Identify logical components (e.g., user authentication, payment processing, data analytics). +- **Encapsulate Logic:** Each component should encapsulate its own logic, data, and dependencies. +- **Expose Interfaces:** Define clear interfaces for components to interact with each other. +- **Use Gems or Internal Libraries:** Package components as gems or internal libraries for reuse across multiple projects. + +### Code Splitting Strategies + +Code splitting can improve performance by reducing the amount of code that needs to be loaded at once. Common strategies include: + +- **Lazy Loading:** Load code only when it's needed. This can be achieved using `require` or `autoload`. +- **Conditional Loading:** Load code based on certain conditions (e.g., user roles, feature flags). +- **Service Objects:** Decompose large controllers/models into service objects which can be loaded as required. + +## 2. Common Patterns and Anti-patterns + +### Design Patterns + +- **Singleton:** Ensures that a class has only one instance and provides a global point of access to it. + + ruby + class Configuration + @instance = Configuration.new + + private_class_method :new + + def self.instance + @instance + end + end + +- **Observer:** Defines a one-to-many dependency between objects so that when one object changes state, all its dependents are notified and updated automatically. + + ruby + class Subject + attr_accessor :observers + + def initialize + @observers = [] + end + + def attach(observer) + @observers << observer + end + + def detach(observer) + @observers.delete(observer) + end + + def notify + @observers.each { |observer| observer.update(self) } + end + end + + +- **Factory:** Provides an interface for creating objects without specifying their concrete classes. + + ruby + class AnimalFactory + def self.create(type) + case type + when :dog + Dog.new + when :cat + Cat.new + else + raise "Unknown animal type" + end + end + end + + +- **Strategy:** Defines a family of algorithms, encapsulates each one, and makes them interchangeable. Strategy lets the algorithm vary independently from clients that use it. + + ruby + class PaymentProcessor + def initialize(strategy) + @strategy = strategy + end + + def process_payment(amount) + @strategy.process(amount) + end + end + + +### Recommended Approaches for Common Tasks + +- **Data Validation:** Use ActiveRecord validations to ensure data integrity. +- **Authentication:** Use Devise gem for authentication. +- **Authorization:** Use CanCanCan or Pundit gem for authorization. +- **Background Processing:** Use Sidekiq or Resque for background jobs. +- **API Development:** Use Rails API or Grape gem for building APIs. + +### Anti-patterns and Code Smells + +- **Fat Models:** Models that contain too much business logic. Move logic to service objects or POROs. +- **God Classes:** Classes that do too much. Break them down into smaller, more focused classes. +- **Duplicated Code:** Code that is repeated in multiple places. Follow the DRY principle and extract it into a shared method or module. +- **Long Methods:** Methods that are too long and complex. Break them down into smaller, more focused methods. +- **Magic Numbers:** Hardcoded numerical values that are difficult to understand. Use constants or enums instead. + +### State Management Best Practices + +- **Session:** Use Rails session for storing user-specific data. +- **Cookies:** Use cookies for storing small amounts of data on the client-side. +- **Cache:** Use Rails cache for storing frequently accessed data. +- **Database:** Use the database for storing persistent data. +- **Redis/Memcached:** For complex state data or background job processing. + +### Error Handling Patterns + +- **Exceptions:** Use exceptions for handling unexpected errors. + + ruby + begin + # Code that might raise an exception + rescue SomeException => e + # Handle the exception + Rails.logger.error("Error: #{e.message}") + end + + +- **Error Objects:** Use error objects for handling expected errors. + + ruby + class Result + attr_reader :success, :error, :data + + def initialize(success:, error: nil, data: nil) + @success = success + @error = error + @data = data + end + + def self.success(data: nil) + new(success: true, data: data) + end + + def self.failure(error:) + new(success: false, error: error) + end + end + + +- **Logging:** Log errors for debugging and monitoring. + + ruby + Rails.logger.error("Error: Something went wrong") + + +## 3. Performance Considerations + +### Optimization Techniques + +- **Database Queries:** Optimize database queries by using indexes, eager loading, and avoiding N+1 queries. + + ruby + # N+1 query problem + users = User.all + users.each { |user| puts user.posts.count } # one query per user + + # Eager loading to solve N+1 + users = User.includes(:posts).all + users.each { |user| puts user.posts.count } # only two queries + + +- **Caching:** Use caching to reduce database load and improve response times. + + ruby + Rails.cache.fetch("user_count", expires_in: 1.hour) do + User.count + end + + +- **Code Profiling:** Use code profiling tools to identify performance bottlenecks. +- **Garbage Collection:** Understand how Ruby's garbage collection works and optimize memory usage. +- **Use Efficient Algorithms:** Choose appropriate algorithms and data structures for performance-critical operations. + +### Memory Management + +- **Avoid Memory Leaks:** Be careful about creating objects that are never garbage collected. +- **Use Object Pooling:** Reuse objects instead of creating new ones. +- **Minimize Object Creation:** Reduce the number of objects created in performance-critical sections of code. + +### Bundle Size Optimization + +- **Remove Unused Gems:** Identify and remove gems that are not being used. +- **Use Lightweight Gems:** Choose smaller, more efficient gems when possible. +- **Compress Assets:** Compress CSS, JavaScript, and image assets. + +### Lazy Loading Strategies + +- **Load Associations Lazily:** Use `lazy_load: true` option in associations. +- **Load Code Lazily:** Use `require` or `autoload` to load code only when it's needed. + +## 4. Security Best Practices + +### Common Vulnerabilities + +- **SQL Injection:** Prevent SQL injection by using parameterized queries and avoiding string interpolation in SQL queries. + + ruby + # Unsafe + User.where("email = '#{params[:email]}'") + + # Safe + User.where(email: params[:email]) + + +- **Cross-Site Scripting (XSS):** Prevent XSS by escaping user input and using content security policies. + + erb + <%= sanitize @user.bio %> # Sanitize user input + + +- **Cross-Site Request Forgery (CSRF):** Protect against CSRF by using CSRF tokens. +- **Mass Assignment:** Protect against mass assignment vulnerabilities by using strong parameters. + + ruby + def user_params + params.require(:user).permit(:name, :email, :password, :password_confirmation) + end + + +### Input Validation + +- **Validate User Input:** Always validate user input to ensure it meets expected criteria. +- **Sanitize User Input:** Sanitize user input to remove potentially harmful characters. +- **Use Strong Parameters:** Use strong parameters to protect against mass assignment vulnerabilities. + +### Authentication and Authorization + +- **Use Devise:** Use Devise gem for authentication. +- **Use CanCanCan or Pundit:** Use CanCanCan or Pundit gem for authorization. +- **Implement Role-Based Access Control (RBAC):** Implement RBAC to control access to resources based on user roles. + +### Data Protection + +- **Encrypt Sensitive Data:** Encrypt sensitive data such as passwords and API keys. +- **Use HTTPS:** Use HTTPS to encrypt communication between the client and server. +- **Store Passwords Securely:** Use bcrypt or other secure password hashing algorithms. + +### Secure API Communication + +- **Use API Keys:** Use API keys to authenticate API requests. +- **Implement Rate Limiting:** Implement rate limiting to prevent abuse. +- **Use OAuth:** Use OAuth for secure authorization. + +## 5. Testing Approaches + +### Unit Testing + +- **Test Individual Components:** Write unit tests to test individual components in isolation. +- **Use Mocks and Stubs:** Use mocks and stubs to isolate components from external dependencies. +- **Test Edge Cases:** Test edge cases to ensure code handles unexpected input or conditions. + +### Integration Testing + +- **Test Interactions Between Components:** Write integration tests to test interactions between components. +- **Test External Dependencies:** Test interactions with external dependencies such as databases and APIs. + +### End-to-End Testing + +- **Test Entire Application Flow:** Write end-to-end tests to test the entire application flow. +- **Use Selenium or Capybara:** Use Selenium or Capybara to automate browser interactions. + +### Test Organization + +- **Mirror App Directory:** Organize tests in a directory structure mirroring the app/ directory. +- **Use Descriptive Names:** Use descriptive names for test files and methods. + +### Mocking and Stubbing + +- **Use RSpec Mocks and Stubs:** Use RSpec's mocking and stubbing features to isolate components. + + ruby + # Mocking + allow(User).to receive(:find).with(1).and_return(mock_user) + + # Stubbing + allow(mock_user).to receive(:name).and_return("John Doe") + + +## 6. Common Pitfalls and Gotchas + +### Frequent Mistakes + +- **Not Using ActiveRecord Validations:** Forgetting to validate data in models can lead to data integrity issues. +- **Not Understanding the Scope of Variables:** Misunderstanding variable scope can lead to unexpected behavior. +- **Not Handling Exceptions:** Failing to handle exceptions can cause the application to crash. +- **Not Writing Tests:** Neglecting to write tests can lead to bugs and regressions. + +### Edge Cases + +- **Handling Empty Datasets:** Ensure code handles empty datasets gracefully. +- **Handling Large Datasets:** Optimize code to handle large datasets efficiently. +- **Handling Time Zones:** Be aware of time zone issues when working with dates and times. + +### Version-Specific Issues + +- **Ruby Version Compatibility:** Ensure code is compatible with the target Ruby version. +- **Rails Version Compatibility:** Ensure code is compatible with the target Rails version. + +### Compatibility Concerns + +- **Database Compatibility:** Be aware of compatibility issues between different databases. +- **Operating System Compatibility:** Be aware of compatibility issues between different operating systems. + +### Debugging Strategies + +- **Use Debugger:** Use a debugger to step through code and inspect variables. +- **Use Logging:** Use logging to track application behavior. +- **Read Error Messages:** Carefully read error messages to understand the cause of the error. +- **Use `binding.pry`:** Insert `binding.pry` into your code to pause execution and inspect variables. + +## 7. Tooling and Environment + +### Recommended Tools + +- **Text Editor:** VSCode, Sublime Text, Atom, RubyMine +- **Debugger:** Byebug, RubyMine Debugger +- **Testing Framework:** RSpec, Minitest +- **Code Analysis:** RuboCop, Reek +- **Package Manager:** Bundler +- **Version Manager:** rbenv, rvm + +### Build Configuration + +- **Use Bundler:** Use Bundler to manage gem dependencies. + + ruby + # Gemfile +source 'https://rubygems.org' + +gem 'rails', '~> 7.0' +gem 'pg' + + +- **Specify Ruby Version:** Specify the Ruby version in the `Gemfile`. + + ruby + # Gemfile + ruby '3.2.2' + + +- **Use `.env` Files:** Use `.env` files to store environment variables. + +### Linting and Formatting + +- **Use RuboCop:** Use RuboCop to enforce Ruby style guidelines. +- **Configure RuboCop:** Configure RuboCop to match project coding standards. + + + # .rubocop.yml +AllCops: + TargetRubyVersion: 3.2 + Exclude: + - 'vendor/*' + +Style/Documentation: + Enabled: false + + +- **Use Prettier:** Use Prettier for code formatting. + +### Deployment + +- **Use a Deployment Tool:** Use a deployment tool such as Capistrano or Heroku. +- **Automate Deployment:** Automate the deployment process to reduce errors and improve efficiency. +- **Use a Production Database:** Use a production-grade database such as PostgreSQL or MySQL. +- **Configure a Web Server:** Configure a web server such as Nginx or Apache. + +### CI/CD Integration + +- **Use a CI/CD Tool:** Use a CI/CD tool such as Jenkins, GitLab CI, or CircleCI. +- **Automate Testing:** Automate testing as part of the CI/CD pipeline. +- **Automate Deployment:** Automate deployment as part of the CI/CD pipeline. \ No newline at end of file diff --git a/.cursor/rules/rust.mdc b/.cursor/rules/rust.mdc new file mode 100644 index 000000000..8449254e0 --- /dev/null +++ b/.cursor/rules/rust.mdc @@ -0,0 +1,417 @@ +--- +description: This rule provides comprehensive best practices for Rust development, covering code organization, common patterns, performance, security, testing, pitfalls, and tooling. It aims to guide developers in writing idiomatic, efficient, secure, and maintainable Rust code. +globs: *.rs +--- +# Rust Best Practices + +This document outlines a comprehensive set of best practices for Rust development, covering various aspects from code organization to security and tooling. Adhering to these guidelines will help you write idiomatic, efficient, secure, and maintainable Rust code. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure + +- **`src/`**: Contains all the Rust source code. + - **`main.rs`**: The entry point for binary crates. + - **`lib.rs`**: The entry point for library crates. + - **`bin/`**: Contains source files for multiple binary executables within the same project. Each file in `bin/` will be compiled into a separate executable. + - **`modules/` or `components/`**: (Optional) For larger projects, group related modules or components into subdirectories. Use descriptive names. + - **`tests/`**: Integration tests. (See Testing section below for more details.) + - **`examples/`**: Example code that demonstrates how to use the library. +- **`benches/`**: Benchmark tests (using `criterion` or similar). +- **`Cargo.toml`**: Project manifest file. +- **`Cargo.lock`**: Records the exact versions of dependencies used. **Do not manually edit.** +- **`.gitignore`**: Specifies intentionally untracked files that Git should ignore. +- **`README.md`**: Project documentation, including usage instructions, build instructions, and license information. +- **`LICENSE`**: Contains the project's license. + + +my_project/ +├── Cargo.toml +├── Cargo.lock +├── src/ +│ ├── main.rs # Entry point for a binary crate +│ ├── lib.rs # Entry point for a library crate +│ ├── modules/ +│ │ ├── module_a.rs # A module within the crate +│ │ └── module_b.rs # Another module +│ └── bin/ +│ ├── cli_tool.rs # A separate binary executable +│ └── worker.rs # Another binary executable +├── tests/ +│ └── integration_test.rs # Integration tests +├── benches/ +│ └── my_benchmark.rs # Benchmark tests using Criterion +├── examples/ +│ └── example_usage.rs # Example code using the library +├── README.md +└── LICENSE + + +### 1.2. File Naming Conventions + +- Rust source files use the `.rs` extension. +- Module files (e.g., `module_a.rs`) should be named after the module they define. +- Use snake_case for file names (e.g., `my_module.rs`). + +### 1.3. Module Organization + +- Use modules to organize code into logical units. +- Declare modules in `lib.rs` or `main.rs` using the `mod` keyword. +- Use `pub mod` to make modules public. +- Create separate files for each module to improve readability and maintainability. +- Use `use` statements to bring items from other modules into scope. + +rust +// lib.rs + +pub mod my_module; + +mod internal_module; // Not public + + +rust +// my_module.rs + +pub fn my_function() { + //... +} + + +### 1.4. Component Architecture + +- For larger applications, consider using a component-based architecture. +- Each component should be responsible for a specific part of the application's functionality. +- Components should communicate with each other through well-defined interfaces (traits). +- Consider using dependency injection to decouple components and improve testability. + +### 1.5. Code Splitting Strategies + +- Split code into smaller, reusable modules. +- Use feature flags to conditionally compile code for different platforms or features. +- Consider using dynamic linking (if supported by your target platform) to reduce binary size. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +- **Builder Pattern**: For constructing complex objects with many optional parameters. +- **Factory Pattern**: For creating objects without specifying their concrete types. +- **Observer Pattern**: For implementing event-driven systems. +- **Strategy Pattern**: For selecting algorithms at runtime. +- **Visitor Pattern**: For adding new operations to existing data structures without modifying them. + +### 2.2. Recommended Approaches for Common Tasks + +- **Data Structures**: Use `Vec` for dynamic arrays, `HashMap` for key-value pairs, `HashSet` for unique elements, `BTreeMap` and `BTreeSet` for sorted collections. +- **Concurrency**: Use `Arc` and `Mutex` for shared mutable state, channels for message passing, and the `rayon` crate for data parallelism. +- **Asynchronous Programming**: Use `async` and `await` for writing asynchronous code. +- **Error Handling**: Use the `Result` type for recoverable errors and `panic!` for unrecoverable errors. + +### 2.3. Anti-patterns and Code Smells + +- **Unnecessary Cloning**: Avoid cloning data unless it is absolutely necessary. Use references instead. +- **Excessive `unwrap()` Calls**: Handle errors properly instead of using `unwrap()`, which can cause the program to panic. +- **Overuse of `unsafe`**: Minimize the use of `unsafe` code and carefully review any unsafe code to ensure it is correct. +- **Ignoring Compiler Warnings**: Treat compiler warnings as errors and fix them. +- **Premature Optimization**: Focus on writing clear, correct code first, and then optimize only if necessary. + +### 2.4. State Management + +- **Immutability by Default**: Prefer immutable data structures and functions that return new values instead of modifying existing ones. +- **Ownership and Borrowing**: Use Rust's ownership and borrowing system to manage memory and prevent data races. +- **Interior Mutability**: Use `Cell`, `RefCell`, `Mutex`, and `RwLock` for interior mutability when necessary, but be careful to avoid data races. + +### 2.5. Error Handling + +- **`Result`**: Use `Result` to represent fallible operations. `T` is the success type, and `E` is the error type. +- **`Option`**: Use `Option` to represent the possibility of a missing value. `Some(T)` for a value, `None` for no value. +- **`?` Operator**: Use the `?` operator to propagate errors up the call stack. +- **Custom Error Types**: Define custom error types using enums or structs to provide more context about errors. +- **`anyhow` and `thiserror` Crates**: Consider using the `anyhow` crate for simple error handling and the `thiserror` crate for defining custom error types. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +- **Profiling**: Use profiling tools (e.g., `perf`, `cargo flamegraph`) to identify performance bottlenecks. +- **Benchmarking**: Use benchmarking tools (e.g., `criterion`) to measure the performance of code changes. +- **Zero-Cost Abstractions**: Leverage Rust's zero-cost abstractions, such as iterators, closures, and generics. +- **Inlining**: Use the `#[inline]` attribute to encourage the compiler to inline functions. +- **LTO (Link-Time Optimization)**: Enable LTO to improve performance by optimizing across crate boundaries. + +### 3.2. Memory Management + +- **Minimize Allocations**: Reduce the number of allocations and deallocations by reusing memory and using stack allocation when possible. +- **Avoid Copying Large Data Structures**: Use references or smart pointers to avoid copying large data structures. +- **Use Efficient Data Structures**: Choose the right data structure for the job based on its performance characteristics. +- **Consider `Box` and `Rc`**: `Box` for single ownership heap allocation, `Rc` and `Arc` for shared ownership (latter thread-safe). + +### 3.3. Rendering Optimization + +- **(Relevant if the Rust application involves rendering, e.g., a game or GUI)** +- **Batch draw calls**: Combine multiple draw calls into a single draw call to reduce overhead. +- **Use efficient data structures**: Use data structures that are optimized for rendering, such as vertex buffers and index buffers. +- **Profile rendering performance**: Use profiling tools to identify rendering bottlenecks. + +### 3.4. Bundle Size Optimization + +- **Strip Debug Symbols**: Remove debug symbols from release builds to reduce binary size. +- **Enable LTO**: LTO can also reduce binary size by removing dead code. +- **Use `minisize` Profile**: Create a `minisize` profile in `Cargo.toml` for optimizing for size. +- **Avoid Unnecessary Dependencies**: Only include the dependencies that are absolutely necessary. + +### 3.5. Lazy Loading + +- **Load Resources on Demand**: Load resources (e.g., images, sounds, data files) only when they are needed. +- **Use a Loading Screen**: Display a loading screen while resources are being loaded. +- **Consider Streaming**: Stream large resources from disk or network instead of loading them all at once. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities + +- **Buffer Overflows**: Prevent buffer overflows by using safe indexing methods (e.g., `get()`, `get_mut()`) and validating input sizes. +- **SQL Injection**: Prevent SQL injection by using parameterized queries and escaping user input. +- **Cross-Site Scripting (XSS)**: Prevent XSS by escaping user input when rendering HTML. +- **Command Injection**: Prevent command injection by avoiding the use of `std::process::Command` with user-supplied arguments. +- **Denial of Service (DoS)**: Protect against DoS attacks by limiting resource usage (e.g., memory, CPU, network connections). +- **Integer Overflows**: Use the `checked_add`, `checked_sub`, `checked_mul`, etc. methods on integers to prevent overflows. +- **Use-After-Free**: Rust's ownership system largely prevents this, but be cautious when using `unsafe` code or dealing with raw pointers. +- **Data Races**: Avoid data races by using appropriate synchronization primitives (`Mutex`, `RwLock`, channels). +- **Uninitialized Memory**: Rust generally initializes memory, but `unsafe` code can bypass this. Be careful when working with uninitialized memory. + +### 4.2. Input Validation + +- **Validate All Input**: Validate all input from external sources, including user input, network data, and file contents. +- **Use a Whitelist Approach**: Define a set of allowed values and reject any input that does not match. +- **Sanitize Input**: Remove or escape any potentially dangerous characters from input. +- **Limit Input Length**: Limit the length of input strings to prevent buffer overflows. +- **Check Data Types**: Ensure that input data is of the expected type. + +### 4.3. Authentication and Authorization + +- **Use Strong Passwords**: Require users to create strong passwords and store them securely using a hashing algorithm like Argon2 or bcrypt. +- **Implement Two-Factor Authentication (2FA)**: Add an extra layer of security by requiring users to authenticate with a second factor, such as a code from their phone. +- **Use JSON Web Tokens (JWT)**: Use JWTs for stateless authentication and authorization. +- **Implement Role-Based Access Control (RBAC)**: Define roles with specific permissions and assign users to those roles. +- **Principle of Least Privilege**: Grant users only the minimum necessary privileges to perform their tasks. +- **Regular Audits**: Perform regular security audits of authentication and authorization mechanisms. + +### 4.4. Data Protection + +- **Encrypt Sensitive Data**: Encrypt sensitive data at rest and in transit using strong encryption algorithms like AES-256. +- **Use HTTPS**: Use HTTPS to encrypt communication between the client and the server. +- **Protect API Keys**: Store API keys securely and restrict their usage to authorized users. +- **Handle Secrets Securely**: Use environment variables or dedicated secret management tools (e.g., Vault, AWS Secrets Manager) to store secrets. +- **Avoid Hardcoding Secrets**: Never hardcode secrets directly into the code. +- **Data Masking/Redaction**: Mask or redact sensitive data when logging or displaying it. + +### 4.5. Secure API Communication + +- **Use TLS/SSL**: Enforce TLS/SSL for all API communication. +- **Validate Certificates**: Properly validate server certificates to prevent man-in-the-middle attacks. +- **Rate Limiting**: Implement rate limiting to prevent abuse and DoS attacks. +- **API Versioning**: Use API versioning to maintain backward compatibility and allow for future changes. +- **Input and Output Validation**: Thoroughly validate both input to and output from the API. +- **Content Security Policy (CSP)**: Use CSP headers to prevent XSS attacks. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +- **Test Individual Units of Code**: Write unit tests to verify the correctness of individual functions, modules, and components. +- **Use the `#[test]` Attribute**: Use the `#[test]` attribute to mark functions as unit tests. +- **Use `assert!` and `assert_eq!`**: Use `assert!` and `assert_eq!` macros to check that the code behaves as expected. +- **Test Driven Development (TDD)**: Consider writing tests before writing code. +- **Table-Driven Tests**: Use parameterized tests or table-driven tests for testing multiple scenarios with different inputs. + +rust +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn test_add() { + assert_eq!(add(2, 3), 5); + } +} + + +### 5.2. Integration Testing + +- **Test Interactions Between Components**: Write integration tests to verify that different components of the application work together correctly. +- **Create a `tests/` Directory**: Place integration tests in a `tests/` directory at the root of the project. +- **Use Separate Test Files**: Create separate test files for each integration test. + +### 5.3. End-to-End Testing + +- **Test the Entire Application**: Write end-to-end tests to verify that the entire application works as expected. +- **Use a Testing Framework**: Use a testing framework (e.g., `cucumber`, `selenium`) to automate end-to-end tests. +- **Test User Flows**: Test common user flows to ensure that the application is usable. + +### 5.4. Test Organization + +- **Group Tests by Functionality**: Organize tests into modules and submodules based on the functionality they test. +- **Use Descriptive Test Names**: Use descriptive test names that clearly indicate what the test is verifying. +- **Keep Tests Separate from Production Code**: Keep tests in separate files and directories to avoid cluttering the production code. +- **Run tests frequently**: Integrate tests into your development workflow and run them frequently to catch errors early. + +### 5.5. Mocking and Stubbing + +- **Use Mocking Libraries**: Use mocking libraries (e.g., `mockall`, `mockito`) to create mock objects for testing. +- **Use Traits for Interfaces**: Define traits for interfaces to enable mocking and stubbing. +- **Avoid Global State**: Avoid global state to make it easier to mock and stub dependencies. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +- **Borrowing Rules**: Misunderstanding Rust's borrowing rules can lead to compile-time errors. Ensure you understand ownership, borrowing, and lifetimes. +- **Move Semantics**: Be aware of move semantics and how they affect ownership. Data is moved by default, not copied. +- **Lifetime Annotations**: Forgetting lifetime annotations can lead to compile-time errors. Annotate lifetimes when necessary. +- **Error Handling**: Not handling errors properly can lead to unexpected panics. Use `Result` and the `?` operator to handle errors gracefully. +- **Unsafe Code**: Overusing or misusing `unsafe` code can lead to undefined behavior and security vulnerabilities. + +### 6.2. Edge Cases + +- **Integer Overflow**: Be aware of integer overflow and use checked arithmetic methods to prevent it. +- **Unicode**: Handle Unicode characters correctly to avoid unexpected behavior. +- **File Paths**: Handle file paths correctly, especially when dealing with different operating systems. +- **Concurrency**: Be careful when writing concurrent code to avoid data races and deadlocks. + +### 6.3. Version-Specific Issues + +- **Check Release Notes**: Review the release notes for new versions of Rust to identify any breaking changes or new features that may affect your code. +- **Use `rustup`**: Use `rustup` to manage multiple versions of Rust. +- **Update Dependencies**: Keep your dependencies up to date to take advantage of bug fixes and new features. + +### 6.4. Compatibility Concerns + +- **C Interoperability**: Be careful when interacting with C code to avoid undefined behavior. +- **Platform-Specific Code**: Use conditional compilation to handle platform-specific code. +- **WebAssembly**: Be aware of the limitations of WebAssembly when targeting the web. + +### 6.5. Debugging Strategies + +- **Use `println!`**: Use `println!` statements to print debugging information. +- **Use a Debugger**: Use a debugger (e.g., `gdb`, `lldb`) to step through the code and inspect variables. +- **Use `assert!`**: Use `assert!` to check that the code behaves as expected. +- **Use Logging**: Use a logging library (e.g., `log`, `tracing`) to record debugging information. +- **Clippy**: Use Clippy to catch common mistakes and improve code quality. +- **cargo-flamegraph**: Use cargo-flamegraph to profile and visualize the execution of your code. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +- **Rustup**: For managing Rust toolchains and versions. +- **Cargo**: The Rust package manager and build tool. +- **IDE/Editor**: VS Code with the rust-analyzer extension, IntelliJ Rust, or other editors with Rust support. +- **Clippy**: A linter for Rust code. +- **Rustfmt**: A code formatter for Rust code. +- **Cargo-edit**: A utility for easily modifying `Cargo.toml` dependencies. +- **Cargo-watch**: Automatically runs tests on file changes. +- **lldb or GDB**: Debuggers for Rust applications. + +### 7.2. Build Configuration + +- **Use `Cargo.toml`**: Configure build settings, dependencies, and metadata in the `Cargo.toml` file. +- **Use Profiles**: Define different build profiles for development, release, and testing. +- **Feature Flags**: Use feature flags to conditionally compile code for different platforms or features. + +toml +[package] +name = "my_project" +version = "0.1.0" +edition = "2021" + +[dependencies] +serde = { version = "1.0", features = ["derive"] } + +[dev-dependencies] +rand = "0.8" + +[features] +default = ["serde"] # 'default' feature enables 'serde' +expensive_feature = [] + +[profile.release] +opt-level = 3 +debug = false +lto = true + + +### 7.3. Linting and Formatting + +- **Use Clippy**: Use Clippy to catch common mistakes and enforce coding standards. +- **Use Rustfmt**: Use Rustfmt to automatically format code according to the Rust style guide. +- **Configure Editor**: Configure your editor to automatically run Clippy and Rustfmt on save. +- **Pre-commit Hooks**: Set up pre-commit hooks to run Clippy and Rustfmt before committing code. + +shell +# Install Clippy +rustup component add clippy + +# Run Clippy +cargo clippy + +# Install Rustfmt +rustup component add rustfmt + +# Run Rustfmt +cargo fmt + + +### 7.4. Deployment Best Practices + +- **Build Release Binaries**: Build your application in release mode (`cargo build --release`) to optimize for performance. +- **Minimize Dependencies**: Reduce the number of dependencies to minimize the size of the deployed application. +- **Containerization (Docker)**: Use Docker to create a consistent and reproducible deployment environment. +- **Static Linking**: Consider static linking to create a single executable file. +- **Process Manager (systemd, supervisord)**: Use a process manager to ensure your application restarts automatically if it crashes. + +### 7.5. CI/CD Integration + +- **Use a CI/CD System**: Use a CI/CD system (e.g., GitHub Actions, GitLab CI, Jenkins) to automate the build, test, and deployment process. +- **Run Tests on CI**: Run unit tests, integration tests, and end-to-end tests on CI. +- **Run Linters and Formatters on CI**: Run Clippy and Rustfmt on CI to enforce coding standards. +- **Automate Deployment**: Automate the deployment process to reduce manual effort and errors. + + +# Example GitHub Actions workflow +name: CI + +on: + push: + branches: + - main + pull_request: + +jobs: + build: + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v3 + - uses: actions/cache@v3 + with: + path: | + ~/.cargo/registry + ~/.cargo/git + target + key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }} + + - name: Install Rust + run: rustup default stable + + - name: Build + run: cargo build --verbose + + - name: Run tests + run: cargo test --verbose + + - name: Run clippy + run: cargo clippy -- -D warnings + + +By following these best practices, you can write high-quality Rust code that is efficient, secure, and maintainable. Remember to stay up-to-date with the latest Rust features and best practices to continuously improve your skills and knowledge. \ No newline at end of file diff --git a/.cursor/rules/sanic.mdc b/.cursor/rules/sanic.mdc new file mode 100644 index 000000000..8f2e2afee --- /dev/null +++ b/.cursor/rules/sanic.mdc @@ -0,0 +1,223 @@ +--- +description: This rule file outlines best practices and coding standards for developing Sanic applications, covering code organization, performance, security, testing, and common pitfalls. +globs: *.py +--- +- This document outlines the best practices for developing applications using the Sanic framework. Following these guidelines will lead to more maintainable, performant, and secure applications. + +## 1. Code Organization and Structure + +- **Directory Structure:** Organize your project into logical directories. A recommended structure is: + + project_root/ + ├── app/ + │ ├── __init__.py + │ ├── main.py # Main application file + │ ├── routes.py # Defines application routes + │ ├── models.py # Data models (if using an ORM) + │ ├── views.py # Request handlers and view logic + │ ├── config.py # Application configuration + │ ├── utils.py # Utility functions and helpers + │ ├── middlewares.py # Sanic middlewares + │ ├── exceptions.py # Custom exception definitions + │ └── services/ # Business logic and service layers + ├── tests/ # Unit and integration tests + │ ├── __init__.py + │ ├── conftest.py # Pytest configuration file + │ ├── unit/ + │ └── integration/ + ├── migrations/ # Database migration scripts + ├── requirements.txt # Project dependencies + ├── .env # Environment variables + └── README.md # Project documentation + + +- **File Naming Conventions:** + - Use descriptive and consistent naming. + - `routes.py`, `models.py`, `views.py`, `config.py`, etc. are good examples. + - For service modules, use `service_name_service.py` (e.g., `user_service.py`). + - Test files should mirror the source file names (e.g., `test_routes.py` for `routes.py`). + +- **Module Organization:** + - Keep modules focused and cohesive. + - Avoid large, monolithic modules. + - Group related functionality into separate modules and packages. + - Use relative imports within the `app` package. + +- **Component Architecture:** + - Adopt a layered architecture (e.g., presentation, business logic, data access). + - Use dependency injection to decouple components. + - Employ service objects for complex business logic. + +- **Code Splitting:** + - Defer loading non-critical modules using dynamic imports. + - Split large route handlers into smaller, manageable functions. + +## 2. Common Patterns and Anti-patterns + +- **Design Patterns:** + - **Dependency Injection:** Use dependency injection to manage dependencies and improve testability. + - **Service Layer:** Encapsulate business logic in service objects. + - **Repository Pattern:** Abstract data access logic using repositories. + - **Middleware Pattern:** Use middleware for request pre-processing and response post-processing. + +- **Recommended Approaches:** + - **Configuration:** Use environment variables for configuration and load them with a library like `python-dotenv`. + - **Database Access:** Utilize an ORM (e.g., SQLAlchemy) for interacting with databases. + - **Asynchronous Tasks:** Use `asyncio` or a task queue (e.g., Celery) for background tasks. + - **Response Handling:** Standardize response formats and error handling using custom exception handlers. + +- **Anti-patterns:** + - **Global State:** Avoid using global variables to store application state; use dependency injection instead. + - **Tight Coupling:** Reduce coupling between components through interfaces and dependency injection. + - **Long Functions:** Break down large functions into smaller, more manageable units. + - **Ignoring Exceptions:** Always handle exceptions properly; avoid bare `except` clauses. + +- **State Management:** + - Use application context or dependency injection to manage shared state. + - Avoid using global variables. + - Utilize a dedicated state management library (if needed). + +- **Error Handling:** + - Use custom exception classes for specific error conditions. + - Implement global exception handlers for uncaught exceptions. + - Log errors with detailed context information. + - Return informative error messages to the client. + +## 3. Performance Considerations + +- **Optimization Techniques:** + - **Asynchronous Operations:** Use `async` and `await` for I/O-bound operations to prevent blocking the event loop. + - **Connection Pooling:** Use connection pooling for database connections to reduce overhead. + - **Caching:** Implement caching strategies to reduce database load. + - **Gzip Compression:** Enable Gzip compression for responses to reduce network bandwidth usage. + +- **Memory Management:** + - Avoid creating unnecessary objects. + - Use generators for large datasets to reduce memory consumption. + - Profile your application to identify memory leaks. + +- **Rendering Optimization:** + - Use efficient template engines (e.g., Jinja2) and cache rendered templates. + - Minimize the amount of data passed to templates. + - Use response streaming for very large responses. + +- **Bundle Size Optimization:** + - Not directly applicable to Sanic (as it's a backend framework), but if serving static files, minimize and compress them. + +- **Lazy Loading:** + - Defer loading non-critical modules until they are needed. + - Use lazy imports to improve startup time. + +## 4. Security Best Practices + +- **Common Vulnerabilities:** + - **SQL Injection:** Prevent SQL injection by using parameterized queries or an ORM. + - **Cross-Site Scripting (XSS):** Sanitize user input to prevent XSS attacks. + - **Cross-Site Request Forgery (CSRF):** Implement CSRF protection for state-changing requests. + - **Authentication and Authorization Issues:** Secure your application with proper authentication and authorization mechanisms. + +- **Input Validation:** + - Validate all user input to prevent injection attacks. + - Use data validation libraries (e.g., `marshmallow`) to enforce data types and constraints. + - Escape special characters in user input. + +- **Authentication and Authorization:** + - Use a secure authentication protocol (e.g., OAuth 2.0, JWT). + - Implement role-based access control (RBAC) to manage user permissions. + - Protect sensitive endpoints with authentication and authorization middleware. + +- **Data Protection:** + - Use HTTPS to encrypt data in transit. + - Store sensitive data (e.g., passwords) securely using hashing and salting. + - Use encryption for data at rest. + +- **Secure API Communication:** + - Use HTTPS for all API endpoints. + - Implement rate limiting to prevent abuse. + - Validate API requests and responses against a schema. + +## 5. Testing Approaches + +- **Unit Testing:** + - Use a testing framework like `pytest` or `unittest`. + - Write unit tests for individual components (e.g., models, services, utilities). + - Mock external dependencies to isolate components. + +- **Integration Testing:** + - Test the interaction between different components. + - Use a test database for integration tests. + - Verify that requests are handled correctly and data is persisted. + +- **End-to-End Testing:** + - Use a tool like `Selenium` or `Playwright` to simulate user interactions. + - Test the entire application flow from the client to the database. + - Verify that the application behaves as expected in a production-like environment. + +- **Test Organization:** + - Organize tests into separate directories (e.g., `unit`, `integration`). + - Use descriptive test names. + - Follow the arrange-act-assert pattern. + +- **Mocking and Stubbing:** + - Use mocking libraries (e.g., `unittest.mock`, `pytest-mock`) to isolate components. + - Create stubs for external dependencies that are difficult to test directly. + +## 6. Common Pitfalls and Gotchas + +- **Frequent Mistakes:** + - **Blocking the Event Loop:** Avoid performing long-running or I/O-bound operations in the main event loop. + - **Incorrect Use of `async` and `await`:** Ensure that all asynchronous operations are properly awaited. + - **Not Handling Exceptions:** Always handle exceptions properly to prevent application crashes. + - **Security Vulnerabilities:** Be aware of common security vulnerabilities and take steps to prevent them. + +- **Edge Cases:** + - **Handling Large Requests:** Properly handle large requests to prevent denial-of-service attacks. + - **Graceful Shutdown:** Implement graceful shutdown to avoid data loss. + - **Concurrency Issues:** Be aware of potential concurrency issues when working with shared resources. + +- **Version-Specific Issues:** + - Consult the Sanic documentation and release notes for version-specific issues and migration guides. + +- **Compatibility Concerns:** + - Ensure that your dependencies are compatible with the version of Sanic you are using. + - Test your application with different versions of Python to ensure compatibility. + +- **Debugging Strategies:** + - Use the built-in debugger or an IDE with debugging support. + - Log detailed information about requests and responses. + - Use profiling tools to identify performance bottlenecks. + - Check Sanic application and server logs when troubleshooting issues. + +## 7. Tooling and Environment + +- **Recommended Development Tools:** + - **IDE:** VS Code, PyCharm + - **Package Manager:** pip, uv + - **Virtual Environment:** venv, virtualenv + - **Testing Framework:** pytest + - **Linting:** flake8, pylint + - **Formatting:** black, autopep8 + - **Debugging:** pdb, ipdb + +- **Build Configuration:** + - Use a `requirements.txt` file to manage dependencies. + - Use a `Makefile` or `tox.ini` file to automate build tasks. + - Use a `Dockerfile` to containerize your application. + +- **Linting and Formatting:** + - Use a linter (e.g., `flake8`, `pylint`) to enforce code style. + - Use a formatter (e.g., `black`, `autopep8`) to automatically format your code. + - Configure your IDE to run linters and formatters automatically. + +- **Deployment Best Practices:** + - Use a process manager like `Gunicorn` or `uvicorn` to serve your application. + - Deploy your application behind a reverse proxy like `Nginx`. + - Use a containerization platform like `Docker` to package and deploy your application. + - Monitor your application using a monitoring tool like `Prometheus` or `Sentry`. + +- **CI/CD Integration:** + - Use a CI/CD platform like `GitHub Actions`, `GitLab CI`, or `Jenkins` to automate the build, test, and deployment process. + - Configure your CI/CD pipeline to run linters, formatters, and tests. + - Use a deployment strategy like blue-green deployment or canary deployment. + +By adhering to these best practices, you can develop robust, scalable, and maintainable applications using the Sanic framework. Remember to stay updated with the latest Sanic documentation and community recommendations. \ No newline at end of file diff --git a/.cursor/rules/scikit-image.mdc b/.cursor/rules/scikit-image.mdc new file mode 100644 index 000000000..058c89aee --- /dev/null +++ b/.cursor/rules/scikit-image.mdc @@ -0,0 +1,295 @@ +--- +description: This rule provides guidelines for best practices and coding standards when using the scikit-image library for image processing in Python. It covers code organization, performance, security, testing, and common pitfalls. +globs: *.py +--- +- Always use UV when installing dependencies for faster and more deterministic dependency resolution. +- Always use Python 3.12 or later to leverage the latest language features and performance improvements. +- Utilize classes instead of standalone functions where appropriate for better code organization and encapsulation, especially when dealing with stateful image processing operations. + +## Scikit-image Best Practices and Coding Standards + +This document outlines the best practices and coding standards for using the scikit-image library in Python for image processing. Following these guidelines will help ensure code clarity, maintainability, performance, and security. + +### Library Information: +- Name: scikit-image +- Tags: python, image-processing, scientific-computing + +### 1. Code Organization and Structure: + +- **Directory Structure:** + - Adopt a modular directory structure to organize your scikit-image projects. + - Example: + + project_name/ + ├── data/ # Contains image data (e.g., input images, sample datasets) + ├── src/ # Source code directory + │ ├── __init__.py # Marks src as a Python package + │ ├── modules/ + │ │ ├── __init__.py + │ │ ├── image_io.py # Image input/output related functions + │ │ ├── processing.py # Core image processing algorithms + │ │ ├── segmentation.py # Segmentation algorithms + │ │ └── feature.py # Feature extraction modules + │ ├── utils.py # Utility functions + │ └── main.py # Main application entry point + ├── tests/ # Unit and integration tests + │ ├── __init__.py + │ ├── test_image_io.py + │ ├── test_processing.py + │ └── test_segmentation.py + ├── notebooks/ # Jupyter notebooks for exploration + ├── requirements.txt # Project dependencies + ├── pyproject.toml # Project metadata and build system + └── README.md # Project documentation + + +- **File Naming Conventions:** + - Use descriptive and consistent file names. + - Module files: `image_io.py`, `processing.py`, `segmentation.py` + - Test files: `test_image_io.py`, `test_processing.py` + - Utility files: `utils.py` + +- **Module Organization:** + - Group related functions and classes into modules. + - Avoid monolithic modules; break down large modules into smaller, more manageable ones. + - Use `__init__.py` files to define packages and control namespace exposure. + - Example (in `src/modules/processing.py`): + python + from skimage import filters + from skimage import morphology + import numpy as np + + def apply_threshold(image, threshold_value=128): + """Applies a simple threshold to an image.""" + return image > threshold_value + + def remove_small_objects(binary_image, min_size=100): + """Removes small connected components from a binary image.""" + return morphology.remove_small_objects(binary_image, min_size=min_size) + + +- **Component Architecture:** + - Design components with clear responsibilities and well-defined interfaces. + - Favor composition over inheritance for greater flexibility. + - Use abstract base classes (ABCs) to define interfaces for components. + - Example: + python + from abc import ABC, abstractmethod + + class ImageProcessor(ABC): + @abstractmethod + def process_image(self, image): + pass + + class GrayscaleConverter(ImageProcessor): + def process_image(self, image): + from skimage.color import rgb2gray + return rgb2gray(image) + + +- **Code Splitting Strategies:** + - Decompose complex image processing pipelines into smaller, reusable functions. + - Use generators or iterators for processing large images in chunks. + - Consider using multiprocessing or multithreading for parallel processing of image regions. + - Utilize lazy loading techniques for large image datasets. + +### 2. Common Patterns and Anti-patterns: + +- **Design Patterns:** + - **Factory Pattern:** Use factory functions or classes to create image processing objects. + - **Strategy Pattern:** Implement different image processing algorithms as strategies that can be swapped at runtime. + - **Observer Pattern:** Notify observers when an image processing operation completes. + +- **Recommended Approaches:** + - Use NumPy arrays as the primary data structure for image representation. + - Leverage scikit-image's functional API for modular and composable image processing pipelines. + - Use `img_as_float` or other data type conversion utilities to ensure consistent data types. + - Document image processing functions and classes using docstrings. + +- **Anti-patterns and Code Smells:** + - **Global State:** Avoid using global variables to store image data or processing parameters. + - **Magic Numbers:** Use named constants instead of hardcoded numerical values. + - **Deeply Nested Loops:** Optimize image processing loops using NumPy's vectorized operations. + - **Ignoring Data Types:** Always be aware of the data types of images and intermediate results. + - **Over-Complicating Code:** Aim for simplicity and readability in your image processing code. + +- **State Management:** + - Encapsulate state within classes or data structures. + - Use immutable data structures where possible. + - Avoid modifying image data in-place unless necessary. + +- **Error Handling:** + - Use try-except blocks to handle potential exceptions (e.g., file I/O errors, invalid image formats). + - Log errors and warnings using the `logging` module. + - Provide informative error messages to the user. + - Consider custom exception types for scikit-image related errors. + - Example: + python + import logging + from skimage import io + + logging.basicConfig(level=logging.ERROR) + + def load_image(filepath): + try: + image = io.imread(filepath) + return image + except FileNotFoundError: + logging.error(f"File not found: {filepath}") + return None + except Exception as e: + logging.exception(f"Error loading image: {e}") + return None + + +### 3. Performance Considerations: + +- **Optimization Techniques:** + - Vectorize image processing operations using NumPy's broadcasting and array manipulation features. + - Use Cython to optimize performance-critical sections of code. + - Explore Numba for just-in-time (JIT) compilation of image processing functions. + - Utilize `skimage.util.apply_parallel` for parallel processing of image regions. + - Example using NumPy vectorization: + python + import numpy as np + + def brighten_image(image, factor=1.5): + """Brightens an image by multiplying each pixel by a factor.""" + return np.clip(image * factor, 0, 255).astype(image.dtype) # Clip values to valid range + + +- **Memory Management:** + - Use appropriate data types to minimize memory usage (e.g., `uint8` for grayscale images). + - Release large image arrays when they are no longer needed. + - Avoid creating unnecessary copies of image data. + - Consider using memory-mapped arrays for very large images. + +- **Rendering Optimization:** + - Optimize image display using appropriate colormaps and scaling. + - Use hardware acceleration (e.g., OpenGL) for faster rendering. + +- **Bundle Size Optimization:** + - Minimize dependencies in your scikit-image projects. + - Use tree shaking to remove unused code from your bundles. + - Compress image assets using appropriate compression algorithms. + +- **Lazy Loading:** + - Load large images only when they are needed. + - Use generators or iterators to process images in chunks. + - Implement caching mechanisms to avoid redundant image loading. + +### 4. Security Best Practices: + +- **Common Vulnerabilities:** + - **Denial-of-Service (DoS) Attacks:** Protect against DoS attacks by limiting the size of input images. + - **Code Injection:** Sanitize user-provided image processing parameters to prevent code injection attacks. + +- **Input Validation:** + - Validate the format, size, and data type of input images. + - Check for malicious image headers or metadata. + - Sanitize user-provided parameters to prevent code injection. + - Example: + python + from skimage import io + + def process_image(filepath, resize_factor): + if not isinstance(resize_factor, (int, float)): + raise ValueError("Resize factor must be a number.") + if resize_factor <= 0: + raise ValueError("Resize factor must be positive.") + + try: + image = io.imread(filepath) + # Perform image processing operations using resize_factor + except Exception as e: + print(f"Error processing image: {e}") + + +- **Authentication and Authorization:** + - Implement authentication and authorization mechanisms to control access to image processing resources. + - Use secure protocols (e.g., HTTPS) for API communication. + +- **Data Protection:** + - Encrypt sensitive image data at rest and in transit. + - Implement access control policies to protect image data. + - Use secure storage mechanisms for image data. + +- **Secure API Communication:** + - Use HTTPS for all API communication. + - Implement rate limiting to prevent abuse. + - Use input validation to prevent injection attacks. + +### 5. Testing Approaches: + +- **Unit Testing:** + - Write unit tests for individual functions and classes. + - Use mocking and stubbing to isolate components during testing. + - Test edge cases and boundary conditions. + +- **Integration Testing:** + - Write integration tests to verify the interaction between components. + - Test complex image processing pipelines. + - Use realistic test data. + +- **End-to-End Testing:** + - Write end-to-end tests to verify the entire application workflow. + - Use automated testing frameworks (e.g., Selenium). + +- **Test Organization:** + - Organize tests into separate directories for unit, integration, and end-to-end tests. + - Use descriptive test names. + - Follow a consistent testing style. + +- **Mocking and Stubbing:** + - Use mocking libraries (e.g., `unittest.mock`) to replace external dependencies with mock objects. + - Use stubbing to provide predefined outputs for specific function calls. + +### 6. Common Pitfalls and Gotchas: + +- **Data Type Issues:** + - Be aware of the data types of images and intermediate results. + - Use `img_as_float` or other data type conversion utilities to ensure consistent data types. + - Pay attention to data type ranges (e.g., 0-255 for `uint8`, 0.0-1.0 for float). + +- **Coordinate Conventions:** + - Understand the coordinate conventions used by scikit-image (row, col) and NumPy. + +- **Memory Consumption:** + - Avoid creating unnecessary copies of image data. + - Process large images in chunks or tiles. + +- **Version Compatibility:** + - Be aware of version-specific API changes and deprecations. + - Check scikit-image's changelog for breaking changes. + +- **Image I/O Issues:** + - Use appropriate image formats for your application. + - Handle file I/O errors gracefully. + +### 7. Tooling and Environment: + +- **Recommended Development Tools:** + - IDE: VS Code, PyCharm + - Debugger: pdb, ipdb + - Profiler: cProfile, line_profiler + +- **Build Configuration:** + - Use `pyproject.toml` to manage project metadata and build dependencies. + - Use `requirements.txt` to specify project dependencies. + +- **Linting and Formatting:** + - Use a linter (e.g., flake8, pylint) to enforce coding style and detect errors. + - Use a formatter (e.g., black, autopep8) to automatically format code. + - Configure your IDE to run linters and formatters automatically. + +- **Deployment:** + - Use virtual environments to isolate project dependencies. + - Containerize your scikit-image applications using Docker. + - Deploy your applications to cloud platforms (e.g., AWS, Azure, GCP). + +- **CI/CD Integration:** + - Use a CI/CD pipeline (e.g., GitHub Actions, GitLab CI, Jenkins) to automate testing, building, and deployment. + - Run linters, formatters, and tests in your CI/CD pipeline. + - Use code coverage tools to measure the effectiveness of your tests. + +By adhering to these best practices, you can develop robust, maintainable, and performant image processing applications using the scikit-image library. \ No newline at end of file diff --git a/.cursor/rules/scikit-learn.mdc b/.cursor/rules/scikit-learn.mdc new file mode 100644 index 000000000..36ea219ba --- /dev/null +++ b/.cursor/rules/scikit-learn.mdc @@ -0,0 +1,193 @@ +--- +description: Enforces best practices and coding standards for scikit-learn projects, promoting maintainability, performance, and security. This rule provides guidelines on code organization, common patterns, performance optimization, testing, and common pitfalls. +globs: *.py +--- +# Scikit-learn Best Practices and Coding Standards + +This document outlines best practices and coding standards for developing with scikit-learn, aiming to improve code quality, maintainability, performance, and security. + +## 1. Code Organization and Structure + +- **Directory Structure:** + - Organize your project into logical modules or components. A typical structure might include: + - `src/`: Source code for your project. + - `models/`: Contains the implementation of machine learning models. + - `features/`: Feature engineering and processing logic. + - `utils/`: Utility functions and helper modules. + - `data/`: Datasets used for training and testing. + - `raw/`: Original, unprocessed data. + - `processed/`: Cleaned and preprocessed data. + - `notebooks/`: Jupyter notebooks for experimentation and analysis. + - `tests/`: Unit and integration tests. + - `scripts/`: Scripts for training, evaluation, and deployment. + - `config/`: Configuration files (e.g., YAML, JSON). + - Example: + + + project_root/ + ├── src/ + │ ├── models/ + │ │ ├── __init__.py + │ │ ├── model_1.py + │ │ └── model_2.py + │ ├── features/ + │ │ ├── __init__.py + │ │ ├── feature_engineering.py + │ │ └── feature_selection.py + │ ├── utils/ + │ │ ├── __init__.py + │ │ └── helpers.py + │ ├── __init__.py + │ └── main.py + ├── data/ + │ ├── raw/ + │ │ └── data.csv + │ └── processed/ + │ └── processed_data.csv + ├── notebooks/ + │ └── exploratory_data_analysis.ipynb + ├── tests/ + │ ├── __init__.py + │ ├── test_models.py + │ └── test_features.py + ├── scripts/ + │ ├── train.py + │ └── evaluate.py + ├── config/ + │ └── config.yaml + ├── README.md + └── requirements.txt + + +- **File Naming Conventions:** + - Use descriptive and consistent names for files and modules. + - Prefer snake_case for Python files and variables (e.g., `model_training.py`, `n_samples`). + +- **Module Organization:** + - Each module should have a clear and specific purpose. + - Use `__init__.py` files to define packages and modules within directories. + - Minimize dependencies between modules to improve maintainability. + +- **Component Architecture:** + - Design your application with loosely coupled components. + - Implement interfaces or abstract classes to define contracts between components. + - Use dependency injection to manage dependencies and promote testability. + +- **Code Splitting Strategies:** + - Split large files into smaller, more manageable modules. + - Group related functions and classes into modules based on functionality. + - Consider splitting code based on layers (e.g., data access, business logic, presentation). + +## 2. Common Patterns and Anti-patterns + +- **Design Patterns:** + - **Pipeline:** Use scikit-learn's `Pipeline` class to chain together multiple data preprocessing and modeling steps. This helps prevent data leakage and ensures consistent data transformations. + - **Model Selection:** Employ techniques like cross-validation and grid search to select the best model and hyperparameters for your data. Use `GridSearchCV` or `RandomizedSearchCV`. + - **Ensemble Methods:** Leverage ensemble methods like Random Forests, Gradient Boosting, and Voting Classifiers to improve model accuracy and robustness. + - **Custom Transformers:** Create custom transformers using `BaseEstimator` and `TransformerMixin` to encapsulate complex feature engineering logic. + +- **Recommended Approaches:** + - **Data Preprocessing:** Always split your data into training and testing sets *before* any preprocessing steps. + - **Feature Scaling:** Apply appropriate feature scaling techniques (e.g., StandardScaler, MinMaxScaler) to numerical features before training models. + - **Categorical Encoding:** Use one-hot encoding or ordinal encoding for categorical features. + - **Missing Value Imputation:** Handle missing values using imputation techniques like mean, median, or k-NN imputation. + - **Model Persistence:** Save trained models to disk using `joblib` or `pickle` for later use. + +- **Anti-patterns and Code Smells:** + - **Data Leakage:** Avoid data leakage by preprocessing the entire dataset before splitting into training and testing sets. + - **Overfitting:** Be cautious of overfitting by using regularization techniques and cross-validation. + - **Ignoring Data Distributions:** Always visualize and understand data distributions before applying transformations or choosing models. + - **Hardcoding Parameters:** Avoid hardcoding parameters directly in the code. Use configuration files or command-line arguments instead. + - **Ignoring Warnings:** Pay attention to scikit-learn warnings, as they often indicate potential problems with your code or data. + +- **State Management:** + - Use classes to encapsulate state and behavior related to models or data processing steps. + - Avoid global variables and mutable state whenever possible. + - Use immutable data structures where appropriate to prevent unintended side effects. + +- **Error Handling:** + - Use try-except blocks to handle exceptions and prevent your application from crashing. + - Log errors and warnings to a file or logging service. + - Provide informative error messages to users. + - Implement retry mechanisms for transient errors. + +## 3. Performance Considerations + +- **Optimization Techniques:** + - **Vectorization:** Utilize NumPy's vectorized operations to perform calculations on entire arrays instead of looping through individual elements. Use `.values` from pandas DataFrames when passing to scikit-learn. + - **Memory Optimization:** Minimize memory usage by using appropriate data types (e.g., `int8`, `float32`) and avoiding unnecessary copies of data. + - **Algorithm Selection:** Choose efficient algorithms that are well-suited for your data and task. For example, use `MiniBatchKMeans` for large datasets. + - **Parallelization:** Use scikit-learn's built-in support for parallel processing to speed up training and prediction. + +- **Memory Management:** + - Use memory profiling tools to identify memory leaks and optimize memory usage. + - Release unused memory by deleting variables or using the `gc` module. + - Use data streaming techniques for large datasets that don't fit into memory. + +- **Lazy Loading Strategies:** + - Load data and models only when they are needed. + - Use generators or iterators to process large datasets in chunks. + +## 4. Security Best Practices + +- **Common Vulnerabilities:** + - **Model Poisoning:** Protect against model poisoning attacks by validating input data and sanitizing user-provided data. + - **Adversarial Attacks:** Be aware of adversarial attacks that can manipulate model predictions. Consider using techniques like adversarial training to improve model robustness. + +- **Input Validation:** + - Validate all input data to ensure it conforms to the expected format and range. + - Sanitize user-provided data to prevent injection attacks. + +- **Data Protection:** + - Encrypt sensitive data at rest and in transit. + - Use access control mechanisms to restrict access to data and models. + +## 5. Testing Approaches + +- **Unit Testing:** + - Write unit tests for individual functions and classes. + - Use mocking to isolate components and test them in isolation. + - Assert that your code produces the expected output for different inputs. + +- **Integration Testing:** + - Write integration tests to verify that different components work together correctly. + - Test the interaction between your code and external dependencies. + +- **Test Organization:** + - Organize your tests into a separate directory (e.g., `tests`). + - Use descriptive names for your test files and functions. + +- **Mocking and Stubbing:** + - Use mocking libraries like `unittest.mock` or `pytest-mock` to create mock objects for testing. + - Use stubs to replace complex dependencies with simpler implementations. + +## 6. Common Pitfalls and Gotchas + +- **Data Type Mismatches:** Ensure that the data types of your input features are compatible with the model you are using. +- **Feature Scaling Issues:** Use the correct scaling method for your features, understanding how each scaler behaves. +- **Improper Cross-Validation:** Make sure the cross-validation strategy you choose is appropriate for your dataset and model. +- **Version Compatibility:** Be aware of compatibility issues between different versions of scikit-learn and its dependencies. + +## 7. Tooling and Environment + +- **Recommended Development Tools:** + - **IDE:** Use an IDE like VS Code, PyCharm, or JupyterLab. + - **Version Control:** Use Git for version control. + - **Package Manager:** Use pip or Conda for managing dependencies. + - **Virtual Environments:** Create virtual environments for each project to isolate dependencies. + +- **Linting and Formatting:** + - Use linters like pylint or flake8 to enforce code style and identify potential errors. + - Use formatters like black or autopep8 to automatically format your code. + +- **CI/CD Integration:** + - Integrate your code with a CI/CD system like Jenkins, Travis CI, or CircleCI. + - Automatically run tests and linters on every commit. + - Deploy your application to a production environment automatically. + +## General Coding Style +- Use underscores to separate words in non-class names: `n_samples` rather than `nsamples`. +- Avoid multiple statements on one line. +- Use relative imports for references inside scikit-learn (except for unit tests, which should use absolute imports). + +By following these best practices, you can write cleaner, more maintainable, and more efficient scikit-learn code. \ No newline at end of file diff --git a/.cursor/rules/scipy.mdc b/.cursor/rules/scipy.mdc new file mode 100644 index 000000000..b12a359ed --- /dev/null +++ b/.cursor/rules/scipy.mdc @@ -0,0 +1,56 @@ +--- +description: This rule outlines coding standards, best practices, and common pitfalls for developing scientific computing applications using the SciPy library. It emphasizes clarity, maintainability, performance, and security for efficient SciPy development. +globs: *.py +--- +- Adhere to PEP 8 style guidelines for Python code. This includes consistent indentation (4 spaces), line length (79 characters for code, 72 for docstrings), and naming conventions (e.g., `lower_case_with_underscores` for functions and variables, `CamelCase` for classes). +- Write comprehensive docstrings for all functions, classes, and modules. Docstrings should follow the NumPy/SciPy docstring standard, detailing parameters, return values, and examples. +- Use meaningful and descriptive variable names to enhance code readability. Avoid single-character variable names (except for loop counters) and abbreviations that are not widely understood. +- Break down complex tasks into smaller, modular functions. Each function should have a single, well-defined purpose. +- Employ object-oriented programming principles (classes, inheritance, polymorphism) when appropriate to structure and organize your code. +- Implement unit tests for all critical functions and classes. Use the `unittest` or `pytest` framework to ensure code correctness and prevent regressions. +- Utilize version control (e.g., Git) to track changes, collaborate effectively, and manage different versions of your code. +- Comment your code to explain complex logic, algorithms, or design decisions. Comments should be concise and up-to-date. +- Employ virtual environments (e.g., `venv` or `conda`) to manage project dependencies and ensure reproducibility. +- Use a linter (e.g., `flake8` or `pylint`) to automatically check your code for style violations, errors, and potential bugs. +- Format your code using a formatter (e.g., `black` or `autopep8`) to ensure consistent code style. +- Consider using type hints (using `typing` module) to improve code readability and catch type-related errors early on. +- Be mindful of performance considerations when using SciPy functions. Vectorize operations whenever possible to avoid explicit loops. +- Choose appropriate SciPy functions and algorithms based on the specific problem and data characteristics. +- Avoid unnecessary data copies to minimize memory usage and improve performance. +- Utilize SciPy's sparse matrix functionality when dealing with large, sparse datasets. +- Profile your code using tools like `cProfile` to identify performance bottlenecks. +- Consider using Numba or Cython to accelerate computationally intensive SciPy code. +- Implement error handling using `try...except` blocks to gracefully handle exceptions and prevent program crashes. +- Log errors and warnings using the `logging` module to facilitate debugging and monitoring. +- Validate user inputs to prevent security vulnerabilities, such as code injection or data corruption. +- Store sensitive data securely using appropriate encryption and access control mechanisms. +- Keep your SciPy library up-to-date to benefit from bug fixes, performance improvements, and new features. +- Be aware of the compatibility between different versions of SciPy and other libraries in your project. +- Refer to the SciPy documentation and community resources for guidance and best practices. +- Avoid modifying SciPy arrays in place when it can lead to unexpected side effects. Instead, create copies of the arrays and perform the modifications on the copies. +- When working with random numbers, use `numpy.random.default_rng()` for more modern and controllable random number generation. +- When writing custom functions that operate on NumPy arrays, make sure to handle different data types correctly. +- When possible, avoid creating intermediate arrays, by chaining operations and making the maximum use of SciPy functions features. This can decrease memory consumption especially on large arrays. +- Use the `optimize` module functions whenever possible to avoid manual implementation of optimization algorithms. +- Use sparse matrices when working with large matrices that have mostly zero values. This will save memory and improve performance. +- Use the `fft` module for fast Fourier transforms when working with signals and images. +- Use the `signal` module for signal processing tasks such as filtering, windowing, and spectral analysis. +- Use the `ndimage` module for image processing tasks such as filtering, segmentation, and feature extraction. +- Use the `integrate` module for numerical integration tasks such as quadrature and differential equation solving. +- Use the `interpolate` module for interpolation tasks such as spline interpolation and polynomial interpolation. +- Use the `stats` module for statistical analysis tasks such as hypothesis testing, probability distributions, and regression analysis. +- When selecting a statistical test, ensure it's appropriate for your data type (continuous vs. discrete) and number of samples, and take care to interpret p-values correctly. +- Do not assume that optimized SciPy functions automatically make your overall code efficient; always profile your code. +- Be cautious when combining SciPy functions with external C/C++ code, ensuring data type consistency and memory management. +- Document and share your SciPy-based code using appropriate version control and licensing to facilitate collaboration. +- If you encounter performance issues with SciPy functions, consider trying alternative algorithms or implementations. Different problems may require different solutions. +- Remember that NumPy is a dependency of SciPy. SciPy builds upon NumPy; leverage the strengths of both libraries in your code. Master basic NumPy array manipulation before diving deeply into SciPy. +- Make sure to always use `pip install -U scipy` when installing or upgrading scipy package. +- If possible try to use conda to install dependencies. The management of binary dependencies is much simpler to solve. +- When collaborating with others, define common interfaces and expectations on what can be changed and what cannot. Do not change interfaces without notifying the other developers. +- Separate the code from data. Do not include hardcoded data or configuration into the code. Use files, databases, or environment variables. +- Use code reviews whenever is possible. Code reviews are useful not only to detect errors and bugs, but also to share knowledge and best practices. +- When generating documentation, if possible add badges that show the test coverage and the status of continuous integration. +- Test numerical code with randomized inputs to ensure code stability. Numerical algorithms are difficult to be completely covered only with fixed test cases. +- Try to avoid global shared mutable state, specially if your application needs to run in parallel. +- Try to use parallelization to improve your code efficiency, but be careful with shared mutable state. Use message passing to synchronize data between processes or threads. \ No newline at end of file diff --git a/.cursor/rules/scrapy.mdc b/.cursor/rules/scrapy.mdc new file mode 100644 index 000000000..502a5a2a9 --- /dev/null +++ b/.cursor/rules/scrapy.mdc @@ -0,0 +1,288 @@ +--- +description: This rule provides comprehensive best practices for Scrapy development, including code organization, performance, security, testing, and common pitfalls to avoid. It aims to guide developers in building robust, efficient, and maintainable web scraping applications with Scrapy. +globs: *.py +--- +# Scrapy Best Practices + +This document outlines the recommended best practices for developing Scrapy web scraping applications. Following these guidelines will help you create robust, efficient, secure, and maintainable scrapers. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure + +- **Project Root:** Contains `scrapy.cfg`, project directory, and any `README.md`, `LICENSE`, or other project-level files. +- **Project Directory (e.g., `my_project`):** + - `__init__.py`: Marks the directory as a Python package. + - `items.py`: Defines the data structures (Scrapy Items) for the scraped data. + - `middlewares.py`: Contains the Scrapy middleware components, used for request/response processing. + - `pipelines.py`: Defines the data processing pipelines, used for cleaning, validating, and storing the scraped data. + - `settings.py`: Configures the Scrapy project, including settings for pipelines, middleware, concurrency, etc. + - `spiders/`: + - `__init__.py`: Marks the directory as a Python package. + - `my_spider.py`: Contains the spider definitions (Scrapy Spiders) responsible for crawling and scraping data. + +Example: + + +my_project/ +├── scrapy.cfg +├── my_project/ +│ ├── __init__.py +│ ├── items.py +│ ├── middlewares.py +│ ├── pipelines.py +│ ├── settings.py +│ └── spiders/ +│ ├── __init__.py +│ └── my_spider.py +└── README.md + + +### 1.2. File Naming Conventions + +- **Spider Files:** `spider_name.py` (e.g., `product_spider.py`, `news_spider.py`) +- **Item Files:** `items.py` (standard naming) +- **Middleware Files:** `middlewares.py` (standard naming) +- **Pipeline Files:** `pipelines.py` (standard naming) +- **Settings Files:** `settings.py` (standard naming) + +### 1.3. Module Organization + +- **Small Projects:** All spiders can reside in the `spiders/` directory. +- **Large Projects:** Consider organizing spiders into subdirectories based on the target website or data type (e.g., `spiders/news/`, `spiders/ecommerce/`). +- **Custom Modules:** Create custom modules (e.g., `utils/`, `lib/`) for reusable code, helper functions, and custom classes. + +### 1.4. Component Architecture + +- **Spiders:** Focus on crawling and extracting raw data. +- **Items:** Define the structure of the scraped data. +- **Pipelines:** Handle data cleaning, validation, transformation, and storage. +- **Middleware:** Manage request/response processing, error handling, and proxy management. + +### 1.5. Code Splitting + +- **Separate Concerns:** Keep spiders lean and focused on crawling logic. Move data processing and storage to pipelines. +- **Reusable Components:** Extract common functionality (e.g., custom item loaders, helper functions) into separate modules or classes. +- **Configuration:** Use `settings.py` to manage project-wide configuration and avoid hardcoding values in spiders. +- **Modular Middleware:** Create small, focused middleware components for specific tasks (e.g., user agent rotation, request retries). + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +- **Strategy Pattern:** Use different parsing strategies within a spider based on the page structure or data type. +- **Factory Pattern:** Create a factory function to instantiate items with default values or based on specific criteria. +- **Singleton Pattern:** (Use sparingly) For global resources like database connections, consider using a singleton pattern, but ensure thread safety. +- **Observer Pattern:** Use Scrapy signals to trigger actions based on specific events (e.g., item scraped, spider closed). + +### 2.2. Recommended Approaches + +- **Item Loaders:** Use Item Loaders to populate items with extracted data, providing data validation and cleaning. +- **CSS and XPath Selectors:** Master CSS and XPath selectors for efficient data extraction. +- **Response Objects:** Utilize the methods provided by the `response` object (e.g., `css()`, `xpath()`, `urljoin()`, `follow()`). +- **Asynchronous Operations:** Understand Scrapy's asynchronous nature and use deferreds for handling asynchronous tasks (though Scrapy abstracts a lot of this away). +- **Robots.txt:** Respect the robots.txt file and configure `ROBOTSTXT_OBEY` accordingly. +- **Settings:** Centralize settings in `settings.py` instead of hardcoding values. + +### 2.3. Anti-patterns and Code Smells + +- **Hardcoded Values:** Avoid hardcoding URLs, selectors, or other configuration values directly in the code. +- **Overly Complex Spiders:** Keep spiders focused on crawling. Move complex data processing to pipelines. +- **Ignoring Errors:** Implement proper error handling and logging to identify and address issues. +- **Excessive Logging:** Avoid verbose logging that can impact performance. Use appropriate log levels (DEBUG, INFO, WARNING, ERROR). +- **Blocking Operations:** Avoid blocking operations (e.g., synchronous network requests) in spiders, as they can significantly reduce performance. Let Scrapy handle the concurrency. +- **Unnecessary Recursion:** Overly complex recursion within `parse` methods can lead to stack overflow errors. +- **Abusing Global State:** Avoid overly relying on global variables, since Scrapy manages concurrency it can lead to race conditions or unpredictable spider behavior. + +### 2.4. State Management + +- **Spider Attributes:** Store spider-specific state in spider attributes (e.g., current page number, filters). +- **Request.meta:** Use `Request.meta` to pass data between callbacks (e.g., passing data extracted from one page to the next). +- **Settings:** Use Scrapy settings to manage project-level configuration. +- **External Databases/Caches:** For persistent state or data sharing between spiders, consider using an external database or cache (e.g., Redis). + +### 2.5. Error Handling + +- **`try...except` Blocks:** Use `try...except` blocks to handle potential exceptions during data extraction or processing. +- **Scrapy Logging:** Utilize Scrapy's logging system to record errors, warnings, and informational messages. +- **Retry Middleware:** Configure the Retry Middleware to automatically retry failed requests. +- **Error Handling in Pipelines:** Implement error handling in pipelines to catch and log errors during data processing. +- **Error Handling in Middlewares:** Implement error handling in middleware to deal with request or response issues + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +- **Concurrency:** Adjust `CONCURRENT_REQUESTS`, `CONCURRENT_REQUESTS_PER_DOMAIN`, and `DOWNLOAD_DELAY` to optimize crawling speed while avoiding server overload. +- **Item Pipelines:** Optimize item pipelines for efficient data processing and storage. +- **Caching:** Use HTTP caching to avoid re-downloading unchanged pages (`HTTPCACHE_ENABLED = True`). +- **Offsite Middleware:** Enable the Offsite Spider Middleware to prevent crawling outside the allowed domains. +- **Keep-Alive:** Ensure HTTP keep-alive is enabled for persistent connections. +- **DNS Caching:** Optimize DNS resolution by enabling DNS caching. +- **Efficient Selectors:** Optimize XPath and CSS selectors for faster data extraction. +- **Asynchronous Request Handling:** Scrapy's asynchronous architecture handles concurrency, utilize it by avoid blocking operations + +### 3.2. Memory Management + +- **Large Datasets:** For very large datasets, consider using Scrapy's built-in support for chunked responses or processing data in batches. +- **Avoid Storing Everything in Memory:** Process items in pipelines and avoid storing the entire scraped data in memory at once. +- **Limit Response Body Size:** Set `DOWNLOAD_MAXSIZE` to limit the maximum size of downloaded responses to prevent memory exhaustion. +- **Garbage Collection:** Manually trigger garbage collection if necessary to reclaim memory (use with caution). + +### 3.3. Rendering Optimization + +- **Splash/Selenium:** If JavaScript rendering is required, use Scrapy Splash or Selenium, but be aware of the performance overhead. +- **Render Only When Necessary:** Only render pages that require JavaScript execution. Avoid rendering static HTML pages. +- **Minimize Browser Interactions:** Reduce the number of interactions with the browser (e.g., clicks, form submissions) to improve rendering performance. +- **Caching Rendered Results:** Cache rendered HTML to avoid redundant rendering. + +### 3.4. Bundle Size Optimization + +- Scrapy does not have bundles like web development frameworks, so this section does not apply. + +### 3.5. Lazy Loading + +- **Pagination:** Implement pagination to crawl websites in smaller chunks. +- **Lazy Item Processing:** Defer item processing in pipelines until necessary to reduce memory consumption. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities and Prevention + +- **Cross-Site Scripting (XSS):** Sanitize scraped data before displaying it on a website to prevent XSS attacks. Don't output raw scraped content. +- **SQL Injection:** If storing scraped data in a database, use parameterized queries or ORMs to prevent SQL injection vulnerabilities. +- **Command Injection:** Avoid executing arbitrary commands based on scraped data to prevent command injection attacks. +- **Denial of Service (DoS):** Implement rate limiting and politeness policies to avoid overwhelming target websites. +- **Data Poisoning:** Validate scraped data to ensure its integrity and prevent data poisoning. + +### 4.2. Input Validation + +- **Validate Scraped Data:** Implement validation logic in item pipelines to ensure that scraped data conforms to expected formats and ranges. +- **Data Type Validation:** Check the data type of scraped values to prevent unexpected errors. +- **Regular Expressions:** Use regular expressions to validate string values and enforce specific patterns. + +### 4.3. Authentication and Authorization + +- **HTTP Authentication:** Use Scrapy's built-in support for HTTP authentication to access password-protected websites. +- **Cookies:** Manage cookies properly to maintain sessions and avoid authentication issues. +- **API Keys:** Store API keys securely and avoid exposing them in the code. +- **OAuth:** Implement OAuth authentication if required to access protected resources. + +### 4.4. Data Protection + +- **Encryption:** Encrypt sensitive data during storage and transmission. +- **Anonymization:** Anonymize or pseudonymize personal data to protect user privacy. +- **Access Control:** Implement access control mechanisms to restrict access to sensitive data. + +### 4.5. Secure API Communication + +- **HTTPS:** Always use HTTPS for secure communication with APIs. +- **SSL/TLS:** Ensure that SSL/TLS certificates are properly configured. +- **API Authentication:** Use API keys or tokens for authentication. +- **Rate Limiting:** Implement rate limiting to prevent abuse and protect APIs. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +- **Spider Logic:** Unit test individual components of spiders, such as selector logic, data extraction, and URL generation. +- **Item Pipelines:** Unit test item pipelines to verify data cleaning, validation, and transformation. +- **Middleware:** Unit test middleware components to ensure proper request/response processing. +- **Helper Functions:** Unit test helper functions to ensure correct behavior. + +### 5.2. Integration Testing + +- **Spider Integration:** Integrate spiders with item pipelines to test the entire data flow. +- **Middleware Integration:** Integrate middleware components with spiders to test request/response handling. +- **External Services:** Integrate with external services (e.g., databases, APIs) to test data storage and retrieval. + +### 5.3. End-to-End Testing + +- **Full Crawl:** Run a full crawl of the target website and verify that all data is extracted correctly. +- **Data Validation:** Validate the scraped data against expected values or schemas. +- **Performance Testing:** Measure the performance of the scraper and identify potential bottlenecks. + +### 5.4. Test Organization + +- **Separate Test Directory:** Create a separate `tests/` directory to store test files. +- **Test Modules:** Organize tests into modules based on the component being tested (e.g., `tests/test_spiders.py`, `tests/test_pipelines.py`). +- **Test Fixtures:** Use test fixtures to set up test data and dependencies. + +### 5.5. Mocking and Stubbing + +- **Mock Responses:** Mock HTTP responses to test spider logic without making actual network requests. +- **Stub External Services:** Stub external services (e.g., databases, APIs) to isolate components during testing. +- **Mock Item Pipelines:** Mock item pipelines to prevent actual data storage during testing. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +- **Incorrect Selectors:** Using incorrect or brittle CSS/XPath selectors that break when the website structure changes. +- **Ignoring Robots.txt:** Ignoring the `robots.txt` file and potentially violating the website's terms of service. +- **Overloading the Target Website:** Making too many requests too quickly and potentially causing a denial of service. +- **Not Handling JavaScript:** Failing to handle JavaScript-rendered content. +- **Not Handling Pagination:** Failing to properly handle pagination and missing data from multiple pages. +- **Incorrectly handling Cookies:** Failing to persist or handle cookies properly can cause unpredictable behavior. + +### 6.2. Edge Cases + +- **Website Structure Changes:** Websites frequently change their structure, breaking existing scrapers. Implement robust selectors and monitoring to detect changes. +- **Anti-Scraping Measures:** Websites may implement anti-scraping measures (e.g., CAPTCHAs, IP blocking) to prevent scraping. Use appropriate techniques to bypass these measures. +- **Dynamic Content:** Websites may use dynamic content (e.g., AJAX, WebSockets) to load data. Use appropriate techniques to handle dynamic content. +- **Rate Limiting:** Websites may implement rate limiting to restrict the number of requests from a single IP address. Use proxies or distributed crawling to overcome rate limits. + +### 6.3. Version-Specific Issues + +- **Scrapy API Changes:** Be aware of API changes between Scrapy versions and update your code accordingly. +- **Dependency Conflicts:** Manage dependencies carefully to avoid conflicts between Scrapy and other libraries. +- **Python Version Compatibility:** Ensure that your code is compatible with the supported Python versions. + +### 6.4. Compatibility Concerns + +- **JavaScript Rendering Libraries:** Ensure that the JavaScript rendering library is compatible with the target website and Scrapy. +- **Data Storage Libraries:** Ensure that the data storage library is compatible with Scrapy and the chosen data format. +- **Proxy Management Libraries:** Ensure that the proxy management library is compatible with Scrapy. + +### 6.5. Debugging Strategies + +- **Scrapy Shell:** Use the Scrapy shell to test selectors and extract data interactively. +- **Logging:** Use Scrapy's logging system to record errors, warnings, and informational messages. +- **Debugging Tools:** Use Python debugging tools (e.g., `pdb`, `ipdb`) to step through the code and inspect variables. +- **Middleware Debugging:** Use middleware to inspect requests and responses and identify potential issues. +- **Verbose Output:** Use `-v` or `-vv` when running spiders to get more verbose output. + +## 7. Tooling and Environment + +### 7.1. Recommended Tools + +- **IDE:** Use a capable IDE such as VS Code, PyCharm, or Sublime Text. +- **Virtual Environment:** Use virtual environments (e.g., `venv`, `conda`) to manage dependencies and isolate projects. +- **Scrapy Shell:** Use the Scrapy shell for interactive testing and debugging. +- **Browser Developer Tools:** Use browser developer tools to inspect website structure and identify selectors. + +### 7.2. Build Configuration + +- **`setup.py`:** Use a `setup.py` file to define project dependencies and metadata for larger projects and for distribution. +- **`requirements.txt`:** Use a `requirements.txt` file to list project dependencies for easy installation. +- **`pip freeze`:** Use `pip freeze > requirements.txt` to generate a list of installed packages and their versions. + +### 7.3. Linting and Formatting + +- **PEP 8:** Follow PEP 8 style guidelines for code readability. +- **Linters:** Use linters (e.g., `flake8`, `pylint`) to identify code style issues and potential errors. +- **Formatters:** Use code formatters (e.g., `black`, `autopep8`) to automatically format code according to PEP 8. + +### 7.4. Deployment + +- **Scrapyd:** Use Scrapyd to deploy and manage Scrapy spiders on a server. +- **Docker:** Use Docker to containerize Scrapy applications for easy deployment and scalability. +- **Cloud Platforms:** Deploy Scrapy applications on cloud platforms such as AWS, Google Cloud, or Azure. +- **Scheduled Tasks:** Use scheduled tasks (e.g., cron jobs) to run Scrapy spiders on a regular basis. + +### 7.5. CI/CD Integration + +- **Testing:** Integrate unit and integration tests into the CI/CD pipeline to ensure code quality. +- **Linting and Formatting:** Integrate linters and formatters into the CI/CD pipeline to enforce code style. +- **Automated Deployment:** Automate the deployment process to deploy new versions of the scraper automatically. +- **Monitoring:** Integrate monitoring tools to track the performance and health of the scraper in production. \ No newline at end of file diff --git a/.cursor/rules/seaborn.mdc b/.cursor/rules/seaborn.mdc new file mode 100644 index 000000000..ff4d0dc84 --- /dev/null +++ b/.cursor/rules/seaborn.mdc @@ -0,0 +1,5 @@ +--- +description: This rule provides best practices for coding standards in Seaborn, emphasizing clear, reproducible code, optimal performance, and secure data handling within AI and machine learning data science development. +globs: *.py +--- +By following these best practices and coding standards, you can write clear, reproducible, and maintainable Seaborn code that is optimized for performance and security. \ No newline at end of file diff --git a/.cursor/rules/selenium.mdc b/.cursor/rules/selenium.mdc new file mode 100644 index 000000000..3c8a3426e --- /dev/null +++ b/.cursor/rules/selenium.mdc @@ -0,0 +1,328 @@ +--- +description: This rule provides best practices and coding standards for using the Selenium library in Python. It covers code organization, performance, security, testing, common pitfalls, and tooling to ensure maintainable and efficient Selenium projects. +globs: *.py +--- +# Selenium Best Practices and Coding Standards + +This guide outlines the best practices and coding standards for developing robust, maintainable, and efficient Selenium-based projects in Python. + +Library Information: +- Name: Selenium +- Tags: python, web-scraping, browser-automation, testing + +## 1. Code Organization and Structure + +### 1.1. Directory Structure + +Adopt a well-defined directory structure to maintain code clarity and modularity. A recommended structure includes: + + +project_root/ +├── src/ +│ ├── pages/ +│ │ ├── base_page.py # Base class for all page objects +│ │ ├── login_page.py # Page object for the login page +│ │ ├── home_page.py # Page object for the home page +│ │ └── ... +│ ├── components/ +│ │ ├── search_bar.py # Reusable component for the search bar +│ │ ├── navigation.py # Reusable component for site navigation +│ │ └── ... +│ ├── utils/ +│ │ ├── config.py # Configuration settings +│ │ ├── logger.py # Logging utilities +│ │ ├── helpers.py # Helper functions (e.g., element finding, waits) +│ │ └── ... +│ ├── drivers/ +│ │ ├── chromedriver.exe # Or geckodriver.exe, etc. +│ │ └── ... +│ ├── __init__.py +├── tests/ +│ ├── unit/ +│ │ ├── test_login_page.py # Unit tests for the login page +│ │ └── ... +│ ├── integration/ +│ │ ├── test_home_page_integration.py # Integration tests +│ │ └── ... +│ ├── e2e/ +│ │ ├── test_login_flow.py # End-to-end tests for login flow +│ │ └── ... +│ ├── conftest.py # pytest configuration file +│ ├── __init__.py +├── requirements.txt +├── README.md +├── .gitignore +└── ... + + +- `src/`: Contains the main application code. + - `pages/`: Holds page object models. + - `components/`: Contains reusable UI components. + - `utils/`: Includes utility functions, configuration, and logging. + - `drivers/`: Stores browser driver executables. +- `tests/`: Contains all test-related code. + - `unit/`: Unit tests. + - `integration/`: Integration tests. + - `e2e/`: End-to-end tests. + - `conftest.py`: pytest configuration file to manage fixtures, command-line options, and plugins. +- `requirements.txt`: Lists project dependencies. +- `README.md`: Project documentation. +- `.gitignore`: Specifies intentionally untracked files that Git should ignore. + +### 1.2. File Naming Conventions + +- **Python files:** Use lowercase with underscores (e.g., `login_page.py`, `base_page.py`). +- **Classes:** Use PascalCase (e.g., `LoginPage`, `BasePage`). +- **Functions/Methods:** Use lowercase with underscores (e.g., `login`, `get_title`). +- **Variables:** Use lowercase with underscores (e.g., `username`, `password`). +- **Constants:** Use uppercase with underscores (e.g., `DEFAULT_TIMEOUT`, `LOGIN_URL`). + +### 1.3. Module Organization + +- **Keep modules focused:** Each module should have a clear and specific purpose. Avoid creating large, monolithic modules. +- **Use packages:** Group related modules into packages using `__init__.py` files. +- **Explicit imports:** Use explicit imports (`from module import item`) rather than wildcard imports (`from module import *`) to improve code readability and prevent naming conflicts. +- **Relative imports:** Use relative imports (`from . import module`) within packages to maintain internal dependencies. + +### 1.4. Component Architecture + +- **Page Object Model (POM):** Implement the Page Object Model design pattern for better code organization and maintainability. Each web page is represented as a class, and its elements and actions are defined as methods within that class. +- **Reusable Components:** Identify and create reusable UI components (e.g., search bars, navigation menus) as separate classes or functions. +- **Abstraction:** Abstract away Selenium-specific details (e.g., element locators, WebDriver calls) within page objects and components to make the code more resilient to UI changes. + +### 1.5. Code Splitting Strategies + +- **Functional Decomposition:** Break down complex tasks into smaller, more manageable functions. +- **Class-Based Decomposition:** Use classes to encapsulate related data and behavior. +- **Module-Based Decomposition:** Split code into separate modules based on functionality (e.g., page objects, utilities). + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +- **Page Object Model (POM):** As mentioned earlier, POM is crucial for Selenium projects. It promotes code reuse, reduces redundancy, and simplifies maintenance. +- **Factory Pattern:** Use a factory pattern to create WebDriver instances with different configurations (e.g., different browsers, headless mode). +- **Singleton Pattern:** Use a singleton pattern for configuration objects to ensure consistent access to settings throughout the project. However, be mindful of the potential for global state issues. +- **Strategy Pattern:** Useful for implementing different waiting strategies (explicit vs. implicit) or different authentication methods. + +### 2.2. Recommended Approaches for Common Tasks + +- **Finding Elements:** + - **Prioritize specific locators:** Use `ID`, `NAME`, or custom attributes whenever possible. They are usually more stable than `XPATH` or `CSS_SELECTOR`. + - **Use explicit waits:** Always use `WebDriverWait` to wait for elements to be present, visible, or clickable before interacting with them. This avoids common `NoSuchElementException` and `ElementNotInteractableException` errors. + - **Dynamic locators:** If necessary, use dynamic locators with caution, ensuring they are robust and unlikely to break with minor UI changes. +- **Handling Forms:** + - **Clear input fields:** Always clear input fields using `element.clear()` before sending keys. + - **Submit forms correctly:** Use `element.submit()` on a form element to submit the form, rather than clicking a submit button. This handles edge cases more reliably. + - **Handle dropdowns:** Use the `Select` class to interact with dropdown menus. +- **Handling Alerts and Popups:** + - **Switch to alerts:** Use `driver.switch_to.alert` to interact with JavaScript alerts and confirmation dialogs. + - **Handle windows:** Use `driver.switch_to.window` to switch between browser windows and tabs. + +### 2.3. Anti-patterns and Code Smells + +- **Implicit Waits:** Avoid using implicit waits (`driver.implicitly_wait`). They can lead to unpredictable behavior and make it difficult to debug timing issues. Prefer explicit waits using `WebDriverWait`. +- **Hardcoded Waits:** Avoid using `time.sleep()` for waiting. It's unreliable and inefficient. Use explicit waits instead. +- **Fragile Locators:** Avoid using overly complex or brittle locators that are prone to breaking with minor UI changes. +- **Code Duplication:** Avoid duplicating code, especially locator definitions and common actions. Use POM and reusable components to reduce redundancy. +- **Ignoring Exceptions:** Avoid catching exceptions without handling them properly. Log exceptions and re-raise them if necessary. +- **Global State:** Minimize the use of global variables and shared state, which can make tests difficult to reason about and prone to conflicts. +- **Over-reliance on XPATH:** While XPATH is powerful, avoid using overly complex XPATH expressions when simpler, more robust locators are available. +- **Assuming Immediate Availability:** Don't assume elements are immediately available. Websites load asynchronously, and you need to explicitly wait for elements to appear. + +### 2.4. State Management + +- **Stateless Tests:** Design tests to be as stateless as possible. Each test should be independent and not rely on the state of previous tests. +- **Fixture-Based Setup:** Use test fixtures (e.g., pytest fixtures) to set up and tear down test environments consistently. +- **Avoid Shared WebDriver Instances:** Use a new WebDriver instance for each test or test suite to avoid conflicts and ensure isolation. +- **Clear Cookies and Cache:** Clear cookies and cache before each test or test suite to start with a clean browser state. + +### 2.5. Error Handling + +- **Specific Exception Handling:** Catch specific Selenium exceptions (e.g., `NoSuchElementException`, `TimeoutException`) rather than general `Exception` to handle errors more precisely. +- **Logging:** Log all exceptions and errors with detailed information (e.g., element locator, URL, screenshot). +- **Retries:** Implement retry mechanisms for flaky tests or actions that may fail intermittently due to network issues or timing problems. +- **Screenshots on Failure:** Capture screenshots when tests fail to aid in debugging and identify UI issues. +- **Graceful Shutdown:** Ensure that WebDriver instances are properly closed and resources are released even when tests fail. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +- **Headless Mode:** Run tests in headless mode (without a GUI) to reduce resource consumption and improve execution speed. +- **Parallel Execution:** Run tests in parallel using a test runner that supports parallel execution (e.g., pytest-xdist). +- **Efficient Locators:** Use the most efficient locators possible (e.g., `ID`, `NAME`) to minimize element lookup time. +- **Lazy Loading:** If applicable, implement lazy loading for images and other resources to reduce initial page load time. +- **Connection Pooling:** If using a remote WebDriver, consider connection pooling to reuse connections and reduce overhead. + +### 3.2. Memory Management + +- **Close WebDriver Instances:** Ensure that WebDriver instances are properly closed after each test or test suite to release memory. +- **Avoid Large Data Structures:** Avoid storing large amounts of data in memory during test execution. +- **Use Generators:** Use generators for processing large datasets to avoid loading the entire dataset into memory at once. +- **Garbage Collection:** Be aware of Python's garbage collection and consider using `gc.collect()` to force garbage collection if necessary. + +### 3.3. Rendering Optimization (If Applicable) + +- **Minimize DOM Manipulation:** Minimize DOM manipulation in JavaScript code to reduce rendering time. +- **Optimize CSS:** Optimize CSS selectors and styles to reduce rendering time. +- **Hardware Acceleration:** Enable hardware acceleration in the browser to improve rendering performance. + +### 3.4. Bundle Size Optimization (If Applicable) + +- **Tree Shaking:** Use tree shaking to remove unused code from JavaScript bundles. +- **Code Splitting:** Split JavaScript bundles into smaller chunks that can be loaded on demand. +- **Minification:** Minify JavaScript and CSS code to reduce bundle size. +- **Compression:** Use gzip or Brotli compression to reduce the size of transferred resources. + +### 3.5. Lazy Loading + +- **Image Lazy Loading:** Implement lazy loading for images to improve initial page load time. +- **Component Lazy Loading:** Lazy load components that are not immediately visible to the user. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities + +- **Cross-Site Scripting (XSS):** Prevent XSS attacks by properly escaping user input and avoiding the use of `eval()` or other unsafe JavaScript functions. +- **SQL Injection:** Prevent SQL injection attacks by using parameterized queries or ORM frameworks. +- **Clickjacking:** Prevent clickjacking attacks by setting the `X-Frame-Options` header to `DENY` or `SAMEORIGIN`. +- **Man-in-the-Middle (MITM):** Use HTTPS to encrypt communication between the client and server and prevent MITM attacks. + +### 4.2. Input Validation + +- **Validate All User Input:** Validate all user input on both the client-side and server-side to prevent malicious data from entering the system. +- **Use Whitelists:** Use whitelists to define the allowed characters, formats, and values for user input. +- **Escape User Input:** Escape user input before displaying it on the page to prevent XSS attacks. + +### 4.3. Authentication and Authorization + +- **Use Strong Passwords:** Enforce the use of strong passwords and store passwords securely using hashing algorithms (e.g., bcrypt). +- **Multi-Factor Authentication (MFA):** Implement MFA to add an extra layer of security. +- **Role-Based Access Control (RBAC):** Implement RBAC to control access to resources based on user roles. +- **Session Management:** Use secure session management techniques to prevent session hijacking. + +### 4.4. Data Protection + +- **Encrypt Sensitive Data:** Encrypt sensitive data at rest and in transit. +- **Data Masking:** Mask sensitive data when displaying it on the page or in logs. +- **Data Retention Policies:** Implement data retention policies to ensure that data is not stored longer than necessary. +- **Access Control:** Implement strict access control policies to limit access to sensitive data. + +### 4.5. Secure API Communication + +- **Use HTTPS:** Use HTTPS to encrypt communication between the client and server. +- **API Keys:** Use API keys to authenticate requests to external APIs. +- **Rate Limiting:** Implement rate limiting to prevent abuse of APIs. +- **Input Validation:** Validate all input to external APIs to prevent injection attacks. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +- **Test Individual Components:** Unit tests should focus on testing individual components (e.g., page objects, utilities) in isolation. +- **Mock External Dependencies:** Mock external dependencies (e.g., WebDriver, APIs) to isolate the component being tested. +- **Assert Expected Behavior:** Use assertions to verify that the component behaves as expected. +- **Test Edge Cases:** Test edge cases and error conditions to ensure that the component is robust. + +### 5.2. Integration Testing + +- **Test Interactions Between Components:** Integration tests should focus on testing the interactions between different components. +- **Use Real Dependencies:** Use real dependencies (e.g., WebDriver) to test the component's behavior in a realistic environment. +- **Verify System Behavior:** Verify that the system as a whole behaves as expected. + +### 5.3. End-to-End Testing + +- **Test Complete Workflows:** End-to-end tests should focus on testing complete workflows from start to finish. +- **Simulate User Interactions:** Simulate user interactions to test the system's behavior in a realistic scenario. +- **Verify Business Requirements:** Verify that the system meets the business requirements. + +### 5.4. Test Organization + +- **Separate Test Files:** Create separate test files for each component or feature. +- **Use Descriptive Names:** Use descriptive names for test files and test functions. +- **Group Tests:** Group related tests into test suites or test classes. +- **Use Test Fixtures:** Use test fixtures to set up and tear down test environments consistently. + +### 5.5. Mocking and Stubbing + +- **Mock WebDriver:** Mock the WebDriver instance to isolate components from the browser. +- **Stub API Responses:** Stub API responses to test the component's behavior with different data. +- **Verify Method Calls:** Verify that methods are called with the expected arguments. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +- **Incorrect Locator Strategy:** Choosing the wrong locator strategy (e.g., relying on fragile XPATH expressions). +- **Not Waiting for Elements:** Not waiting for elements to be present or visible before interacting with them. +- **Ignoring Exceptions:** Ignoring exceptions and not handling errors properly. +- **Code Duplication:** Duplicating code and not using reusable components. +- **Global State:** Using global state and not isolating tests properly. + +### 6.2. Edge Cases + +- **Dynamic Content:** Handling dynamic content that changes frequently. +- **Asynchronous Operations:** Dealing with asynchronous operations and race conditions. +- **Complex User Interactions:** Simulating complex user interactions (e.g., drag and drop, file uploads). +- **Cross-Browser Compatibility:** Ensuring that tests work correctly across different browsers. +- **Mobile Testing:** Testing on mobile devices and emulators. + +### 6.3. Version-Specific Issues + +- **WebDriver Compatibility:** Ensuring that the WebDriver version is compatible with the browser version. +- **Selenium API Changes:** Being aware of changes to the Selenium API in different versions. +- **Python Version Compatibility:** Ensuring compatibility with the correct Python version. + +### 6.4. Compatibility Concerns + +- **Framework Conflicts:** Conflicts with other testing frameworks or libraries. +- **Browser Extensions:** Interference from browser extensions. +- **Operating System Differences:** Differences in behavior between different operating systems. + +### 6.5. Debugging Strategies + +- **Logging:** Use logging to track the execution of tests and identify errors. +- **Screenshots:** Capture screenshots when tests fail to aid in debugging. +- **Debugging Tools:** Use debugging tools to step through the code and inspect variables. +- **Remote Debugging:** Use remote debugging to debug tests running on remote machines. +- **Browser Developer Tools:** Utilize browser developer tools (e.g., Chrome DevTools) to inspect the DOM and network traffic. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +- **IDE:** PyCharm, VS Code with Python extension +- **Test Runner:** pytest, unittest +- **WebDriver Manager:** webdriver-manager +- **Linter:** pylint, flake8 +- **Formatter:** black, autopep8 +- **Virtual Environment:** virtualenv, venv, conda + +### 7.2. Build Configuration + +- **Use a Build System:** Use a build system (e.g., Make, tox) to automate the build process. +- **Define Dependencies:** Define project dependencies in a `requirements.txt` file. +- **Use Virtual Environments:** Use virtual environments to isolate project dependencies. +- **Configure Test Execution:** Configure test execution options (e.g., browser, headless mode, parallel execution) in a configuration file. + +### 7.3. Linting and Formatting + +- **Use a Linter:** Use a linter (e.g., pylint, flake8) to enforce coding standards and identify potential errors. +- **Use a Formatter:** Use a formatter (e.g., black, autopep8) to automatically format code according to coding standards. +- **Configure Editor Integration:** Configure editor integration to automatically run linters and formatters when saving files. + +### 7.4. Deployment + +- **Containerization:** Use containerization (e.g., Docker) to package the application and its dependencies into a single unit. +- **Cloud Deployment:** Deploy the application to a cloud platform (e.g., AWS, Azure, GCP). +- **Continuous Integration:** Integrate the deployment process with a continuous integration system. + +### 7.5. CI/CD Integration + +- **Use a CI/CD System:** Use a CI/CD system (e.g., Jenkins, Travis CI, CircleCI, GitHub Actions) to automate the build, test, and deployment process. +- **Configure Triggers:** Configure triggers to automatically run the CI/CD pipeline when code is pushed to the repository. +- **Automate Testing:** Automate the execution of unit tests, integration tests, and end-to-end tests in the CI/CD pipeline. +- **Automate Deployment:** Automate the deployment process in the CI/CD pipeline. + +By following these best practices, you can develop robust, maintainable, and efficient Selenium-based projects in Python. \ No newline at end of file diff --git a/.cursor/rules/sentry.mdc b/.cursor/rules/sentry.mdc new file mode 100644 index 000000000..d4ea62010 --- /dev/null +++ b/.cursor/rules/sentry.mdc @@ -0,0 +1,146 @@ +--- +description: This rule provides comprehensive best practices for integrating and utilizing Sentry in your projects. It covers code organization, performance, security, testing, and common pitfalls when using Sentry for error tracking and performance monitoring. +globs: *.js,*.jsx,*.ts,*.tsx,*.py,*.java,*.kt,*.swift,*.go,*.rb,*.php,*.cs,*.cpp,*.c,*.m,*.mm,*.clj,*.cljs,*.scala,*.groovy,*.lua,*.erl,*.hrl,*.fs,*.fsi,*.vb,*.vba,*.sh,*.bash,*.zsh,*.ps1,*.psm1,*.sql,*.html,*.htm,*.css,*.scss,*.less,*.graphql,*.gql +--- +- Ensure that your codebase is connected to Sentry, with source code management integrations (e.g., GitHub, GitLab, Bitbucket) and source maps properly configured. This enables Sentry to link errors to specific lines of code, dramatically improving debugging efficiency. +- Implement custom alerts tailored to your needs, avoiding alert fatigue by prioritizing alerts based on severity or user impact. Set up different notification channels for varying alert priorities (e.g., Slack for low-priority, PagerDuty for high-priority). +- Leverage Sentry's performance monitoring capabilities to track application performance metrics and identify slow transactions. Use distributed tracing to visualize request flows and pinpoint performance bottlenecks. +- Implement alerts based on session-affected percentages when applicable to handle dynamic traffic. + +## 1. Code Organization and Structure + +- **Directory Structure:** + - Group Sentry-related initialization and configuration files in a dedicated directory (e.g., `sentry/`) for easy management. + - Consider separate files for Sentry SDK initialization, custom integrations, and alert configurations. +- **File Naming Conventions:** + - Use descriptive names for Sentry-related files (e.g., `sentry.init.js`, `sentry.config.py`, `sentry.integrations.ts`). +- **Module Organization:** + - Encapsulate Sentry-related functionality within dedicated modules or classes to promote code reusability and maintainability. + - Avoid direct Sentry SDK calls throughout the application; instead, use an abstraction layer or helper functions. +- **Component Architecture:** + - When using Sentry in UI frameworks (e.g., React, Vue), wrap key components with error boundaries to catch and report errors gracefully. + - Create reusable components or hooks for common Sentry interactions, such as capturing user feedback or breadcrumbs. +- **Code Splitting:** + - Lazy-load Sentry SDK and related code to minimize initial bundle size and improve application loading times. + - Use dynamic imports or similar techniques to load Sentry code only when it's needed. + +## 2. Common Patterns and Anti-patterns + +- **Design Patterns:** + - **Singleton:** Use a singleton pattern for the Sentry client instance to ensure consistent configuration and behavior. + - **Observer:** Implement an observer pattern to capture and report application-specific events to Sentry. + - **Decorator:** Utilize decorators to automatically capture exceptions or performance metrics for specific functions or methods. +- **Recommended Approaches:** + - Use breadcrumbs to capture user actions and application state leading up to an error, providing valuable context for debugging. + - Set user context (ID, email, username) to associate errors with specific users, facilitating troubleshooting and impact assessment. + - Capture release information (version, environment) to track errors across different deployments and environments. +- **Anti-patterns:** + - Avoid hardcoding sensitive information (DSN, API keys) directly in the code; use environment variables or configuration files. + - Do not capture personally identifiable information (PII) without proper anonymization or consent; adhere to privacy regulations. + - Refrain from disabling Sentry entirely in production; instead, configure appropriate sampling rates or filters to manage data volume. +- **State Management:** + - In applications using state management libraries (e.g., Redux, Vuex), integrate Sentry middleware to capture state changes and actions leading up to an error. + - Serialize relevant state information and include it in the error context for debugging purposes. +- **Error Handling:** + - Use try-catch blocks to handle potential errors gracefully and report them to Sentry. + - Implement global error handlers to catch uncaught exceptions and unhandled rejections. + - Provide informative error messages to users and avoid exposing sensitive implementation details. + +## 3. Performance Considerations + +- **Optimization Techniques:** + - Enable compression for Sentry payloads to reduce network bandwidth and improve transmission speed. + - Use Sentry's sampling options to control the number of events captured, balancing data accuracy with performance impact. + - Configure Sentry's throttling options to prevent excessive event submission and potential performance bottlenecks. +- **Memory Management:** + - Be mindful of memory usage when capturing large payloads or attachments; avoid unnecessary data serialization. + - Periodically flush the Sentry event queue to release memory and prevent out-of-memory errors. +- **Rendering Optimization:** + - For UI frameworks, defer Sentry initialization until after the initial render to avoid blocking the user interface. + - Optimize breadcrumb and event data collection to minimize performance overhead during rendering. +- **Bundle Size Optimization:** + - Tree-shake the Sentry SDK to remove unused modules and reduce bundle size. + - Use code splitting to load Sentry code only when it's needed. +- **Lazy Loading:** + - Implement lazy loading for Sentry components and modules to improve initial page load times. + - Use dynamic imports or similar techniques to load Sentry code on demand. + +## 4. Security Best Practices + +- **Common Vulnerabilities:** + - **Data Leakage:** Ensure that sensitive information (PII, API keys) is not inadvertently exposed in Sentry events or breadcrumbs. + - **Cross-Site Scripting (XSS):** Sanitize user-generated content before capturing it in Sentry to prevent XSS attacks. + - **Denial of Service (DoS):** Implement rate limiting and throttling to prevent excessive event submission from malicious actors. +- **Input Validation:** + - Validate user input on both the client and server sides to prevent malicious data from reaching Sentry. + - Sanitize and escape user input before capturing it in Sentry events or breadcrumbs. +- **Authentication and Authorization:** + - Implement strong authentication and authorization mechanisms to protect Sentry API endpoints and data. + - Restrict access to Sentry projects and events based on user roles and permissions. +- **Data Protection:** + - Implement data anonymization and masking techniques to protect sensitive user information in Sentry events. + - Comply with data privacy regulations (e.g., GDPR, CCPA) when collecting and processing user data with Sentry. +- **Secure API Communication:** + - Use HTTPS for all communication with Sentry API endpoints to encrypt data in transit. + - Verify the SSL/TLS certificates of Sentry API endpoints to prevent man-in-the-middle attacks. + +## 5. Testing Approaches + +- **Unit Testing:** + - Write unit tests to verify the functionality of Sentry-related modules and components. + - Mock or stub Sentry SDK functions to isolate the code under test. +- **Integration Testing:** + - Perform integration tests to ensure that Sentry is properly integrated with the application and that events are captured correctly. + - Use a test Sentry project to capture and verify events during integration testing. +- **End-to-End Testing:** + - Conduct end-to-end tests to simulate real-world user scenarios and verify that errors are captured and reported to Sentry. + - Use a staging environment or test Sentry project for end-to-end testing. +- **Test Organization:** + - Organize Sentry-related tests in a dedicated directory or module. + - Follow a consistent naming convention for Sentry-related test files and functions. +- **Mocking and Stubbing:** + - Use mocking libraries (e.g., Jest, Sinon) to create mock Sentry SDK objects for testing purposes. + - Stub Sentry SDK functions to control their behavior and verify that they are called correctly. + +## 6. Common Pitfalls and Gotchas + +- **Frequent Mistakes:** + - Forgetting to initialize the Sentry SDK properly. + - Capturing sensitive information without proper anonymization. + - Overloading Sentry with too many events or attachments. + - Ignoring Sentry alerts or dashboards. +- **Edge Cases:** + - Handling errors during Sentry SDK initialization. + - Dealing with network connectivity issues when sending events. + - Managing Sentry SDK configuration in different environments. +- **Version-Specific Issues:** + - Review Sentry SDK release notes for breaking changes and compatibility issues. + - Test Sentry SDK upgrades in a staging environment before deploying to production. +- **Compatibility Concerns:** + - Ensure compatibility between the Sentry SDK and other libraries or frameworks used in the application. + - Resolve any conflicts or version mismatches that may arise. +- **Debugging Strategies:** + - Use Sentry's debug mode to log SDK activity and troubleshoot integration issues. + - Inspect Sentry events in the Sentry UI to verify data accuracy and identify potential problems. + - Utilize Sentry's breadcrumbs and user context to gain insights into the cause of errors. + +## 7. Tooling and Environment + +- **Recommended Tools:** + - Sentry CLI for managing Sentry projects and releases. + - Source code management system (e.g., Git) for version control. + - Package manager (e.g., npm, pip) for managing Sentry SDK dependencies. +- **Build Configuration:** + - Integrate Sentry SDK into the build process to capture build errors and performance metrics. + - Use environment variables or configuration files to manage Sentry SDK settings in different environments. +- **Linting and Formatting:** + - Configure linters (e.g., ESLint, PyLint) to enforce code style and best practices for Sentry code. + - Use code formatters (e.g., Prettier, Black) to automatically format Sentry code and ensure consistency. +- **Deployment:** + - Deploy Sentry SDK and related code as part of the application deployment process. + - Configure Sentry SDK to capture deployment information (version, environment) for tracking errors across releases. +- **CI/CD Integration:** + - Integrate Sentry SDK into the CI/CD pipeline to automate error capture and reporting during testing and deployment. + - Use Sentry's release tracking features to associate errors with specific deployments and releases. + +By adhering to these best practices, you can effectively leverage Sentry to improve the reliability, performance, and security of your applications. \ No newline at end of file diff --git a/.cursor/rules/servemux.mdc b/.cursor/rules/servemux.mdc new file mode 100644 index 000000000..7bed61204 --- /dev/null +++ b/.cursor/rules/servemux.mdc @@ -0,0 +1,406 @@ +--- +description: This rule enforces best practices for using the `net/http` ServeMux in Go, promoting clean, maintainable, and efficient code. It covers routing, handler design, and error handling specifics to help developers leverage ServeMux effectively. +globs: *.go +--- +--- +# servemux Best Practices and Coding Standards + +This document provides comprehensive guidelines for using the `net/http` ServeMux in Go, promoting best practices, coding standards, and efficient development. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure Best Practices + +- **Modular Design:** Organize your code into logical modules, each responsible for a specific set of functionalities. +- **Handler Grouping:** Group related handlers within the same directory or package. For instance, all user-related handlers could reside in a `users` package. +- **Middleware Directory:** Create a dedicated `middleware` directory for reusable middleware functions. +- **Internal vs. External:** Utilize the `internal` directory to encapsulate code that should not be exposed outside your module. This enhances encapsulation and reduces the API surface. + + +project-root/ +├── cmd/ +│ └── my-app/ +│ └── main.go +├── internal/ +│ ├── handlers/ +│ │ ├── users.go +│ │ └── products.go +│ └── middleware/ +│ ├── auth.go +│ └── logging.go +├── pkg/ +│ └── utils/ +│ └── utils.go +└── go.mod + + +### 1.2 File Naming Conventions + +- **Descriptive Names:** Use clear and descriptive file names that reflect the functionality they contain. For example, `users.go` for user-related handlers, and `auth.go` for authentication middleware. +- **Handler Specifics:** If a file contains a specific handler, name it accordingly. For example, `get_user_handler.go` or `create_product_handler.go`. +- **Lowercase:** Use lowercase for all file names. + +### 1.3 Module Organization + +- **`go.mod`:** Ensure your project has a `go.mod` file to manage dependencies. Run `go mod init ` to create one. +- **Semantic Versioning:** Follow semantic versioning for your modules. Use tags (e.g., `v1.0.0`) to mark releases. +- **Vendor Dependencies:** Consider vendoring dependencies using `go mod vendor` to ensure reproducibility. + +### 1.4 Component Architecture + +- **Separation of Concerns:** Design your components to have a single responsibility. Separate handler logic from business logic and data access. +- **Interfaces:** Use interfaces to define contracts between components, promoting loose coupling and testability. For example: + + go +type UserStore interface { + GetUser(id int) (*User, error) + CreateUser(user *User) error +} + +type UserHandler struct { + store UserStore +} + + +### 1.5 Code Splitting Strategies + +- **Package-Based Splitting:** Divide your application into packages based on functionality. Each package should be cohesive and have a clear purpose. +- **Feature-Based Splitting:** Organize code based on features. Each feature gets its own directory or package containing all relevant components. +- **Layered Architecture:** Implement a layered architecture (e.g., presentation, business logic, data access) and split the code accordingly. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns Specific to servemux + +- **Middleware Chaining:** Implement middleware as a chain of functions that decorate the handler. This allows for modular and reusable logic. Example: + + go +func LoggingMiddleware(next http.Handler) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + log.Printf("Request: %s %s", r.Method, r.URL.Path) + next.ServeHTTP(w, r) + }) +} + +// Usage: +mux := http.NewServeMux() +mux.Handle("/", LoggingMiddleware(http.HandlerFunc(myHandler))) + + +- **Context-Aware Handlers:** Utilize the `context.Context` to pass request-scoped values to handlers. This facilitates tracing, request cancellation, and data sharing. + + go +func MyHandler(w http.ResponseWriter, r *http.Request) { + userID := r.Context().Value("userID").(int) + // ... +} + +// Setting context value in middleware: +ctx := context.WithValue(r.Context(), "userID", 123) +r = r.WithContext(ctx) + + +- **Route Grouping (with custom muxes):** Use separate ServeMux instances for different route groups (e.g., API v1, API v2, admin routes). This improves organization and maintainability. + + go +v1Mux := http.NewServeMux() +v1Mux.HandleFunc("/users", v1UsersHandler) +v1Mux.HandleFunc("/products", v1ProductsHandler) + +v2Mux := http.NewServeMux() +v2Mux.HandleFunc("/users", v2UsersHandler) +v2Mux.HandleFunc("/products", v2ProductsHandler) + +http.Handle("/api/v1/", v1Mux) +http.Handle("/api/v2/", v2Mux) + + +### 2.2 Recommended Approaches for Common Tasks + +- **Creating a New ServeMux:** Always use `http.NewServeMux()` to create a new ServeMux instance. Avoid using the default `http.DefaultServeMux` to prevent global state issues. Example: + + go +mux := http.NewServeMux() +mux.HandleFunc("/", myHandler) +http.ListenAndServe(":8080", mux) + + +- **Registering Handlers:** Use `mux.HandleFunc()` or `mux.Handle()` to register handlers with the ServeMux. + + go +mux.HandleFunc("/users", usersHandler) +mux.Handle("/products", productHandler{}) + + +- **Serving Static Files:** Use `http.FileServer()` to serve static files. + + go +fs := http.FileServer(http.Dir("./static")) +mux.Handle("/static/", http.StripPrefix("/static/", fs)) + + +- **Handling HTTP Methods:** Use `http.MethodGet`, `http.MethodPost`, etc., constants to check the HTTP method. With Go 1.22+, specify methods in the pattern itself for `HandleFunc`. + + go +mux.HandleFunc("GET /users/{id}", getUserHandler) +mux.HandleFunc("POST /users", createUserHandler) + + +### 2.3 Anti-patterns and Code Smells to Avoid + +- **Global State:** Avoid using global variables or the default `http.DefaultServeMux` for request handling, as it can lead to race conditions and difficult debugging. +- **Overlapping Routes:** Be mindful of overlapping routes. Go 1.22+ will panic if conflicting patterns are registered. Ensure that your routes are distinct and do not cause conflicts. Properly consider precedence rules. +- **Ignoring Errors:** Always handle errors returned by functions. Log errors and return appropriate HTTP status codes to the client. +- **Long Handler Functions:** Keep handler functions short and focused. Delegate complex logic to other functions or services. +- **Hardcoding Values:** Avoid hardcoding configuration values. Use environment variables or configuration files. +- **Lack of Input Validation:** Always validate user input to prevent security vulnerabilities. + +### 2.4 State Management Best Practices + +- **Stateless Handlers:** Design handlers to be stateless whenever possible. If state is required, store it in the request context or an external data store. +- **Request-Scoped Values:** Use the `context.Context` to store request-scoped values. Avoid using global variables to store state related to a specific request. +- **Sessions:** Use secure session management techniques to maintain user sessions. + +### 2.5 Error Handling Patterns + +- **Centralized Error Handling:** Implement a centralized error handling middleware to catch and log errors. Return appropriate HTTP status codes and error messages to the client. + + go +func ErrorHandlingMiddleware(next http.Handler) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + defer func() { + if err := recover(); err != nil { + log.Printf("Panic: %v", err) + http.Error(w, "Internal Server Error", http.StatusInternalServerError) + } + }() + next.ServeHTTP(w, r) + }) +} + + +- **Custom Error Types:** Define custom error types to provide more context about the error. + + go +type ValidationError struct { + Field string + Message string +} + +func (e *ValidationError) Error() string { + return fmt.Sprintf("Validation error: %s - %s", e.Field, e.Message) +} + + +- **Logging Errors:** Log errors with sufficient detail for debugging. +- **Returning Appropriate Status Codes:** Return appropriate HTTP status codes based on the error. Use descriptive error messages in the response body. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +- **Connection Pooling:** Use connection pooling for database connections to reduce overhead. +- **Caching:** Implement caching for frequently accessed data to reduce database load. +- **Gzip Compression:** Enable Gzip compression for responses to reduce bandwidth usage. + + go +func GzipMiddleware(next http.Handler) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if strings.Contains(r.Header.Get("Accept-Encoding"), "gzip") { + gz, err := gzip.NewWriterLevel(w, gzip.BestSpeed) + if err != nil { + http.Error(w, err.Error(), http.StatusInternalServerError) + return + } + defer gz.Close() + w.Header().Set("Content-Encoding", "gzip") + next.ServeHTTP(gzipResponseWriter{Writer: gz, ResponseWriter: w}, r) + } else { + next.ServeHTTP(w, r) + } + }) +} + +type gzipResponseWriter struct { + io.Writer + http.ResponseWriter +} + +func (w gzipResponseWriter) Write(b []byte) (int, error) { + return w.Writer.Write(b) +} + + +- **Keep-Alive Connections:** Enable keep-alive connections to reduce connection establishment overhead. +- **Efficient Data Structures:** Use efficient data structures and algorithms for data processing. + +### 3.2 Memory Management + +- **Avoid Memory Leaks:** Be aware of potential memory leaks, especially when using goroutines. Ensure that all goroutines are properly terminated. +- **Object Pooling:** Consider using object pooling for frequently allocated objects. +- **String Conversions:** Minimize unnecessary string conversions, as they can be expensive. +- **Defer Statements:** Use `defer` statements carefully. While convenient, they can add overhead if overused in performance-critical sections. + +### 3.3 Rendering Optimization + +- **Template Caching:** Cache templates to reduce parsing overhead. +- **Efficient Template Rendering:** Use efficient template rendering techniques. Consider using pre-compiled templates. +- **Minimize DOM Updates:** Minimize DOM updates in the client-side JavaScript code. + +### 3.4 Bundle Size Optimization + +- **Code Minification:** Minify JavaScript and CSS code to reduce bundle size. +- **Tree Shaking:** Use tree shaking to remove unused code from the bundle. +- **Image Optimization:** Optimize images to reduce file size. + +### 3.5 Lazy Loading Strategies + +- **Lazy Loading Images:** Lazy load images that are not immediately visible. +- **Code Splitting:** Split the code into smaller chunks that can be loaded on demand. +- **Dynamic Imports:** Use dynamic imports to load modules only when they are needed. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and How to Prevent Them + +- **SQL Injection:** Prevent SQL injection by using parameterized queries or ORMs. +- **Cross-Site Scripting (XSS):** Prevent XSS by escaping user input before rendering it in HTML. +- **Cross-Site Request Forgery (CSRF):** Prevent CSRF by using CSRF tokens. +- **Authentication Bypass:** Implement robust authentication and authorization mechanisms to prevent authentication bypass. +- **Denial-of-Service (DoS):** Implement rate limiting and request validation to prevent DoS attacks. + +### 4.2 Input Validation + +- **Validate All Inputs:** Validate all user inputs to ensure that they conform to the expected format and values. Use libraries like `ozzo-validation` for complex validation rules. +- **Sanitize Inputs:** Sanitize inputs to remove potentially malicious characters. +- **Whitelist Inputs:** Use a whitelist approach to only allow specific characters or patterns. + +### 4.3 Authentication and Authorization Patterns + +- **Authentication Middleware:** Implement authentication middleware to verify user credentials. +- **Authorization Middleware:** Implement authorization middleware to check user permissions. +- **JWT (JSON Web Tokens):** Use JWT for stateless authentication. +- **OAuth 2.0:** Use OAuth 2.0 for delegated authorization. +- **Role-Based Access Control (RBAC):** Implement RBAC to manage user permissions. + +### 4.4 Data Protection Strategies + +- **Encryption:** Encrypt sensitive data at rest and in transit. +- **Hashing:** Hash passwords with a strong hashing algorithm like bcrypt. +- **Salting:** Use salts to prevent rainbow table attacks. +- **Data Masking:** Mask sensitive data in logs and error messages. + +### 4.5 Secure API Communication + +- **HTTPS:** Use HTTPS to encrypt communication between the client and the server. +- **TLS (Transport Layer Security):** Use TLS 1.3 or higher for secure communication. +- **Certificate Pinning:** Consider using certificate pinning to prevent man-in-the-middle attacks. +- **Rate Limiting:** Implement rate limiting to prevent abuse. + +## 5. Testing Approaches + +### 5.1 Unit Testing Strategies + +- **Test Individual Units:** Unit test individual functions and methods in isolation. +- **Mock Dependencies:** Use mocks to isolate the unit under test from its dependencies. Use libraries like `gomock` or `testify/mock`. +- **Test Boundary Conditions:** Test boundary conditions and edge cases. +- **Table-Driven Tests:** Use table-driven tests to test multiple scenarios with different inputs and outputs. + +### 5.2 Integration Testing + +- **Test Interactions Between Components:** Integration test the interactions between different components of the application. +- **Test Database Interactions:** Test the interactions with the database. +- **Test API Endpoints:** Test the API endpoints. + +### 5.3 End-to-End Testing + +- **Test the Entire Application:** End-to-end test the entire application flow. +- **Use Automated Testing Tools:** Use automated testing tools like Selenium or Cypress. + +### 5.4 Test Organization + +- **Keep Tests Close to Code:** Keep test files in the same directory as the code they test. Use the `_test.go` suffix for test files. +- **Separate Test Packages:** Consider creating separate test packages for integration and end-to-end tests. +- **Descriptive Test Names:** Use descriptive test names that clearly indicate what is being tested. + +### 5.5 Mocking and Stubbing + +- **Use Interfaces:** Define interfaces for dependencies to facilitate mocking. +- **Use Mocking Libraries:** Use mocking libraries like `gomock` or `testify/mock` to generate mocks. +- **Create Stub Implementations:** Create stub implementations for dependencies that are difficult to mock. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes Developers Make + +- **Forgetting to Handle Errors:** Always check and handle errors returned by functions. +- **Ignoring Context Cancellation:** Respect context cancellation signals and terminate long-running operations. +- **Not Validating Inputs:** Always validate user inputs to prevent security vulnerabilities. +- **Using Global State:** Avoid using global variables for request handling, as it can lead to race conditions. +- **Overlapping Routes:** Failing to understand route precedence, especially in Go 1.22+ can lead to unexpected behavior or panics during server initialization. Thoroughly test route registrations. + +### 6.2 Edge Cases to Be Aware Of + +- **Trailing Slashes:** Be aware of how trailing slashes affect route matching. In Go 1.22+, use `{$}` to match only the path with the trailing slash. +- **Wildcard Matching:** Understand how wildcards match different segments of the path. Use `{pathname...}` to match all remaining segments. +- **Method Matching:** Ensure that the correct HTTP method is being used for each route. With Go 1.22+, be aware that `GET` also matches `HEAD` requests. + +### 6.3 Version-Specific Issues + +- **Go 1.22 Routing Enhancements:** Be aware of the new routing enhancements in Go 1.22, including method matching and wildcard support. Understand the new precedence rules. +- **Compatibility Issues:** Check for compatibility issues when upgrading to new versions of Go or third-party libraries. + +### 6.4 Compatibility Concerns + +- **Backwards Compatibility:** Ensure that changes do not break backwards compatibility. Use feature flags to gradually introduce new features. +- **API Versioning:** Use API versioning to manage changes to the API. +- **Client Compatibility:** Ensure that client applications are compatible with the API. + +### 6.5 Debugging Strategies + +- **Logging:** Use logging to track the execution flow and identify errors. +- **Debugging Tools:** Use debugging tools like `delve` to step through the code and inspect variables. +- **Profiling:** Use profiling tools to identify performance bottlenecks. +- **Tracing:** Use tracing tools to track requests across different services. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +- **GoLand:** A powerful IDE for Go development. +- **Visual Studio Code with Go Extension:** A lightweight and versatile code editor with excellent Go support. +- **Delve:** A Go debugger. +- **Profiling Tools:** `pprof` is the built-in Go profiler. + +### 7.2 Build Configuration + +- **Makefile:** Use a Makefile to automate common build tasks. +- **`go build`:** Use `go build` to compile the application. +- **`go test`:** Use `go test` to run tests. + +### 7.3 Linting and Formatting + +- **`go fmt`:** Use `go fmt` to format the code according to the Go style guidelines. +- **`go vet`:** Use `go vet` to identify potential errors in the code. +- **`golangci-lint`:** Use `golangci-lint` for more advanced linting. + +### 7.4 Deployment Best Practices + +- **Containerization:** Use containers (e.g., Docker) to package the application and its dependencies. +- **Orchestration:** Use orchestration tools (e.g., Kubernetes) to manage and scale the application. +- **Immutable Infrastructure:** Deploy immutable infrastructure to ensure consistency and reproducibility. + +### 7.5 CI/CD Integration + +- **Continuous Integration:** Integrate with a CI/CD pipeline to automate the build, test, and deployment process. +- **Automated Testing:** Run automated tests as part of the CI/CD pipeline. +- **Automated Deployment:** Automate the deployment process using tools like Jenkins, CircleCI, or GitHub Actions. + +## Additional Best Practices + +- **Meaningful Handler Names:** Use meaningful, concise names for handlers, avoiding repetition of context in function names (as per the Google Style Guide). +- **Document Code:** Write clear and concise documentation for all functions, methods, and types. +- **Code Reviews:** Conduct regular code reviews to ensure code quality and identify potential issues. +- **Stay Updated:** Stay up-to-date with the latest Go best practices and security vulnerabilities. + +By following these best practices and coding standards, you can develop robust, maintainable, and efficient applications using the `net/http` ServeMux in Go. \ No newline at end of file diff --git a/.cursor/rules/setuptools.mdc b/.cursor/rules/setuptools.mdc new file mode 100644 index 000000000..f7fb3a3ef --- /dev/null +++ b/.cursor/rules/setuptools.mdc @@ -0,0 +1,239 @@ +--- +description: This rule provides guidance on best practices for using setuptools in Python projects, covering code organization, performance, security, testing, and common pitfalls. +globs: **/setup.py +--- +# setuptools Best Practices + +This document outlines best practices for using `setuptools` in Python projects. Following these guidelines ensures maintainable, performant, and secure code. + +## Library Information: +- Name: setuptools +- Tags: development, build-tool, python, packaging + +## 1. Code Organization and Structure: + +- **Directory Structure:** + - At the root: + - `setup.py`: The main setup script. + - `setup.cfg`: Configuration file for `setup.py` (optional but recommended). + - `pyproject.toml`: Build system configuration (modern approach). + - `README.md` or `README.rst`: Project description. + - `LICENSE.txt`: License information. + - `.gitignore`: Specifies intentionally untracked files that Git should ignore. + - A top-level package directory (e.g., `my_package`): + - `my_package/`: + - `__init__.py`: Makes the directory a Python package. + - `module1.py`: Module containing code. + - `module2.py`: Another module. + - `data/`: Directory for package data (optional). + - `tests/`: + - `test_module1.py`: Unit tests for `module1.py`. + - `test_module2.py`: Unit tests for `module2.py`. + - `conftest.py`: Configuration file for pytest. + - `__init__.py` (optional). + - `docs/` (optional): + - Project documentation (e.g., using Sphinx). + +- **File Naming Conventions:** + - Python modules: `module_name.py` (snake_case). + - Test files: `test_module_name.py` or `module_name_test.py`. + - Package directories: `package_name/` (lowercase). + - Configuration files: `setup.cfg`, `pyproject.toml`, `MANIFEST.in`. + +- **Module Organization:** + - Group related functions and classes within a module. + - Keep modules focused on a specific responsibility. + - Use clear and descriptive module names. + - Utilize subpackages for logical grouping of modules within larger projects. + +- **Component Architecture:** + - Favor modular design, breaking down the project into reusable components. + - Define clear interfaces between components. + - Follow the Single Responsibility Principle (SRP) for each component. + - Consider using a layered architecture if appropriate for your project's complexity. + +- **Code Splitting Strategies:** + - Split large modules into smaller, more manageable files. + - Decompose complex functions into smaller, well-defined functions. + - Extract reusable code into separate modules or packages. + - Employ lazy loading for modules that are not immediately needed. + +## 2. Common Patterns and Anti-patterns: + +- **Design Patterns:** + - **Factory Pattern:** Use factory functions or classes to create instances of objects, especially when complex initialization is required. + - **Dependency Injection:** Inject dependencies into classes or functions to improve testability and reduce coupling. + - **Facade Pattern:** Provide a simplified interface to a complex subsystem. + - **Singleton Pattern:** Use sparingly; ensure thread safety if needed. + - **Observer Pattern:** Useful for event handling and asynchronous operations. + +- **Recommended Approaches:** + - **Declaring Dependencies:** Use `install_requires` in `setup.py` or `pyproject.toml` (preferred) to specify project dependencies. + - **Managing Package Data:** Use `package_data` in `setup.py` to include non-code files (e.g., data files, templates) within your package. + - **Creating Entry Points:** Define console scripts using `entry_points` in `setup.py` or `pyproject.toml` to create command-line tools. + - **Versioning:** Follow Semantic Versioning (SemVer) to clearly communicate the nature of changes in each release. + - **Using `pyproject.toml`:** Utilize `pyproject.toml` for build system configuration, build dependencies and other metadata. This aligns with modern packaging practices. + +- **Anti-patterns and Code Smells:** + - **Large `setup.py` Files:** Keep `setup.py` concise; move complex logic to separate modules. + - **Hardcoded Paths:** Avoid hardcoding file paths; use relative paths or the `pkg_resources` module to access package data. + - **Global State:** Minimize the use of global variables; prefer passing state as arguments to functions or methods. + - **Ignoring Errors:** Always handle exceptions appropriately; never ignore errors without logging or taking corrective action. + - **Over-engineering:** Avoid unnecessary complexity; keep the design simple and focused. + +- **State Management:** + - For CLI tools, use configuration files or environment variables to store persistent state. + - For more complex applications, consider using a database or other persistent storage mechanism. + - Avoid storing sensitive information in plain text configuration files; use encryption or secure storage. + +- **Error Handling:** + - Use `try...except` blocks to handle exceptions gracefully. + - Log errors with sufficient context for debugging. + - Raise custom exceptions to provide more specific error information. + - Avoid catching generic exceptions (`except Exception:`) unless you re-raise them or log the error. + +## 3. Performance Considerations: + +- **Optimization Techniques:** + - **Code Profiling:** Use profiling tools (e.g., `cProfile`) to identify performance bottlenecks. + - **Algorithm Optimization:** Choose efficient algorithms and data structures. + - **Caching:** Implement caching mechanisms to reduce redundant computations. + - **Code Optimization:** Use efficient code constructs and avoid unnecessary operations. + - **Concurrency/Parallelism:** Consider using threading or multiprocessing for CPU-bound tasks. + +- **Memory Management:** + - Use generators or iterators to process large datasets without loading everything into memory. + - Release resources (e.g., file handles, network connections) promptly. + - Avoid creating unnecessary copies of data. + - Be mindful of memory leaks; use memory profiling tools to detect them. + +- **Bundle Size Optimization:** + - Minimize the size of your package by excluding unnecessary files (e.g., test files, documentation) from the distribution. + - Use compression to reduce the size of package data. + - Consider using a smaller version of a dependency or only requiring features of the dependency that you need. + +- **Lazy Loading:** + - Defer loading of modules or data until they are actually needed. + - Use the `importlib` module to dynamically import modules at runtime. + - Implement lazy properties to defer the computation of attribute values. + +## 4. Security Best Practices: + +- **Common Vulnerabilities:** + - **Dependency Confusion:** Prevent dependency confusion attacks by using a unique package name and verifying dependencies. + - **Arbitrary Code Execution:** Avoid using `eval()` or `exec()` to execute untrusted code. + - **Injection Attacks:** Sanitize user inputs to prevent injection attacks (e.g., SQL injection, command injection). + - **Cross-Site Scripting (XSS):** Encode output to prevent XSS attacks, particularly when dealing with web interfaces. + +- **Input Validation:** + - Validate all user inputs to ensure they conform to expected formats and ranges. + - Use regular expressions or validation libraries to enforce input constraints. + - Sanitize inputs to remove or escape potentially malicious characters. + +- **Authentication and Authorization:** + - Use secure authentication mechanisms (e.g., OAuth 2.0, JWT) to verify user identities. + - Implement authorization checks to ensure that users only have access to the resources they are authorized to access. + - Store passwords securely using hashing algorithms (e.g., bcrypt, scrypt). + +- **Data Protection:** + - Encrypt sensitive data at rest and in transit. + - Use HTTPS to secure communication between clients and servers. + - Implement data masking or anonymization techniques to protect personally identifiable information (PII). + +- **Secure API Communication:** + - Use API keys or tokens to authenticate API requests. + - Enforce rate limiting to prevent denial-of-service attacks. + - Validate API requests and responses to ensure data integrity. + +## 5. Testing Approaches: + +- **Unit Testing:** + - Write unit tests for each module and function to verify their correctness. + - Use mocking or stubbing to isolate units of code from their dependencies. + - Aim for high test coverage to ensure that all code paths are tested. + - Use `pytest` and `unittest` modules. + +- **Integration Testing:** + - Write integration tests to verify the interactions between different components of the system. + - Test the integration with external dependencies (e.g., databases, APIs). + - Use test doubles or test environments to simulate external dependencies. + +- **End-to-End Testing:** + - Write end-to-end tests to verify the functionality of the entire system from the user's perspective. + - Use browser automation tools (e.g., Selenium, Playwright) to simulate user interactions. + - Test the system in a realistic environment. + +- **Test Organization:** + - Organize tests into separate directories (e.g., `tests/`). + - Use descriptive test names to clearly indicate what each test is verifying. + - Group related tests into test classes or modules. + - Use test fixtures to set up and tear down test environments. + +- **Mocking and Stubbing:** + - Use mocking libraries (e.g., `unittest.mock`, `pytest-mock`) to create mock objects that simulate the behavior of dependencies. + - Use stubbing to replace dependencies with simplified versions that return predefined values. + - Avoid over-mocking; only mock dependencies that are difficult to test directly. + +## 6. Common Pitfalls and Gotchas: + +- **Frequent Mistakes:** + - **Incorrect Dependency Specifications:** Ensure that dependencies are correctly specified in `install_requires` or `pyproject.toml` with accurate version constraints. + - **Missing Package Data:** Remember to include non-code files in `package_data` if they are required at runtime. + - **Inconsistent Versioning:** Follow Semantic Versioning (SemVer) consistently and update version numbers appropriately. + - **Ignoring Encoding Issues:** Handle text encoding correctly to avoid errors when dealing with non-ASCII characters. + - **Failing to Test:** Write comprehensive tests to catch errors early and prevent regressions. + +- **Edge Cases:** + - **Platform-Specific Issues:** Test your package on different operating systems and Python versions to identify platform-specific issues. + - **Dependency Conflicts:** Be aware of potential dependency conflicts and use virtual environments to isolate your project's dependencies. + - **Unicode Handling:** Handle Unicode characters correctly, especially when dealing with user inputs or file names. + - **Resource Limits:** Be mindful of resource limits (e.g., memory, file handles) when processing large datasets. + +- **Version-Specific Issues:** + - Be aware of compatibility issues between different versions of setuptools. + - Consult the setuptools documentation for version-specific recommendations. + +- **Compatibility Concerns:** + - Test your package with different versions of Python to ensure compatibility. + - Be aware of potential conflicts with other libraries or frameworks. + - Use conditional imports or feature detection to handle compatibility issues gracefully. + +- **Debugging Strategies:** + - Use a debugger to step through your code and inspect variables. + - Add logging statements to track the flow of execution and identify errors. + - Use unit tests to isolate and reproduce bugs. + - Consult online resources (e.g., Stack Overflow) and the setuptools documentation for troubleshooting tips. + +## 7. Tooling and Environment: + +- **Recommended Development Tools:** + - **Virtual Environment Managers:** `venv`, `virtualenv`, `conda` or Poetry to create isolated Python environments. + - **Package Managers:** `pip` or Poetry to install and manage dependencies. + - **Code Editors:** VS Code, PyCharm, Sublime Text, or other IDEs with Python support. + - **Debuggers:** `pdb` or IDE-integrated debuggers. + - **Profilers:** `cProfile` to identify performance bottlenecks. + - **Testing Frameworks:** `pytest` or `unittest` for writing and running tests. + - **Linters and Formatters:** `flake8`, `pylint`, `black`, and `isort` to enforce code style and quality. + - **Build System:** `build` to build distributions. + +- **Build Configuration:** + - Use `setup.cfg` or `pyproject.toml` to configure the build process. + - Specify dependencies, package data, and entry points in the configuration file. + - Use build scripts to automate build tasks. + +- **Linting and Formatting:** + - Use `flake8` or `pylint` to enforce code style guidelines. + - Use `black` to automatically format your code. + - Use `isort` to sort imports alphabetically. + +- **Deployment:** + - Use `twine` to securely upload your package to PyPI. + - Use virtual environments to isolate your application's dependencies in production. + - Consider using containerization (e.g., Docker) to create reproducible deployment environments. + +- **CI/CD:** + - Use CI/CD systems (e.g., GitHub Actions, Jenkins, Travis CI, CircleCI) to automate building, testing, and deploying your package. + - Configure CI/CD pipelines to run tests, linters, and formatters on every commit. + - Automate the release process using CI/CD. + +By adhering to these best practices, you can effectively leverage `setuptools` to create well-structured, maintainable, and high-quality Python packages. \ No newline at end of file diff --git a/.cursor/rules/shadcn.mdc b/.cursor/rules/shadcn.mdc new file mode 100644 index 000000000..adc4e0a1c --- /dev/null +++ b/.cursor/rules/shadcn.mdc @@ -0,0 +1,192 @@ +--- +description: This rule provides comprehensive best practices for developing with Shadcn UI, covering code organization, performance, security, and testing. +globs: *.js,*.jsx,*.ts,*.tsx +--- +# Shadcn UI Best Practices + +This document outlines best practices for developing with Shadcn UI, covering code organization, common patterns, performance considerations, security, testing, common pitfalls, and tooling. + +## 1. Code Organization and Structure + +- **Directory Structure:** + - Organize components into logical directories based on functionality or domain. For example, place form-related components in a `components/forms` directory. + - Separate components into their own files. Each component should have a dedicated file named after the component (e.g., `Button.tsx`). + - Consider using an `index.ts` file within each directory to export all components from that directory, simplifying imports. + - Structure directories to reflect the UI hierarchy. For example, `/components/layout` for layout related components and `/components/ui` for reusable UI elements. + +- **File Naming Conventions:** + - Use PascalCase for component file names (e.g., `MyComponent.tsx`). + - Use camelCase for variable and function names (e.g., `handleClick`). + - Use descriptive names that clearly indicate the purpose of the component or function. + +- **Module Organization:** + - Break down complex components into smaller, reusable modules. + - Keep components focused on a single responsibility. + - Utilize shared utility functions and constants to avoid code duplication. Create a `utils` directory for helper functions. + - Use `components` directory to store UI components. + +- **Component Architecture:** + - Favor composition over inheritance. Create flexible components that can be customized through props. + - Design components with clear separation of concerns: presentational components (UI) and container components (logic). + - Use functional components with hooks for managing state and side effects. + +- **Code Splitting Strategies:** + - Implement lazy loading for non-critical components to improve initial load time. + - Utilize React.lazy and Suspense for code splitting at the component level. + - Configure your bundler (e.g., Webpack, Parcel) to automatically split code into smaller chunks. + - Consider route-based code splitting for larger applications. + +## 2. Common Patterns and Anti-patterns + +- **Design Patterns Specific to Shadcn UI:** + - Leverage the existing components provided by Shadcn UI whenever possible. + - Customize components using styling solutions like Tailwind CSS's utility classes or CSS variables. + - Create compound components by combining existing Shadcn UI components to build more complex UI elements. + +- **Recommended Approaches for Common Tasks:** + - Use Shadcn UI's form components (e.g., `Input`, `Select`) for handling user input. + - Implement accessible components by following ARIA guidelines and using appropriate HTML semantics. + - Use the `cn` utility (classnames library) provided by Shadcn UI to manage CSS class names effectively. + +- **Anti-patterns and Code Smells to Avoid:** + - Directly modifying the Shadcn UI component code. + - Overusing custom CSS, as Shadcn UI is built with Tailwind CSS. + - Neglecting accessibility considerations. + - Creating overly complex components with too many responsibilities. + +- **State Management Best Practices:** + - Use React's built-in `useState` hook for simple component-level state. + - Consider using a state management library like Zustand, Redux, or Recoil for more complex application state. + - Avoid mutating state directly; always use the setState function or a state management library's update methods. + +- **Error Handling Patterns:** + - Implement error boundaries to catch errors in components and prevent the entire application from crashing. + - Use try-catch blocks to handle errors in asynchronous operations and API calls. + - Provide informative error messages to users. + - Log errors to a monitoring service for debugging and analysis. + +## 3. Performance Considerations + +- **Optimization Techniques:** + - Minimize re-renders by using `React.memo` for functional components and `shouldComponentUpdate` for class components. + - Optimize event handlers by using useCallback to prevent unnecessary re-creation of functions. + - Debounce or throttle expensive operations to reduce the frequency of execution. + +- **Memory Management:** + - Avoid memory leaks by properly cleaning up event listeners and timers in the `useEffect` hook. + - Release unused resources, such as large data structures, when they are no longer needed. + +- **Rendering Optimization:** + - Use virtualized lists or grids for rendering large datasets. + - Batch DOM updates to minimize reflows and repaints. + - Use CSS containment to isolate rendering changes to specific parts of the DOM. + +- **Bundle Size Optimization:** + - Remove unused code and dependencies using tree shaking. + - Minify JavaScript and CSS files to reduce their size. + - Compress images using tools like ImageOptim or TinyPNG. + +- **Lazy Loading Strategies:** + - Implement lazy loading for images and other media assets. + - Use the Intersection Observer API to detect when elements are visible in the viewport and load them on demand. + +## 4. Security Best Practices + +- **Common Vulnerabilities and How to Prevent Them:** + - Prevent cross-site scripting (XSS) attacks by sanitizing user input and escaping HTML entities. + - Protect against cross-site request forgery (CSRF) attacks by using anti-CSRF tokens. + - Avoid storing sensitive information, such as API keys or passwords, in client-side code. + +- **Input Validation:** + - Validate user input on both the client-side and server-side. + - Use a validation library like Zod or Yup to define data schemas and enforce validation rules. + - Sanitize user input to remove potentially harmful characters or code. + +- **Authentication and Authorization Patterns:** + - Use a secure authentication protocol, such as OAuth 2.0 or OpenID Connect. + - Implement role-based access control (RBAC) to restrict access to sensitive resources. + - Store user credentials securely using hashing and salting. + +- **Data Protection Strategies:** + - Encrypt sensitive data at rest and in transit. + - Use HTTPS to protect data transmitted between the client and server. + - Implement data masking to hide sensitive information from unauthorized users. + +- **Secure API Communication:** + - Use HTTPS for all API requests. + - Implement rate limiting to prevent abuse and denial-of-service attacks. + - Validate API responses to ensure data integrity. + +## 5. Testing Approaches + +- **Unit Testing Strategies:** + - Write unit tests for individual components and functions. + - Use a testing framework like Jest or Mocha. + - Test component behavior with different props and inputs. + +- **Integration Testing:** + - Write integration tests to verify that components work together correctly. + - Test the interaction between components and APIs. + +- **End-to-End Testing:** + - Write end-to-end tests to simulate user interactions and verify that the application functions as expected. + - Use a testing framework like Cypress or Playwright. + +- **Test Organization:** + - Organize tests into separate files based on the component or feature being tested. + - Use descriptive test names that clearly indicate the purpose of the test. + +- **Mocking and Stubbing:** + - Use mocking and stubbing to isolate components and functions during testing. + - Mock external dependencies, such as APIs or third-party libraries. + +## 6. Common Pitfalls and Gotchas + +- **Frequent Mistakes Developers Make:** + - Forgetting to handle edge cases. + - Overcomplicating components. + - Neglecting accessibility. + - Ignoring performance considerations. + +- **Edge Cases to Be Aware Of:** + - Handling different screen sizes and devices. + - Dealing with slow network connections. + - Handling invalid or unexpected user input. + +- **Version-Specific Issues:** + - Be aware of breaking changes between Shadcn UI versions. + - Consult the Shadcn UI changelog for migration instructions. + +- **Compatibility Concerns:** + - Ensure that your application is compatible with the target browsers and devices. + - Test your application on different browsers and devices. + +- **Debugging Strategies:** + - Use browser developer tools to inspect the DOM and debug JavaScript code. + - Use console logging to track the flow of execution and identify errors. + - Use a debugger to step through code and inspect variables. + +## 7. Tooling and Environment + +- **Recommended Development Tools:** + - Visual Studio Code (VS Code) with extensions for React, TypeScript, and Tailwind CSS. + - A browser with developer tools (e.g., Chrome DevTools, Firefox Developer Tools). + - A terminal for running commands and scripts. + +- **Build Configuration:** + - Use a build tool like Webpack, Parcel, or Rollup to bundle your application. + - Configure your build tool to optimize code for production. + +- **Linting and Formatting:** + - Use ESLint to enforce code style and identify potential errors. + - Use Prettier to automatically format code. + - Configure your editor to automatically lint and format code on save. + +- **Deployment Best Practices:** + - Deploy your application to a reliable hosting provider. + - Use a content delivery network (CDN) to serve static assets. + - Configure your server to serve compressed files. + +- **CI/CD Integration:** + - Use a continuous integration and continuous deployment (CI/CD) pipeline to automate the build, test, and deployment process. + - Integrate your CI/CD pipeline with your version control system. \ No newline at end of file diff --git a/.cursor/rules/smolagents.mdc b/.cursor/rules/smolagents.mdc new file mode 100644 index 000000000..8943f5633 --- /dev/null +++ b/.cursor/rules/smolagents.mdc @@ -0,0 +1,286 @@ +--- +description: This rule provides comprehensive best practices for developing with the smolagents library, covering code organization, performance, security, testing, and common pitfalls. It aims to guide developers in building robust, maintainable, and efficient AI agent applications. +globs: *.py +--- +# smolagents Best Practices and Coding Standards + +This document outlines the best practices and coding standards for developing with the smolagents library. Following these guidelines will help you create robust, maintainable, and efficient AI agent applications. + +## Library Information: +- Name: smolagents +- Tags: ai, ml, llm, python, agent-framework, lightweight + +## Core Principles +- **Simplicity:** Embrace smolagents' focus on minimal code and clear abstractions. +- **Documentation:** Thoroughly document your code, as AI models lack memory and context. +- **Security:** Prioritize secure coding practices to prevent vulnerabilities. +- **Code Review:** Regularly review code to catch design errors and maintain quality. +- **Code-First Approach:** Utilize smolagents' code-first approach, where agents write their actions in Python. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure Best Practices + + +project_root/ +├── agents/ +│ ├── __init__.py +│ ├── my_agent.py # Agent definitions +│ └── ... +├── tools/ +│ ├── __init__.py +│ ├── web_search_tool.py # Custom tool definitions +│ └── ... +├── models/ +│ ├── __init__.py +│ ├── llm_model.py # Model configurations +│ └── ... +├── config/ +│ ├── config.py # Configuration settings +│ └── ... +├── tests/ +│ ├── __init__.py +│ ├── test_agents.py # Unit and integration tests +│ └── ... +├── utils/ +│ ├── __init__.py +│ ├── helper_functions.py # Helper functions +│ └── ... +├── main.py # Entry point for the application +├── requirements.txt # Project dependencies +├── README.md # Project documentation +└── .env # Environment variables + + +- **agents/:** Contains the definitions for your AI agents. +- **tools/:** Holds custom tools that your agents can use. +- **models/:** Defines configurations for LLMs (e.g., Hugging Face models). +- **config/:** Stores configuration settings for your application. +- **tests/:** Includes unit, integration, and end-to-end tests. +- **utils/:** Contains helper functions and utilities. +- **main.py:** The entry point for your application. +- **.env:** Store sensitive information and configuration variables. + +### 1.2 File Naming Conventions + +- Use descriptive and consistent names for files and modules. +- Agent files: `my_agent.py`, `task_specific_agent.py` +- Tool files: `web_search_tool.py`, `data_analysis_tool.py` +- Model files: `llm_model.py`, `embedding_model.py` +- Configuration files: `config.py`, `api_keys.py` +- Test files: `test_agents.py`, `test_tools.py` + +### 1.3 Module Organization + +- Group related functionalities into modules. +- Use `__init__.py` files to define packages. +- Keep modules focused on a specific task or domain. + +### 1.4 Component Architecture + +- Design your application with reusable components. +- Agents, tools, and models should be modular and easily swappable. +- Use interfaces to define contracts between components. + +### 1.5 Code Splitting Strategies + +- Break down large agent implementations into smaller, manageable functions and classes. +- Consider splitting code based on different stages of the agent's workflow (e.g., planning, execution, evaluation). +- Use separate modules for reusable logic. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns + +- **Tool Pattern:** Encapsulate external functionalities (e.g., web search, data analysis) into reusable tools. Example: + python + from smolagents import tool + + @tool + def search_wikipedia(query: str) -> str: + """Searches Wikipedia for the given query.""" + # Implementation + pass + +- **Chain of Responsibility:** Implement agent workflows as a chain of tasks or sub-agents. +- **Strategy Pattern:** Allow agents to switch between different strategies (e.g., different reasoning approaches) at runtime. + +### 2.2 Recommended Approaches + +- **Secure Code Execution:** Use sandboxed environments (e.g., E2B) to execute agent code safely. +- **Hub Integration:** Leverage the Hugging Face Hub for tool sharing and discovery. +- **Model Agnosticism:** Design your agents to work with different LLMs. + +### 2.3 Anti-patterns and Code Smells + +- **Hardcoding API Keys:** Store API keys in environment variables or configuration files, not directly in the code. +- **Ignoring Errors:** Implement proper error handling and logging. +- **Overly Complex Agents:** Keep agent logic simple and focused. +- **Lack of Documentation:** Always document your code, especially agent logic. + +### 2.4 State Management + +- Use classes to encapsulate agent state. +- Consider using a state management library for more complex applications. +- Avoid global state. + +### 2.5 Error Handling + +- Use `try-except` blocks to handle exceptions. +- Log errors with meaningful messages. +- Implement retry mechanisms for transient errors. +- Raise custom exceptions for specific error conditions. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +- **Caching:** Cache LLM responses to avoid redundant API calls. +- **Asynchronous Operations:** Use asynchronous operations for I/O-bound tasks. +- **Efficient Data Structures:** Choose appropriate data structures for your use case. +- **Prompt Optimization:** Craft prompts carefully to reduce LLM processing time. + +### 3.2 Memory Management + +- Be mindful of memory usage when processing large datasets. +- Use generators to process data in chunks. +- Delete unnecessary objects to free up memory. + +### 3.3 Rendering Optimization (If Applicable) + +- If your agent interacts with a user interface, optimize rendering performance by minimizing DOM updates and using efficient rendering techniques. + +### 3.4 Bundle Size Optimization (If Applicable) + +- If you are deploying your agent as a web application, optimize bundle size by removing unused code and using code splitting. + +### 3.5 Lazy Loading + +- Load modules and data only when they are needed to improve startup time. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities + +- **Prompt Injection:** Protect against malicious prompts that can manipulate the agent's behavior. +- **Code Injection:** Prevent agents from executing arbitrary code. +- **Data Leakage:** Ensure that sensitive data is not exposed. + +### 4.2 Input Validation + +- Validate all inputs to your agents to prevent injection attacks. +- Sanitize inputs to remove potentially harmful characters. + +### 4.3 Authentication and Authorization + +- Implement authentication and authorization to control access to your agents. +- Use secure credentials management practices. + +### 4.4 Data Protection + +- Encrypt sensitive data at rest and in transit. +- Use secure storage solutions for sensitive data. + +### 4.5 Secure API Communication + +- Use HTTPS for all API communication. +- Validate API responses to prevent data tampering. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +- Write unit tests for individual components (agents, tools, models). +- Use mocking to isolate components during testing. + +### 5.2 Integration Testing + +- Test the interaction between different components. +- Verify that the agent workflow works as expected. + +### 5.3 End-to-End Testing + +- Test the entire application from end to end. +- Simulate user interactions to verify functionality. + +### 5.4 Test Organization + +- Organize tests into separate modules based on functionality. +- Use clear and descriptive test names. + +### 5.5 Mocking and Stubbing + +- Use mocking to replace external dependencies with controlled substitutes. +- Use stubbing to provide predefined responses for specific inputs. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +- **Ignoring Documentation:** Failing to read the smolagents documentation thoroughly. +- **Overcomplicating Agents:** Creating agents that are too complex and difficult to maintain. +- **Not Using Sandboxed Environments:** Running agent code without proper security measures. +- **Failing to Update Documentation:** Allowing code documentation to become outdated. + +### 6.2 Edge Cases + +- Handle edge cases and unexpected inputs gracefully. +- Test your agents with a variety of inputs to identify potential issues. + +### 6.3 Version-Specific Issues + +- Be aware of version-specific issues and compatibility concerns when upgrading smolagents. +- Consult the release notes for information on breaking changes. + +### 6.4 Compatibility Concerns + +- Ensure that smolagents is compatible with other technologies you are using. +- Test your application with different versions of dependencies. + +### 6.5 Debugging Strategies + +- Use logging to track the execution of your agents. +- Use a debugger to step through code and inspect variables. +- Use print statements to debug simple issues. + +## 7. Tooling and Environment + +### 7.1 Recommended Tools + +- **VS Code:** A popular code editor with excellent Python support and extensions for AI development. +- **PyCharm:** A dedicated Python IDE with advanced features for debugging and testing. +- **Docker:** A containerization platform for deploying your agents in isolated environments. +- **E2B:** A secure sandboxed environment for executing agent code. + +### 7.2 Build Configuration + +- Use a build system like `poetry` or `pipenv` to manage dependencies. +- Define build scripts for common tasks like testing and linting. + +### 7.3 Linting and Formatting + +- Use a linter like `pylint` or `flake8` to enforce code style. +- Use a formatter like `black` to automatically format your code. + +### 7.4 Deployment + +- Deploy your agents to a cloud platform like AWS, Azure, or Google Cloud. +- Use containerization to ensure consistent deployment across different environments. + +### 7.5 CI/CD + +- Integrate your project with a CI/CD pipeline to automate testing and deployment. +- Use a CI/CD tool like Jenkins, GitLab CI, or GitHub Actions. + +## Additional Recommendations + +- **Prompt Engineering:** + - Be extremely careful about how you craft the prompts being sent to the LLM. The quality of the prompt directly correlates to the quality of the output. Consider using techniques like few-shot learning, chain-of-thought prompting, and prompt templates. +- **Modular Tools:** + - Design tools to be as reusable and composable as possible. This will allow you to create more complex agent workflows by combining simpler tools. +- **Cost Tracking:** + - When using paid LLM APIs, it's important to track the cost of each agent run. This can help you optimize your prompts and agent workflows to reduce costs. +- **Monitoring:** + - Monitor your agents in production to identify issues and performance bottlenecks. Implement logging and alerting to catch errors early. + +By following these best practices, you can develop robust, maintainable, and efficient AI agent applications with smolagents. \ No newline at end of file diff --git a/.cursor/rules/socket-io.mdc b/.cursor/rules/socket-io.mdc new file mode 100644 index 000000000..2ec76f3df --- /dev/null +++ b/.cursor/rules/socket-io.mdc @@ -0,0 +1,302 @@ +--- +description: This rule provides guidelines and best practices for developing robust, scalable, and secure real-time applications using Socket.IO. It covers code organization, performance optimization, security considerations, testing strategies, and common pitfalls to avoid when working with Socket.IO. +globs: *.js,*.ts,*.jsx,*.tsx +--- +# Socket.IO Best Practices + +This document outlines best practices for developing robust, scalable, and secure real-time applications using Socket.IO. It covers various aspects, from code organization to security considerations. + +## Library Information: +- Name: socket-io +- Tags: websockets, real-time, javascript, communication + +## 1. Code Organization and Structure + +A well-organized codebase is crucial for maintainability and scalability. For Socket.IO projects, consider the following structure: + + +project-root/ +├── node_modules/ +├── src/ +│ ├── app.ts (or app.js) # Main application entry point +│ ├── config/ # Configuration files +│ │ └── socket.ts # Socket.IO configuration +│ ├── socket/ # Socket event handlers and logic +│ │ ├── index.ts # Centralized socket connection and event handling +│ │ ├── events/ # Directory for different socket event modules +│ │ │ ├── chat.ts # Chat-related events +│ │ │ ├── notifications.ts # Notification-related events +│ │ │ └── ... # Other event modules +│ │ ├── middleware/ # Socket.IO middleware for authentication, etc. +│ │ │ └── auth.ts # Example authentication middleware +│ │ └── utils/ # Utility functions for socket operations +│ ├── models/ # Data models +│ ├── services/ # Business logic services +│ ├── utils/ # Utility functions +│ ├── types/ # TypeScript type definitions +│ └── public/ # Static assets (if applicable) +├── tests/ # Unit and integration tests +├── .env # Environment variables +├── package.json # Project dependencies and scripts +├── tsconfig.json # TypeScript configuration (if using TypeScript) +└── README.md # Project documentation + + +### File Naming Conventions + +* Use descriptive names for files and directories. +* Follow a consistent naming convention (e.g., `camelCase` or `kebab-case`). +* For TypeScript projects, use `.ts` for source files and `.d.ts` for declaration files. + +### Module Organization + +* Break down your application into smaller, modular components. +* Use ES modules (import/export) or CommonJS (require) for module organization. +* Consider using a dependency injection container for managing dependencies. + +### Component Architecture + +* Adopt a component-based architecture to promote reusability and separation of concerns. +* Create reusable components for common Socket.IO tasks, such as message handling or authentication. +* Utilize design patterns like the Observer pattern for managing socket events. + +### Code Splitting + +* For large applications, consider using code splitting to reduce the initial bundle size. +* Load socket event handlers on demand when they are needed. + +## 2. Common Patterns and Anti-patterns + +### Design Patterns + +* **Observer Pattern:** Used extensively in Socket.IO to manage real-time updates. +* **Factory Pattern:** Create socket instances and event handlers. +* **Middleware Pattern:** Implement authentication, authorization, and data validation. + +### Recommended Approaches + +* Use a class-based approach to encapsulate socket-related logic. +* Organize your code into namespaces and rooms to manage different parts of your application effectively. +* Implement automatic reconnection and handle disconnections gracefully. +* Use heartbeats to maintain active connections. + +### Anti-patterns + +* **Global Socket Instance:** Avoid using a global socket instance. Instead, pass the socket instance to the relevant components. +* **Overly Complex Event Handlers:** Keep event handlers small and focused. Delegate complex logic to separate functions or services. +* **Ignoring Errors:** Always handle errors properly and log them for debugging purposes. +* **Sending Large Payloads:** Avoid sending large payloads over Socket.IO. Optimize your data structures and compress data if necessary. +* **Tight Coupling:** Avoid tight coupling between your socket event handlers and your application logic. Use dependency injection or other techniques to decouple your code. + +### State Management + +* Use a centralized state management solution (e.g., Redux, Zustand, or a simple in-memory store) to manage the state of your Socket.IO application. +* Keep the state synchronized between the client and the server. +* Use immutable data structures to simplify state management and prevent unexpected side effects. + +### Error Handling + +* Use try-catch blocks to handle synchronous errors. +* Use promise rejections to handle asynchronous errors. +* Implement a global error handler to catch unhandled exceptions. +* Log all errors to a file or a monitoring service. +* Consider using a circuit breaker pattern to prevent cascading failures. +* Inform the client about errors in a user-friendly way. + +## 3. Performance Considerations + +### Optimization Techniques + +* Minimize data transmission size and frequency. +* Use data compression techniques (e.g., gzip or brotli) to reduce payload sizes. +* Optimize message payloads (e.g., use binary data instead of JSON strings). +* Use namespaces and rooms to target messages to specific clients. +* Implement pagination or filtering to reduce the amount of data sent to the client. + +### Memory Management + +* Monitor memory usage and identify memory leaks. +* Use garbage collection to reclaim unused memory. +* Avoid creating large objects in memory. +* Use streams to process large data sets. + +### Rendering Optimization + +* Use virtual DOM techniques to minimize DOM updates. +* Batch DOM updates to improve performance. +* Use CSS transforms and animations instead of JavaScript animations. + +### Bundle Size Optimization + +* Use a bundler (e.g., Webpack, Parcel, or Rollup) to optimize your JavaScript bundles. +* Minify and compress your JavaScript code. +* Remove unused code (dead code elimination). +* Use code splitting to load code on demand. + +### Lazy Loading + +* Load socket event handlers on demand when they are needed. +* Lazy load images and other assets. +* Use dynamic imports to load modules on demand. + +## 4. Security Best Practices + +### Common Vulnerabilities + +* **Cross-Site Scripting (XSS):** Prevent XSS attacks by sanitizing user input and encoding output. +* **SQL Injection:** Prevent SQL injection attacks by using parameterized queries or an ORM. +* **Denial-of-Service (DoS):** Prevent DoS attacks by limiting request rates and using a CDN. +* **Man-in-the-Middle (MitM):** Prevent MitM attacks by using HTTPS and validating SSL/TLS certificates. +* **Unauthorized Access:** Prevent unauthorized access by implementing proper authentication and authorization mechanisms. + +### Input Validation + +* Validate all incoming data on the server-side. +* Use a schema validation library (e.g., Joi or Yup) to define and enforce data schemas. +* Sanitize user input to prevent XSS attacks. +* Escape special characters to prevent SQL injection attacks. + +### Authentication and Authorization + +* Use a strong authentication mechanism (e.g., JSON Web Tokens (JWT) or OAuth 2.0). +* Implement role-based access control (RBAC) to restrict access to sensitive resources. +* Use HTTPS to protect authentication credentials in transit. +* Store passwords securely using a hashing algorithm (e.g., bcrypt). +* Implement two-factor authentication (2FA) for enhanced security. +* Use Socket.IO middleware to authenticate users before allowing them to connect. + +### Data Protection + +* Encrypt sensitive data at rest and in transit. +* Use a strong encryption algorithm (e.g., AES-256). +* Store encryption keys securely. +* Implement data masking to protect sensitive data from unauthorized access. +* Comply with relevant data privacy regulations (e.g., GDPR or CCPA). + +### Secure API Communication + +* Use HTTPS for all API communication. +* Validate SSL/TLS certificates. +* Implement rate limiting to prevent DoS attacks. +* Use API keys or tokens to authenticate API requests. +* Log all API requests and responses for auditing purposes. + +## 5. Testing Approaches + +### Unit Testing + +* Write unit tests for individual components and functions. +* Use a testing framework (e.g., Jest, Mocha, or Jasmine). +* Mock dependencies to isolate the component being tested. +* Test edge cases and error conditions. +* Aim for high code coverage. + +### Integration Testing + +* Write integration tests to verify that different parts of your application work together correctly. +* Test the interaction between Socket.IO clients and the server. +* Use a testing framework (e.g., Supertest or Cypress). +* Set up a test environment that mimics the production environment. + +### End-to-End Testing + +* Write end-to-end tests to simulate real-world user scenarios. +* Use a testing framework (e.g., Selenium, Puppeteer, or Cypress). +* Test the entire application stack, from the client to the database. +* Test performance and scalability under load. + +### Test Organization + +* Organize your tests in a separate directory (e.g., `tests`). +* Use a consistent naming convention for test files. +* Group tests by component or feature. +* Write clear and concise test descriptions. + +### Mocking and Stubbing + +* Use mocking and stubbing to isolate components and simplify testing. +* Use a mocking library (e.g., Sinon.js or Jest's built-in mocking capabilities). +* Mock external dependencies, such as databases or APIs. +* Stub functions to control their behavior during testing. + +## 6. Common Pitfalls and Gotchas + +### Frequent Mistakes + +* Forgetting to handle disconnections gracefully. +* Not validating user input. +* Not securing WebSocket connections. +* Using a global socket instance. +* Overly complex event handlers. +* Ignoring errors. +* Sending large payloads. +* Tight coupling. +* Not monitoring memory usage. +* Not testing the application thoroughly. + +### Edge Cases + +* Handling network interruptions and reconnections. +* Dealing with slow or unreliable connections. +* Managing multiple concurrent connections. +* Handling large data sets. +* Dealing with different browser implementations of WebSockets. + +### Version-Specific Issues + +* Be aware of breaking changes between Socket.IO versions. +* Consult the Socket.IO changelog for information about version-specific issues. +* Test your application thoroughly after upgrading Socket.IO. + +### Compatibility Concerns + +* Ensure that your Socket.IO client and server versions are compatible. +* Be aware of compatibility issues between Socket.IO and other technologies, such as load balancers or firewalls. + +### Debugging Strategies + +* Use the Socket.IO client and server debug logs to troubleshoot issues. +* Use browser developer tools to inspect WebSocket traffic. +* Use a network monitoring tool (e.g., Wireshark) to capture and analyze network packets. +* Use a code debugger (e.g., VS Code's built-in debugger) to step through your code and inspect variables. + +## 7. Tooling and Environment + +### Recommended Tools + +* **Code Editor:** VS Code, Sublime Text, or Atom +* **Testing Framework:** Jest, Mocha, or Jasmine +* **Bundler:** Webpack, Parcel, or Rollup +* **Linting and Formatting:** ESLint and Prettier +* **Network Monitoring:** Wireshark +* **Load Testing:** Apache JMeter or Artillery + +### Build Configuration + +* Use a build tool (e.g., Webpack or Parcel) to automate the build process. +* Configure your build tool to minify and compress your JavaScript code. +* Use environment variables to configure your application for different environments (e.g., development, testing, and production). + +### Linting and Formatting + +* Use a linter (e.g., ESLint) to enforce code style and identify potential errors. +* Use a code formatter (e.g., Prettier) to automatically format your code. +* Configure your linter and formatter to work together seamlessly. +* Use a Git hook to run your linter and formatter before committing code. + +### Deployment Best Practices + +* Use a process manager (e.g., PM2 or Nodemon) to manage your Node.js application. +* Deploy your application to a cloud platform (e.g., AWS, Azure, or Google Cloud). +* Use a load balancer to distribute traffic across multiple servers. +* Use a CDN to serve static assets. +* Monitor your application's performance and uptime. + +### CI/CD Integration + +* Use a CI/CD pipeline to automate the build, test, and deployment process. +* Use a CI/CD tool (e.g., Jenkins, Travis CI, or CircleCI). +* Run unit tests, integration tests, and end-to-end tests as part of your CI/CD pipeline. +* Automate the deployment process to reduce the risk of human error. + +By following these best practices, you can develop robust, scalable, and secure real-time applications using Socket.IO. Remember to adapt these guidelines to your specific project requirements and context. \ No newline at end of file diff --git a/.cursor/rules/solidity.mdc b/.cursor/rules/solidity.mdc new file mode 100644 index 000000000..5c62fb674 --- /dev/null +++ b/.cursor/rules/solidity.mdc @@ -0,0 +1,308 @@ +--- +description: This rule provides best practices and coding standards for Solidity smart contract development, covering code organization, security, performance, and testing. +globs: *.sol +--- +# Solidity Smart Contract Best Practices and Coding Standards + +This guide provides comprehensive best practices and coding standards for writing secure, efficient, and maintainable Solidity smart contracts. It covers various aspects of smart contract development, from code organization to security considerations. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure + +* **contracts/:** Contains Solidity contract files (.sol). +* **interfaces/:** Contains interface definitions (.sol). +* **libraries/:** Contains reusable libraries (.sol). +* **test/:** Contains test scripts (e.g., using Hardhat, Foundry, or Truffle). +* **scripts/:** Contains deployment and utility scripts. +* **deployments/:** Stores contract deployment information (addresses, ABIs, etc.). +* **artifacts/:** (Typically generated) Contains compiled contract ABIs and bytecode. +* **node_modules/:** (Typically generated) Contains installed npm packages. + +Example: + + +my-project/ +├── contracts/ +│ ├── MyContract.sol +│ └── ERC20.sol +├── interfaces/ +│ └── IMyContract.sol +├── libraries/ +│ └── SafeMath.sol +├── test/ +│ ├── MyContract.test.js +│ └── ERC20.test.js +├── scripts/ +│ ├── deploy.js +│ └── utils.js +├── deployments/ +│ └── rinkeby/ +│ └── MyContract.json +├── artifacts/ +│ └── contracts/ +│ ├── MyContract.json +│ └── ERC20.json +├── node_modules/ +├── hardhat.config.js +├── package.json +└── README.md + + +### 1.2. File Naming Conventions + +* Use PascalCase for contract names (e.g., `MyContract.sol`). +* Use PascalCase starting with I for interface names (e.g., `IMyContract.sol`). +* Use PascalCase for library names (e.g., `SafeMath.sol`). +* Filenames should match the contract name defined within the file. + +### 1.3. Module Organization + +* Keep contracts small and focused on a single responsibility (Single Responsibility Principle). +* Use libraries for reusable code blocks and mathematical operations. +* Use interfaces to define the functionality of contracts without implementation. +* Group related contracts into modules or subdirectories. + +### 1.4. Component Architecture + +* **Modular Design:** Break down complex systems into smaller, manageable, and reusable components. +* **Layered Architecture:** Separate concerns into different layers (e.g., data access, business logic, presentation). +* **Dependency Injection:** Decouple components by injecting dependencies instead of hardcoding them. + +### 1.5. Code Splitting Strategies + +* Use libraries for common functionality to reduce contract size and gas costs. +* Consider using the delegatecall pattern for complex logic, but be aware of the security implications. +* Split large contracts into smaller, more manageable contracts using inheritance or composition. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +* **Ownable:** Restricts access to certain functions to the contract owner. +* **Pausable:** Allows pausing contract functionality in case of emergency. +* **Pull over Push:** Favors withdrawing funds over automatically sending them. +* **Proxy Pattern:** Allows upgrading contract logic without changing the contract address (e.g., using UUPS or transparent proxies). +* **Factory Pattern:** Used to create new contract instances. +* **State Machine:** Manages different states of a contract. +* **Circuit Breaker:** Prevents a function from being called if it has repeatedly failed. + +### 2.2. Recommended Approaches + +* Use SafeMath library for arithmetic operations to prevent overflows and underflows (or use Solidity >= 0.8.0, which includes built-in overflow/underflow protection). +* Use events to log important state changes. +* Use modifiers to enforce access control and preconditions. +* Follow the Checks-Effects-Interactions pattern to prevent reentrancy attacks. + +### 2.3. Anti-patterns and Code Smells + +* **Unchecked Arithmetic:** Not using SafeMath or built-in overflow/underflow protection. +* **Reentrancy:** Vulnerable to reentrancy attacks. +* **Timestamp Dependence:** Relying on block timestamps for critical logic. +* **Integer Overflow/Underflow:** Not handling potential overflows and underflows. +* **Gas Limit Issues:** Causing transactions to run out of gas. +* **Centralization:** Over-reliance on a single administrator or owner. +* **Magic Numbers:** Using hardcoded values without explanation. +* **Tight Coupling:** Components are highly dependent on each other, making it difficult to modify or reuse them. + +### 2.4. State Management Best Practices + +* Minimize state variables to reduce gas costs. +* Use appropriate data types for state variables. +* Initialize state variables correctly. +* Consider using immutables for constants. +* Use structs and mappings to organize related data. + +### 2.5. Error Handling Patterns + +* Use `require()` to validate inputs and preconditions. +* Use `revert()` to handle errors and return informative error messages. +* Use custom errors to provide more context about errors (Solidity >= 0.8.4). +* Avoid using `assert()` in production code (it consumes all remaining gas). +* Consider using try/catch blocks for external calls. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +* **Gas Optimization:** Aim to reduce gas consumption by using efficient data structures, minimizing storage writes, and optimizing control flow. +* **Storage Optimization:** Minimize storage usage by using appropriate data types and packing variables. +* **Function Optimization:** Optimize functions by reducing the number of operations and minimizing gas costs. +* **Loop Optimization:** Avoid unnecessary loops and optimize loop conditions. +* **Use Calldata:** Pass function arguments as calldata instead of memory when possible. +* **Short Circuiting:** Leverage short circuiting in conditional statements. + +### 3.2. Memory Management + +* Minimize memory usage by using calldata for function arguments when possible. +* Avoid creating large arrays in memory. +* Use memory efficiently in loops. +* Consider using assembly for memory-intensive operations. + +### 3.3. Rendering Optimization (Applicable for front-ends interacting with contracts) + +* This section refers to optimization of the front-end, typically written in Javascript, used to display data from the blockchain. Data retrieved from the blockchain should be displayed in an efficient manner. This is more applicable to traditional software engineering, and less applicable to Solidity itself. For the sake of completeness, it's added here. +* Use virtualization or pagination for large datasets. +* Optimize rendering logic to minimize DOM updates. +* Use caching to avoid unnecessary re-renders. + +### 3.4. Bundle Size Optimization (Applicable for front-ends interacting with contracts) +* This section refers to optimization of the front-end, typically written in Javascript, used to display data from the blockchain. The bundle size of the front end app should be as minimal as possible, to provide a performant user experience. This is more applicable to traditional software engineering, and less applicable to Solidity itself. For the sake of completeness, it's added here. +* Remove unused code and dependencies. +* Use code splitting to load only necessary code. +* Compress assets (images, JavaScript, CSS). + +### 3.5. Lazy Loading Strategies (Applicable for front-ends interacting with contracts) +* This section refers to optimization of the front-end, typically written in Javascript, used to display data from the blockchain. Lazy loading can be used to improve the performance of a front-end that interacts with a blockchain. This is more applicable to traditional software engineering, and less applicable to Solidity itself. For the sake of completeness, it's added here. +* Load images and other resources only when they are needed. +* Use lazy loading for off-screen components. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities and Prevention + +* **Reentrancy:** Prevent reentrancy attacks by following the Checks-Effects-Interactions pattern, using reentrancy guards, or using `transfer()`/`send()` to limit gas available for callbacks. +* **Integer Overflow/Underflow:** Use SafeMath or built-in overflow/underflow protection (Solidity >= 0.8.0). +* **Denial of Service (DoS):** Protect against DoS attacks by limiting gas consumption, avoiding unbounded loops, and using pull-over-push pattern. +* **Timestamp Dependence:** Avoid relying on block timestamps for critical logic, as they can be manipulated by miners. +* **Front Running:** Protect against front running by using commit-reveal schemes or off-chain order matching. +* **Gas Limit Issues:** Ensure that functions do not consume excessive gas, which can lead to transactions running out of gas. +* **Delegatecall:** Be extremely careful when using `delegatecall`, as the called contract can modify the caller's storage. +* **Uninitialized Storage Pointers:** Ensure all storage pointers are initialized before use. +* **Access Control:** Implement robust access control mechanisms to restrict access to sensitive functions and data. +* **Signature Replay:** Use nonces or other mechanisms to prevent signature replay attacks. + +### 4.2. Input Validation + +* Validate all inputs to ensure they are within expected ranges. +* Check for zero addresses and invalid values. +* Sanitize inputs to prevent injection attacks. +* Use appropriate data types for inputs. + +### 4.3. Authentication and Authorization Patterns + +* **Ownable:** Use the Ownable pattern to restrict access to certain functions to the contract owner. +* **Role-Based Access Control (RBAC):** Implement RBAC to assign different roles to users and grant them specific permissions. +* **Multi-Signature Wallets:** Use multi-signature wallets for critical operations that require multiple approvals. +* **Access Control Lists (ACLs):** Use ACLs to define fine-grained access control rules. + +### 4.4. Data Protection Strategies + +* Use encryption to protect sensitive data. +* Store sensitive data off-chain. +* Use access control to restrict access to sensitive data. +* Consider using zero-knowledge proofs for privacy. + +### 4.5. Secure API Communication + +* Use HTTPS for secure communication. +* Validate all data received from external sources. +* Implement rate limiting to prevent abuse. +* Use authentication and authorization to protect APIs. + +## 5. Testing Approaches + +### 5.1. Unit Testing Strategies + +* Write unit tests for all functions. +* Test all possible scenarios and edge cases. +* Use assertions to verify expected behavior. +* Use code coverage tools to ensure that all code is tested. +* Test state transitions and event emissions. + +### 5.2. Integration Testing + +* Test interactions between multiple contracts. +* Test integration with external systems. +* Test deployment and upgrade processes. + +### 5.3. End-to-End Testing + +* Test the entire system from end to end. +* Simulate real-world scenarios. +* Test user interactions and workflows. + +### 5.4. Test Organization + +* Organize tests into separate files or directories. +* Use descriptive test names. +* Group tests by functionality or module. + +### 5.5. Mocking and Stubbing + +* Use mocking to isolate contracts and simulate dependencies. +* Use stubbing to replace external calls with predefined responses. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +* **Unchecked Arithmetic:** Not using SafeMath or built-in overflow/underflow protection. +* **Reentrancy:** Not protecting against reentrancy attacks. +* **Timestamp Dependence:** Relying on block timestamps for critical logic. +* **Gas Limit Issues:** Causing transactions to run out of gas. +* **Ignoring Event Emissions:** Failing to emit events for important state changes, making off-chain monitoring and indexing difficult. + +### 6.2. Edge Cases + +* **Zero Values:** Handling cases where input values are zero. +* **Large Numbers:** Handling cases where input values are very large. +* **Empty Strings:** Handling cases where input strings are empty. +* **Extreme Conditions:** Testing the contract under extreme network conditions, such as high gas prices or network congestion. + +### 6.3. Version-Specific Issues + +* Be aware of breaking changes between Solidity versions. +* Test your contracts with the target Solidity version. +* Use the latest compiler version to leverage improvements and security patches. + +### 6.4. Compatibility Concerns + +* Ensure compatibility with different Ethereum clients and networks. +* Test your contracts on different testnets before deploying to mainnet. +* Consider compatibility with different wallets and browsers. + +### 6.5. Debugging Strategies + +* Use debugging tools like Remix, Hardhat, or Truffle. +* Use console.log statements for debugging. +* Use event logs to track state changes. +* Use revert reasons to understand why transactions failed. +* Use fuzzing to find unexpected behavior and vulnerabilities. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +* **Remix:** Online IDE for writing, compiling, and deploying Solidity contracts. +* **Hardhat:** Development environment for Ethereum smart contracts. +* **Truffle:** Development framework for Ethereum with testing and deployment capabilities. +* **Foundry:** Blazing-fast, portable and modular toolkit for Ethereum application development +* **VS Code:** Code editor with Solidity extensions for syntax highlighting, linting, and debugging. + +### 7.2. Build Configuration + +* Use a build tool like Hardhat or Truffle to manage the build process. +* Configure the compiler version and optimization settings. +* Generate contract ABIs and bytecode. + +### 7.3. Linting and Formatting + +* Use a linter like Solhint to enforce code style and best practices. +* Use a formatter like Prettier to format code consistently. +* Configure linting and formatting rules to match your project's standards. + +### 7.4. Deployment Best Practices + +* Deploy contracts to a testnet before deploying to mainnet. +* Verify contract code on Etherscan. +* Use a deployment script to automate the deployment process. +* Store contract deployment information (addresses, ABIs, etc.). + +### 7.5. CI/CD Integration + +* Integrate testing and deployment into your CI/CD pipeline. +* Run tests automatically on every commit. +* Automate the deployment process to reduce errors. + +By following these best practices, you can write more secure, efficient, and maintainable Solidity smart contracts. \ No newline at end of file diff --git a/.cursor/rules/solidjs.mdc b/.cursor/rules/solidjs.mdc new file mode 100644 index 000000000..d6ffb7673 --- /dev/null +++ b/.cursor/rules/solidjs.mdc @@ -0,0 +1,322 @@ +--- +description: This comprehensive guide outlines best practices for SolidJS development, covering code organization, performance, security, testing, and common pitfalls. It provides actionable guidelines for building maintainable, efficient, and secure SolidJS applications. +globs: *.js,*.jsx,*.ts,*.tsx +--- +# SolidJS Best Practices: A Comprehensive Guide + +This document provides a detailed overview of best practices for SolidJS development, covering various aspects from code organization to security and performance. It aims to guide developers in building robust, scalable, and maintainable SolidJS applications. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure Best Practices + +A well-defined directory structure is crucial for maintaining a large SolidJS application. Here's a recommended structure: + + +my-solid-app/ +├── src/ +│ ├── components/ +│ │ ├── Button/ +│ │ │ ├── Button.tsx +│ │ │ ├── Button.module.css +│ │ │ ├── Button.test.tsx +│ │ ├── Card/ +│ │ │ ├── Card.tsx +│ │ │ ├── Card.module.css +│ │ ├── index.ts # Exports all components +│ ├── contexts/ +│ │ ├── ThemeContext.tsx +│ ├── hooks/ +│ │ ├── useTheme.ts +│ ├── utils/ +│ │ ├── api.ts +│ │ ├── helpers.ts +│ ├── routes/ +│ │ ├── Home.tsx +│ │ ├── About.tsx +│ ├── styles/ +│ │ ├── global.css +│ ├── App.tsx +│ ├── index.tsx +│ ├── types.d.ts # Global type definitions +│ ├── vite-env.d.ts # vite type definitions +├── public/ +│ ├── favicon.svg +│ ├── assets/ +├── .env +├── vite.config.ts +├── tsconfig.json +├── package.json +├── README.md + + +**Explanation:** + +* **src/:** Contains the source code of your application. + * **components/:** Reusable UI components. Each component should have its own directory containing the component file, styles, and tests. + * **contexts/:** SolidJS Context providers for managing global state and shared data. + * **hooks/:** Custom hooks for reusable logic. + * **utils/:** Utility functions such as API calls, data formatting, etc. + * **routes/:** Components that define different routes in your application. + * **styles/:** Global styles or shared CSS modules. + * **App.tsx:** The root component of your application. + * **index.tsx:** Entry point for rendering the App component into the DOM. + * **types.d.ts:** Global typescript type definitions for the project + * **vite-env.d.ts:** vite environment type definitions +* **public/:** Static assets such as images, fonts, etc. +* **.env:** Stores environment variables. +* **vite.config.ts:** Vite configuration file. +* **tsconfig.json:** TypeScript configuration file. +* **package.json:** Node.js package file. +* **README.md:** Project documentation. + +### 1.2 File Naming Conventions + +* **Components:** Use PascalCase for component file names (e.g., `Button.tsx`, `UserProfile.jsx`). +* **Hooks:** Use camelCase prefixed with `use` for hook file names (e.g., `useTheme.ts`, `useFetchData.js`). +* **Utilities:** Use camelCase for utility file names (e.g., `api.ts`, `formatDate.js`). +* **Styles:** Use the `.module.css` extension for CSS modules (e.g., `Button.module.css`). If using other styling solutions like styled-components, follow their recommended practices. + +### 1.3 Module Organization + +* **Keep components modular:** Each component should have a single responsibility. +* **Export components:** Export components to make them reusable in other parts of the application (e.g., `export default Button;`). +* **Create an `index.ts` file:** In each directory (e.g., `components/`) export all the modules in that directory. This allows simpler imports from other components. e.g., `import {Button, Card} from '../components'` + +### 1.4 Component Architecture + +* **Functional Components:** Use functional components with Solid's reactivity system for managing state and UI updates. Avoid class components. +* **Props:** Use props to pass data from parent to child components. Be explicit about prop types using TypeScript. +* **Context API:** Use Solid's Context API for sharing data between components without prop drilling. Use cases: themes, user authentication, global configuration. +* **Signals:** Employ SolidJS Signals for managing component-level state. Prefer derived signals or memoization for expensive computations. + +### 1.5 Code Splitting Strategies + +* **Route-Based Splitting:** Use dynamic imports to load components only when a route is visited. Leverage Solid Router's lazy loading capabilities with `lazy`. +* **Component-Based Splitting:** Split large components into smaller chunks that can be loaded on demand. Use `Suspense` to handle loading states. +* **Library Splitting:** If using large libraries, consider splitting them into smaller chunks or using tree-shaking to remove unused code. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns Specific to SolidJS + +* **Renderless Components:** Create components that encapsulate logic without rendering any UI elements. They can be used as hooks or utility functions to share logic between components. +* **Compound Components:** Build components that work together implicitly, often using Context to share state. (e.g., Tabs, Accordions). +* **Reactivity-Based Composition:** Compose components based on shared reactive state, rather than inheritance. This allows more flexibility and better performance. +* **Custom Control Flow Components:** Create reusable components to handle conditional rendering or list iteration with specific optimization strategies. Examples include more performant alternatives to `` in certain specific use cases. + +### 2.2 Recommended Approaches for Common Tasks + +* **Fetching Data:** Use `createResource` for handling asynchronous data fetching. Use `AbortController` for cancelling requests on component unmount or data changes. +* **Managing Forms:** Use signals to track form input values. Implement validation logic and handle form submission. +* **Handling Events:** Attach event handlers to elements using JSX attributes (e.g., `onClick`). Use arrow functions or bind methods to ensure correct `this` context. +* **Global State Management:** Use Solid's Context API or a dedicated state management library (e.g., `solid-zustand`, `solid-js/store`) for sharing data between components. + +### 2.3 Anti-patterns and Code Smells to Avoid + +* **Reading and Writing to the Same Signal in an Effect:** This creates an infinite loop. Use the functional update form `setCount(count => count + 1)` or derived signals. +* **Writing to Signals in Effects for Derived Values:** Prefer derived signals or memoization for expensive computations (using `createMemo`). +* **Destructuring Props:** This breaks reactivity. Access props directly or use `mergeProps` and `splitProps`. +* **Using `.map` and Ternary Operators for Control Flow:** Use Solid's control flow components (``, ``, ``) to avoid unnecessary DOM elements. +* **Mutating Resource Data Directly:** Always create new objects when mutating resource data to prevent unnecessary re-renders. use `createMutable` judiciously only where mutability is needed. +* **Ignoring Cleanup Functions:** Neglecting `onCleanup` can lead to memory leaks and unexpected behavior when components unmount. + +### 2.4 State Management Best Practices + +* **Component-Level State:** Use signals for managing simple, component-specific state. +* **Context API:** For sharing data between components without prop drilling, especially themes or user authentication. +* **Global Stores:** Use store solutions or third-party libraries (e.g., `solid-zustand`, `solid-js/store`) for complex global state management. +* **Immutability:** Treat state as immutable. Always create new objects/arrays when updating state, especially with resources or stores. +* **Single Source of Truth:** Ensure that each piece of state has a single, authoritative source. Avoid duplicating state across components. + +### 2.5 Error Handling Patterns + +* **`Suspense` for Loading States:** Wrap asynchronous operations like `createResource` with `Suspense` to display loading indicators. +* **Error Boundaries:** Implement error boundaries to catch errors during rendering and display fallback UI. +* **Try-Catch Blocks:** Use `try-catch` blocks for handling synchronous errors or errors in event handlers. +* **Centralized Error Logging:** Implement a centralized error logging mechanism to track and monitor errors in your application. +* **User-Friendly Error Messages:** Display user-friendly error messages to guide users on how to resolve issues. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +* **Fine-Grained Reactivity:** Solid's fine-grained reactivity ensures that only the necessary parts of the UI are updated when state changes. Leverage this by breaking down components into smaller, more focused units. +* **Derived Signals and Memos:** Use derived signals and memos to avoid recomputing expensive values unnecessarily. +* **Control Flow Components:** Use ``, ``, and `` for efficient conditional rendering and list iteration. +* **Lazy Loading:** Load components and modules only when they are needed. +* **Virtualization:** For rendering large lists, use virtualization techniques to only render the visible items. +* **Debouncing and Throttling:** Implement debouncing or throttling for event handlers that trigger frequent updates. + +### 3.2 Memory Management + +* **Cleanup Functions:** Always use `onCleanup` to release resources when a component is unmounted, such as timers, event listeners, and subscriptions. +* **Avoid Memory Leaks:** Be mindful of potential memory leaks when using closures or retaining references to DOM elements. +* **Weak References:** Use `WeakRef` or `WeakMap` for holding references to objects without preventing them from being garbage collected. + +### 3.3 Rendering Optimization + +* **Minimize DOM Updates:** Solid's reactivity system minimizes DOM updates, but avoid unnecessary renders by optimizing component structure and state management. +* **Batch Updates:** Use `batchedUpdates` to batch multiple state updates into a single render cycle (use with caution). +* **Avoid Inline Styles:** Prefer CSS classes or CSS modules for styling components. +* **Use CSS Transitions and Animations:** Leverage CSS transitions and animations for smoother UI updates. + +### 3.4 Bundle Size Optimization + +* **Tree Shaking:** Use tree shaking to remove unused code from your bundle. Vite automatically performs tree shaking. +* **Code Splitting:** Split your application into smaller chunks that can be loaded on demand. +* **Minification and Compression:** Minify and compress your code to reduce bundle size. Vite handles minification by default. +* **Image Optimization:** Optimize images by compressing them and using appropriate formats (e.g., WebP). +* **Dependency Analysis:** Analyze your dependencies to identify and remove unnecessary libraries. + +### 3.5 Lazy Loading Strategies + +* **Route-Based Lazy Loading:** Load components only when a route is visited using dynamic imports and the `lazy` function from Solid Router. +* **Component-Based Lazy Loading:** Split large components into smaller chunks that can be loaded on demand. +* **Conditional Lazy Loading:** Load components based on specific conditions, such as user interactions or device capabilities. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and How to Prevent Them + +* **Cross-Site Scripting (XSS):** Prevent XSS attacks by sanitizing user inputs and escaping data when rendering it in the DOM. Avoid using `dangerouslySetInnerHTML`. +* **Cross-Site Request Forgery (CSRF):** Implement CSRF protection by using anti-CSRF tokens in forms and validating them on the server. +* **SQL Injection:** Prevent SQL injection attacks by using parameterized queries or ORMs when interacting with databases. +* **Authentication and Authorization:** Secure your application by implementing proper authentication and authorization mechanisms. +* **Denial-of-Service (DoS):** Protect against DoS attacks by implementing rate limiting, input validation, and resource quotas. + +### 4.2 Input Validation + +* **Server-Side Validation:** Always validate user inputs on the server to prevent malicious data from being stored in your database. +* **Client-Side Validation:** Provide client-side validation for a better user experience, but never rely on it as the sole means of protection. +* **Sanitize Inputs:** Sanitize user inputs to remove potentially harmful characters or code. +* **Use Validation Libraries:** Use validation libraries to simplify the input validation process. + +### 4.3 Authentication and Authorization Patterns + +* **JSON Web Tokens (JWT):** Use JWTs for managing user authentication and authorization. +* **OAuth 2.0:** Implement OAuth 2.0 for third-party authentication and authorization. +* **Role-Based Access Control (RBAC):** Implement RBAC to control access to resources based on user roles. +* **Multi-Factor Authentication (MFA):** Implement MFA for enhanced security. + +### 4.4 Data Protection Strategies + +* **Encryption:** Encrypt sensitive data at rest and in transit using appropriate encryption algorithms. +* **Data Masking:** Mask sensitive data when displaying it to users or storing it in logs. +* **Data Redaction:** Redact sensitive data from logs and audit trails. +* **Data Retention Policies:** Implement data retention policies to ensure that data is only stored for as long as it is needed. + +### 4.5 Secure API Communication + +* **HTTPS:** Use HTTPS for all API communication to encrypt data in transit. +* **API Keys:** Use API keys for authenticating clients and controlling access to your APIs. +* **Rate Limiting:** Implement rate limiting to prevent abuse of your APIs. +* **Input Validation:** Validate all API inputs to prevent malicious data from being processed. +* **Output Encoding:** Encode API outputs to prevent XSS attacks. + +## 5. Testing Approaches + +### 5.1 Unit Testing Strategies + +* **Test Individual Components:** Unit tests should focus on testing individual components in isolation. +* **Mock Dependencies:** Mock dependencies to isolate the component being tested. +* **Test All Scenarios:** Test all possible scenarios and edge cases. +* **Use Assertions:** Use assertions to verify that the component behaves as expected. + +### 5.2 Integration Testing + +* **Test Component Interactions:** Integration tests should focus on testing the interactions between components. +* **Mock External Services:** Mock external services to isolate the component being tested. +* **Test Data Flow:** Test the flow of data between components. + +### 5.3 End-to-End Testing + +* **Test the Entire Application:** End-to-end tests should focus on testing the entire application from the user's perspective. +* **Use a Testing Framework:** Use a testing framework such as Cypress or Playwright to automate the testing process. +* **Test User Flows:** Test common user flows to ensure that the application works as expected. + +### 5.4 Test Organization + +* **Colocate Tests:** Place test files alongside the components they are testing. +* **Use a Consistent Naming Convention:** Use a consistent naming convention for test files (e.g., `Button.test.tsx`). +* **Organize Tests by Feature:** Organize tests by feature or module. + +### 5.5 Mocking and Stubbing + +* **Use Mocking Libraries:** Use mocking libraries such as Jest or Vitest to create mocks and stubs. +* **Mock External Dependencies:** Mock external dependencies to isolate the component being tested. +* **Stub API Calls:** Stub API calls to control the data returned by the API. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes Developers Make + +* **Incorrectly Using Effects:** Misunderstanding the dependency tracking of `createEffect` can lead to unnecessary re-runs or infinite loops. +* **Forgetting `onCleanup`:** Not using `onCleanup` for resources and subscriptions causes memory leaks. +* **Mutating State Directly:** Directly mutating objects or arrays used as state leads to unexpected behavior and broken reactivity. +* **Over-Optimizing Prematurely:** Focusing on micro-optimizations before addressing fundamental architectural issues. +* **Not Understanding Solid's Reactivity:** Failing to grasp the nuances of Solid's reactive system can lead to inefficient code. + +### 6.2 Edge Cases to Be Aware Of + +* **SSR Hydration Issues:** When doing SSR, ensure that the initial state on the server matches the client to avoid hydration errors. +* **Reactivity with Non-Reactive Values:** Passing plain JS objects or arrays to components without making them reactive can cause issues with updates. +* **Transitions and Resource Updates:** Interactions between transitions and resource updates can sometimes be tricky, especially around loading states. +* **Nested Reactivity:** Managing reactivity in complex, deeply nested data structures can be challenging. + +### 6.3 Version-Specific Issues + +* **Compatibility:** Consider backwards compatibility when upgrading SolidJS or related libraries. +* **Deprecations:** Be aware of deprecated features and APIs and migrate to the recommended alternatives. +* **Bug Fixes:** Stay up-to-date with bug fixes and security patches in newer versions of SolidJS. + +### 6.4 Compatibility Concerns + +* **Browser Compatibility:** Test your application in different browsers to ensure compatibility. +* **Device Compatibility:** Test your application on different devices, such as desktops, tablets, and mobile phones. +* **Accessibility:** Ensure that your application is accessible to users with disabilities. + +### 6.5 Debugging Strategies + +* **Use Browser Developer Tools:** Use the browser's developer tools to inspect the DOM, network requests, and console output. +* **SolidJS Devtools:** Use the SolidJS Devtools extension to inspect components, signals, and resources. +* **Logging:** Add logging statements to your code to track the flow of execution and the values of variables. +* **Debugging Tools:** Use debugging tools such as VS Code's debugger to step through your code and inspect variables. +* **Reproducible Examples:** Create reproducible examples to isolate and debug issues more effectively. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +* **Vite:** Use Vite as the build tool. SolidJS uses Vite as its default. +* **VS Code:** Use VS Code as the code editor, along with extensions for TypeScript, ESLint, and Prettier. +* **SolidJS Devtools:** Use the SolidJS Devtools browser extension to inspect components, signals, and resources. +* **TypeScript:** Utilize TypeScript for type safety and improved code maintainability. + +### 7.2 Build Configuration + +* **Vite Configuration:** Configure Vite to optimize your build for production (e.g., code splitting, minification, compression). +* **TypeScript Configuration:** Configure TypeScript to enforce strict type checking and code quality. +* **Environment Variables:** Use environment variables to store sensitive data and configure your application for different environments. + +### 7.3 Linting and Formatting + +* **ESLint:** Use ESLint to enforce code style and identify potential errors. +* **Prettier:** Use Prettier to automatically format your code. +* **Husky:** Use Husky to run linters and formatters before committing code. + +### 7.4 Deployment Best Practices + +* **Choose a Hosting Platform:** Select a suitable hosting platform for your SolidJS application (e.g., Netlify, Vercel, AWS). +* **Configure a Production Environment:** Configure your hosting platform to use a production environment with appropriate settings. +* **Automate Deployment:** Automate the deployment process using CI/CD tools. +* **Monitor Your Application:** Monitor your application for errors and performance issues. + +### 7.5 CI/CD Integration + +* **Choose a CI/CD Tool:** Select a CI/CD tool such as GitHub Actions, GitLab CI, or CircleCI. +* **Configure CI/CD Pipelines:** Configure CI/CD pipelines to automatically build, test, and deploy your application. +* **Automate Testing:** Automate unit tests, integration tests, and end-to-end tests in your CI/CD pipelines. +* **Automate Deployment:** Automate the deployment process to deploy new versions of your application with minimal manual intervention. + +This document provides a comprehensive set of best practices for SolidJS development. By following these guidelines, developers can build high-quality, maintainable, and secure SolidJS applications. \ No newline at end of file diff --git a/.cursor/rules/spacy.mdc b/.cursor/rules/spacy.mdc new file mode 100644 index 000000000..546959de6 --- /dev/null +++ b/.cursor/rules/spacy.mdc @@ -0,0 +1,317 @@ +--- +description: This rule file provides comprehensive best practices and coding standards for developing projects using spaCy, covering code organization, performance, security, testing, and more. It aims to guide developers in building maintainable, efficient, and secure NLP applications with spaCy. +globs: *.py +--- +# spaCy Best Practices and Coding Standards + +This document outlines the recommended best practices and coding standards for developing projects using spaCy. Adhering to these guidelines will result in more maintainable, efficient, and secure NLP applications. + +## Library Information: +- Name: spaCy +- Tags: python, nlp, text-processing + +## 1. Code Organization and Structure + +### 1.1 Directory Structure Best Practices + +Organizing your spaCy project with a clear directory structure enhances maintainability and collaboration. Here’s a recommended structure: + + +project_root/ +├── data/ # Contains raw and processed datasets +│ ├── raw/ # Raw data files (e.g., .txt, .csv) +│ ├── processed/ # Processed data (e.g., spaCy Doc objects) +│ └── README.md # Description of the data and preprocessing steps +├── models/ # Trained spaCy models +│ ├── en_core_web_sm/ # Example model directory +│ └── README.md # Information about the trained models +├── scripts/ # Python scripts for training, evaluation, and data processing +│ ├── train.py # Script for training the spaCy model +│ ├── evaluate.py # Script for evaluating the model +│ ├── preprocess.py # Script for preprocessing the data +│ └── utils.py # Utility functions +├── components/ # Custom spaCy components +│ ├── custom_ner.py # Custom NER component +│ └── custom_tokenizer.py # Custom tokenizer +├── tests/ # Unit and integration tests +│ ├── unit/ # Unit tests +│ ├── integration/ # Integration tests +│ └── conftest.py # Pytest configuration file +├── notebooks/ # Jupyter notebooks for exploration and prototyping +│ └── data_exploration.ipynb # Example notebook +├── requirements.txt # Project dependencies +├── pyproject.toml # Project configuration and build dependencies (recommended over setup.py) +├── README.md # Project overview and instructions +├── .gitignore # Specifies intentionally untracked files that Git should ignore +└── .cursor/ # Cursor IDE configuration + + +### 1.2 File Naming Conventions + +Follow these conventions for consistency and readability: + +- **Python Files:** `lowercase_with_underscores.py` +- **Data Files:** `lowercase-with-hyphens.csv` +- **Model Files:** `descriptive_name.spacy` +- **Configuration Files:** `config.cfg` or `config.yaml` + +### 1.3 Module Organization + +Organize your code into modules based on functionality: + +- **Data Processing:** Modules for loading, cleaning, and transforming data. +- **Model Training:** Modules for training spaCy models. +- **Custom Components:** Modules for custom spaCy components (e.g., custom NER, custom tokenizer). +- **Evaluation:** Modules for evaluating model performance. +- **Utilities:** Modules for helper functions and shared logic. + +### 1.4 Component Architecture Recommendations + +When building custom spaCy components, follow these guidelines: + +- **Encapsulation:** Each component should have a well-defined purpose and encapsulate its logic. +- **Modularity:** Components should be designed to be reusable in different parts of your project. +- **Configuration:** Make components configurable through parameters passed during initialization. +- **Testing:** Write unit tests for each component to ensure its correctness. + +Example of a custom component structure: + +python +# components/custom_ner.py +import spacy +from spacy.language import Language +from spacy.tokens import Doc, Span +from spacy.util import filter_spans + +@Language.factory( + "custom_ner", + default_config={ + "label": "CUSTOM", + "patterns": [] + }, +) +def create_custom_ner( + nlp: Language, + name: str, + label: str, + patterns: list +): + return CustomNer(nlp, name, label, patterns) + + +class CustomNer: + def __init__(self, nlp: Language, name: str, label: str, patterns: list): + self.label = label + self.patterns = patterns + self.ruler = nlp.add_pipe("entity_ruler") + self.ruler.add_patterns([{"label": self.label, "pattern": p} for p in self.patterns]) + + + def __call__(self, doc: Doc) -> Doc: + #Add the custom patterns to the entity recognizer + doc.ents = filter_spans(doc.ents) + return doc + + +### 1.5 Code Splitting Strategies + +For large projects, split your code into smaller, manageable files: + +- **Functional Decomposition:** Split code based on functionality (e.g., data loading, preprocessing, model training). +- **Component-Based Splitting:** Each custom spaCy component should reside in its own file. +- **Layered Architecture:** If applicable, separate the presentation, business logic, and data access layers. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns + +- **Pipeline Pattern:** Leverage spaCy's pipeline architecture for sequential processing of text. +- **Factory Pattern:** Use factory functions to create spaCy components with specific configurations. +- **Strategy Pattern:** Implement different strategies for text processing (e.g., different tokenization methods) and switch between them based on the input. + +### 2.2 Recommended Approaches for Common Tasks + +- **Text Preprocessing:** Use spaCy's built-in tokenization, lemmatization, and stop word removal for efficient text preprocessing. +- **Named Entity Recognition:** Utilize spaCy's NER capabilities or train custom NER models for specific entity types. +- **Dependency Parsing:** Leverage spaCy's dependency parser to extract relationships between words in a sentence. +- **Similarity Analysis:** Use spaCy's word vectors to compute similarity between documents or phrases. + +### 2.3 Anti-patterns and Code Smells + +- **Ignoring Errors:** Always handle exceptions and log errors appropriately. +- **Global Variables:** Avoid using global variables; use dependency injection or configuration files instead. +- **Hardcoding Paths:** Use relative paths or environment variables for file paths. +- **Overly Complex Functions:** Keep functions short and focused on a single task. +- **Lack of Documentation:** Document your code with docstrings and comments. + +### 2.4 State Management Best Practices + +- **Stateless Components:** Design spaCy components to be stateless whenever possible to avoid unexpected side effects. +- **Configuration Objects:** Use configuration objects to manage component state. +- **Immutability:** Prefer immutable data structures to prevent unintended modifications. + +### 2.5 Error Handling Patterns + +- **Try-Except Blocks:** Use try-except blocks to handle potential exceptions during text processing. +- **Logging:** Log errors and warnings to a file for debugging and monitoring. +- **Custom Exceptions:** Define custom exceptions for specific error conditions. +- **Graceful Degradation:** Implement fallback mechanisms to handle errors gracefully. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +- **Model Selection:** Use smaller spaCy models (e.g., `en_core_web_sm`) for faster processing if accuracy is not critical. +- **Batch Processing:** Process large texts in batches to improve throughput. +- **GPU Acceleration:** Utilize GPU acceleration for faster processing of large datasets (requires `cupy`). +- **Disable Unnecessary Components:** Disable pipeline components that are not needed for a specific task. +- **Cython Optimization:** Write performance-critical code in Cython for faster execution. +- **Vector Data:** Utilize vector data whenever possible for performance as it can be GPU accelerated. + +### 3.2 Memory Management + +- **Object Reuse:** Reuse spaCy Doc objects to reduce memory allocation overhead. +- **Data Streaming:** Stream large data files instead of loading them into memory at once. +- **Garbage Collection:** Explicitly trigger garbage collection when memory usage is high (use with caution). + +### 3.3 Bundle Size Optimization (If applicable) + +- **Tree shaking** If deploying spaCy models to the web or other bundle-size-sensitive environments, use tree shaking to remove unused code from the spaCy library. + +### 3.4 Lazy Loading Strategies + +- **Model Loading on Demand:** Load spaCy models only when they are needed, rather than at application startup. +- **Component Initialization on Demand:** Initialize custom spaCy components only when they are used. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities + +- **Injection Attacks:** Prevent injection attacks by sanitizing user inputs before passing them to spaCy. +- **Denial of Service:** Limit the size of input texts to prevent denial-of-service attacks. +- **Model Poisoning:** Protect against model poisoning by verifying the integrity of trained spaCy models. + +### 4.2 Input Validation + +- **Text Length Limits:** Limit the length of input texts to prevent excessive processing time and memory usage. +- **Character Encoding:** Validate that input texts are encoded in a supported character encoding (e.g., UTF-8). +- **Whitelist Validation:** Use whitelist validation to allow only specific characters or patterns in input texts. + +### 4.3 Authentication and Authorization (If applicable) + +- **API Keys:** Use API keys to authenticate clients accessing spaCy services. +- **Role-Based Access Control:** Implement role-based access control to restrict access to sensitive spaCy features. + +### 4.4 Data Protection + +- **Data Encryption:** Encrypt sensitive data at rest and in transit. +- **Data Masking:** Mask or redact sensitive data in log files and debugging output. Use the retokenizer features to create [REDACTED] names or private identifiable information +- **Anonymization:** Anonymize data before storing it for analysis or research purposes. + +### 4.5 Secure API Communication (If applicable) + +- **HTTPS:** Use HTTPS for all API communication to encrypt data in transit. +- **Input Sanitization:** Sanitize any data received from the user to prevent injections of malicious code. +- **Rate limiting** Implement rate limiting to mitigate denial-of-service (DoS) attacks by restricting the number of requests a user can make within a specific timeframe. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +- **Component Testing:** Write unit tests for each custom spaCy component to ensure its correctness. +- **Function Testing:** Test individual functions in your code to verify their behavior. +- **Mocking:** Use mocking to isolate components and functions during testing. + +### 5.2 Integration Testing + +- **Pipeline Testing:** Test the integration of multiple spaCy components in a pipeline. +- **Data Flow Testing:** Test the flow of data through your application to ensure that it is processed correctly. +- **API Testing:** Test the integration of your spaCy application with external APIs. + +### 5.3 End-to-End Testing + +- **User Interface Testing (if applicable):** Test the user interface of your spaCy application to ensure that it is user-friendly and functions as expected. +- **System Testing:** Test the entire system to ensure that all components work together correctly. + +### 5.4 Test Organization + +- **Separate Test Directory:** Keep tests in a separate `tests/` directory. +- **Test Modules:** Organize tests into modules based on the functionality being tested. +- **Test Fixtures:** Use test fixtures to set up test data and dependencies. +- **Coverage Analysis:** Use coverage analysis to identify untested code. + +### 5.5 Mocking and Stubbing Techniques + +- **Mocking External Dependencies:** Mock external dependencies (e.g., APIs, databases) to isolate your code during testing. +- **Stubbing Function Calls:** Stub function calls to control the behavior of dependencies. +- **Monkey Patching:** Use monkey patching to replace functions or objects during testing (use with caution). + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +- **Not Handling Exceptions:** Failing to handle exceptions properly can lead to unexpected application crashes. +- **Ignoring Performance:** Not optimizing your code for performance can result in slow processing times. +- **Lack of Testing:** Insufficient testing can lead to bugs and regressions. +- **Incorrect model versions:** Using the wrong model version for your spaCy version may cause incompatibility errors. + +### 6.2 Edge Cases + +- **Unicode Characters:** Handle Unicode characters correctly to avoid encoding issues. +- **Rare Words:** Be aware of rare words that may not be in spaCy's vocabulary. +- **Out-of-Vocabulary Words:** Handle out-of-vocabulary words gracefully. + +### 6.3 Version-Specific Issues + +- **API Changes:** Be aware of API changes in spaCy versions and update your code accordingly. +- **Model Compatibility:** Ensure that trained spaCy models are compatible with the spaCy version you are using. + +### 6.4 Compatibility Concerns + +- **Dependency Conflicts:** Be aware of dependency conflicts between spaCy and other libraries. +- **Operating System Compatibility:** Ensure that your spaCy application is compatible with the target operating systems. + +### 6.5 Debugging Strategies + +- **Logging:** Use logging to track the execution flow of your code and identify errors. +- **Debugging Tools:** Use a debugger to step through your code and inspect variables. +- **Print Statements:** Use print statements to output debugging information. +- **Profiling:** Use a profiler to identify performance bottlenecks in your code. +- **Error Messages:** Read and understand error messages to diagnose problems. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +- **IDE:** Visual Studio Code, PyCharm, or other Python IDE. +- **Virtual Environment:** Use virtual environments (e.g., `venv`, `conda`) to isolate project dependencies. +- **Package Manager:** Use `pip` or `conda` to manage project dependencies. + +### 7.2 Build Configuration + +- **`pyproject.toml`:** Use `pyproject.toml` for build configuration and dependency management (recommended). +- **`setup.py`:** Use `setup.py` for older projects or when `pyproject.toml` is not supported. +- **`requirements.txt`:** Use `requirements.txt` to specify project dependencies. + +### 7.3 Linting and Formatting + +- **`flake8`:** Use `flake8` for linting Python code. +- **`pylint`:** Use `pylint` for static code analysis. +- **`black`:** Use `black` for automatic code formatting. +- **`isort`:** Use `isort` for sorting imports. + +### 7.4 Deployment + +- **Docker:** Use Docker to containerize your spaCy application. +- **Cloud Platforms:** Deploy your spaCy application to cloud platforms such as AWS, Google Cloud, or Azure. +- **Serverless Functions:** Deploy your spaCy application as serverless functions (e.g., AWS Lambda, Google Cloud Functions). + +### 7.5 CI/CD Integration + +- **GitHub Actions:** Use GitHub Actions for continuous integration and continuous deployment. +- **Jenkins:** Use Jenkins for continuous integration. +- **Travis CI:** Use Travis CI for continuous integration. +- **Automated Testing:** Automate unit, integration, and end-to-end tests as part of your CI/CD pipeline. +- **Automated Deployment:** Automate the deployment of your spaCy application to staging and production environments. + +By following these best practices, you can build robust, scalable, and maintainable NLP applications with spaCy. \ No newline at end of file diff --git a/.cursor/rules/sphinx.mdc b/.cursor/rules/sphinx.mdc new file mode 100644 index 000000000..ea27c189c --- /dev/null +++ b/.cursor/rules/sphinx.mdc @@ -0,0 +1,170 @@ +--- +description: This rule file provides comprehensive guidelines for writing high-quality Sphinx documentation, covering code style, structure, performance, and best practices. It aims to ensure consistency, readability, and maintainability of Sphinx-based projects. +globs: *.rst +--- +- **Use reStructuredText (reST) or MyST markdown:** These are the primary markup languages for Sphinx. reST is the default and provides powerful semantic markup capabilities. MyST is a markdown flavor compatible with Sphinx. Choose one and stick to it for consistency. + +- **Follow PEP 8 and PEP 257:** Adhere to PEP 8 for Python code style within your documentation, emphasizing readability and consistency (e.g., 4 spaces for indentation, line length limits). Follow PEP 257 for docstring conventions for Python code. + +- **Structure Your Documentation Hierarchically:** Organize documentation into a clear, hierarchical structure with logical sections. Use proper headings (H1, H2, etc.) to delineate content. + +- **Use Semantic Markup:** Leverage reST's semantic markup for functions, classes, modules, and other code elements to enable cross-referencing and automatic linking. Example: `:func:`my_function``. + +- **Cross-Referencing:** Utilize Sphinx's cross-referencing capabilities extensively to link related documentation sections, functions, classes, and other elements. This improves navigation and understanding. Use `:ref:`, `:func:`, `:class:`, `:mod:` roles. + +- **Document Code:** Use Sphinx's `autodoc` extension to automatically generate documentation from Python docstrings. Ensure your docstrings are comprehensive, following PEP 257 conventions. Include parameters, return types, and exceptions. + +- **Use Code Blocks:** Clearly define code blocks using the `.. code-block:: language` directive to enable syntax highlighting and improve readability. + +- **Whitespace and Formatting:** Maintain consistent whitespace for clarity. Use blank lines to separate sections and ensure code blocks are clearly defined. + +- **Maintain Line Length:** Limit line lengths to 79 characters for code and 72 characters for docstrings/comments to improve readability and compatibility with various tools and editors. + +- **Version Control:** Use version control (e.g., Git) to manage your Sphinx documentation project. This allows for tracking changes, collaboration, and easy rollback to previous versions. + +- **Use a Sphinx Theme:** Choose a Sphinx theme that suits your project's style and audience. Many built-in themes (alabaster, classic, sphinxdoc) and third-party themes (sphinx_rtd_theme, pydata_sphinx_theme) are available. + +- **Configure Sphinx Properly:** Carefully configure Sphinx using the `conf.py` file to customize the output format, extensions, theme, and other settings. Use meaningful project metadata (name, version, author). + +- **Test Your Documentation:** Use Sphinx's `doctest` extension to automatically test code snippets within your documentation. This ensures the code examples are correct and up-to-date. + +- **Use Extensions:** Explore and utilize Sphinx extensions to enhance your documentation with features like mathematical equations, diagrams, and other specialized content. + +- **Internationalization:** Consider internationalizing your documentation using Sphinx's internationalization features if your project targets a global audience. + +- **Linkcheck Builder:** Use the linkcheck builder to automatically check for broken links in your documentation. This ensures the documentation remains up-to-date and accurate. + +- **Contribute Extensions**: Create your own sphinx extensions to manage repeated formatting or add complex rendering. + +- **Naming Conventions** + - Use descriptive and consistent names for files, directories, and variables. + - Follow Python's naming conventions for modules, classes, functions, and variables. + - Use all lowercase for modules. Use underscores if it improves readability. Example: `my_module.py` + - Use CapWords for class names. Example: `MyClass` + - Use lowercase with underscores for functions and variables. Example: `my_function`, `my_variable` + - Use UPPERCASE with underscores for constants. Example: `MAX_VALUE` + +- **Common Patterns and Anti-Patterns** + - **Pattern:** Use the Model-View-Controller (MVC) pattern for complex documentation structures, where the Model represents the data, the View renders the data, and the Controller handles user interactions. + - **Anti-Pattern:** Avoid deeply nested directory structures, which can make it difficult to navigate and maintain the documentation. + - **Anti-Pattern:** Avoid inconsistent formatting and style throughout the documentation, which can make it look unprofessional and difficult to read. + +- **Performance Considerations** + - Optimize images by compressing them without sacrificing quality. + - Use lazy loading for large images or other resources that are not immediately visible on the page. + - Use caching to reduce the number of times that resources need to be loaded. + - Minify CSS and JavaScript files to reduce their size. + +- **Security Best Practices** + - Sanitize user inputs to prevent cross-site scripting (XSS) attacks. + - Use HTTPS to encrypt communication between the user's browser and the server. + - Keep Sphinx and its extensions up-to-date to patch security vulnerabilities. + +- **Testing Approaches** + - Write unit tests for custom Sphinx extensions. + - Use integration tests to verify that the different parts of the documentation work together correctly. + - Use end-to-end tests to simulate user interactions with the documentation and verify that the documentation behaves as expected. + +- **Common Pitfalls and Gotchas** + - Be aware of the different versions of Sphinx and their compatibility with different Python versions and extensions. + - Use the appropriate encoding for your documentation files (UTF-8 is recommended). + - Be careful when using the `raw` directive, as it can introduce security vulnerabilities if not used properly. + +- **Tooling and Environment** + - Use a code editor or IDE with syntax highlighting and linting for reStructuredText or Markdown. + - Use a build automation tool (e.g., Make, tox) to automate the documentation build process. + - Use a CI/CD system (e.g., Jenkins, Travis CI, GitHub Actions) to automatically build and deploy the documentation whenever the code changes. + +- **Deployment Best Practices** + - Use a dedicated server or hosting platform for your documentation. + - Use a content delivery network (CDN) to distribute your documentation to users around the world. + - Use HTTPS to secure your documentation. + - Use a robots.txt file to prevent search engines from indexing sensitive parts of your documentation. + +- **Documenting Objects** + - Use `autodoc` to automatically generate documentation for Python objects. + - Use directives like `.. automodule::`, `.. autoclass::`, and `.. autofunction::` to document modules, classes, and functions, respectively. + - Use the `:members:`, `:undoc-members:`, and `:show-inheritance:` options to control which members are documented. + - Write clear and concise docstrings for all objects, following PEP 257 conventions. + +- **Intersphinx** + - Use the `intersphinx` extension to link to documentation of other Sphinx projects. + - Configure `intersphinx_mapping` in `conf.py` to specify the locations of other Sphinx documentation. + - Use the `:py:mod:`, `:py:class:`, and `:py:func:` roles to link to modules, classes, and functions in other projects. + +- **reStructuredText Specific Tips** + - Use titles and sections to structure your documentation. + - Use bullet lists and numbered lists to present information in a clear and concise way. + - Use tables to organize data. + - Use images to illustrate concepts. + - Use footnotes and citations to provide additional information and give credit to sources. + +- **Example `conf.py` settings:** + python + # Configuration file for the Sphinx documentation builder. + # For the full list of built-in configuration values, see the documentation: + # https://www.sphinx-doc.org/en/master/usage/configuration.html + + # -- Project information ----------------------------------------------------- + + project = 'My Project' + copyright = '2023, My Company' + author = 'John Doe' + release = '1.0.0' + + # -- General configuration --------------------------------------------------- + + extensions = [ + 'sphinx.ext.autodoc', + 'sphinx.ext.napoleon', + 'sphinx.ext.intersphinx', + 'sphinx.ext.todo', + 'sphinx.ext.viewcode', + 'sphinx.ext.mathjax', + 'sphinx.ext.ifconfig', + 'sphinx.ext.githubpages', + 'sphinx.ext.graphviz' + ] + + templates_path = ['_templates'] + exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] + + language = 'en' + + # -- Options for HTML output ------------------------------------------------- + + html_theme = 'sphinx_rtd_theme' + html_static_path = ['_static'] + + # -- Extension configuration ------------------------------------------------- + + # napoleon configuration (for Google-style docstrings) + napoleon_google_docstring = True + napoleon_include_init_with_doc = True + napoleon_include_private_with_doc = False + napoleon_include_special_with_doc = True + napoleon_use_rtype = True + napoleon_use_param = True + + # intersphinx configuration + intersphinx_mapping = { + 'python': ('https://docs.python.org/3', None), + 'numpy': ('https://numpy.org/doc/stable/', None), + } + + # todo configuration + todo_include_todos = True + + # graphviz configuration + graphviz_output_format = 'svg' + + + # -- Options for Markdown files -------------------------------------------- + + from recommonmark.parser import CommonMarkParser + +source_parsers = { + '.md': CommonMarkParser, +} + + source_suffix = ['.rst', '.md'] \ No newline at end of file diff --git a/.cursor/rules/spring.mdc b/.cursor/rules/spring.mdc new file mode 100644 index 000000000..aa171d7e0 --- /dev/null +++ b/.cursor/rules/spring.mdc @@ -0,0 +1,208 @@ +--- +description: This rule set provides comprehensive best practices and coding standards for developing robust and maintainable Java backend applications using the Spring Boot framework. It focuses on code structure, performance, security, and testing. +globs: *.java +--- +- **Code Organization and Structure** + - **Directory Structure Best Practices** + - **Layered Architecture:** Organize code into layers: `controller`, `service`, `repository`, `model`, `configuration`, and `exception`. This provides clear separation of concerns. Example: + + src/ + └── main/ + └── java/ + └── com/example/app/ + ├── controller/ + │ └── OrderController.java + ├── service/ + │ └── OrderService.java + ├── repository/ + │ └── OrderRepository.java + ├── model/ + │ └── Order.java + ├── configuration/ + │ └── AppConfig.java + └── exception/ + └── OrderNotFoundException.java + + - **Feature-Based Organization:** Group code by feature rather than layer. This keeps related files together for easier navigation. Use when features are more important than technical separation. + + src/ + └── main/ + └── java/ + └── com/example/app/ + ├── order/ + │ ├── OrderController.java + │ ├── OrderService.java + │ ├── OrderRepository.java + │ └── Order.java + └── customer/ + ├── CustomerController.java + ├── CustomerService.java + ├── CustomerRepository.java + └── Customer.java + + - **File Naming Conventions** + - **Classes:** Use `PascalCase`. Names should be nouns representing the purpose of the class. Example: `OrderService`, `ProductController`. + - **Interfaces:** Use `PascalCase`. Often named with an `-able` suffix (e.g., `Runnable`, `Serializable`) or prefixed with `I` (e.g., `IUserService`). Example: `OrderRepository`, `UserService`. + - **Methods:** Use `camelCase`. Names should be verbs indicating the action performed. Example: `createOrder()`, `calculateTotalPrice()`. + - **Variables:** Use `camelCase`. Descriptive and concise names providing context. Avoid abbreviations unless widely understood. Example: `orderTotal`, `userList`. + - **Constants:** Use `UPPER_SNAKE_CASE`. Example: `MAX_RETRIES`, `DEFAULT_TIMEOUT`. + - **Module Organization** + - **Modular Monolith:** Break down the application into logical modules, each with a specific responsibility. This promotes maintainability and testability without the complexity of microservices. Maven or Gradle can be used to manage modules. + - **Microservices:** When appropriate, consider breaking the application into independent microservices communicating over APIs. This provides scalability and independent deployment. Example: `Order Service`, `Payment Service`, `Customer Service`. + - **Component Architecture** + - **Controllers:** Handle HTTP requests and delegate business logic to services. Keep controllers thin and focused on request mapping and response formatting. + - **Services:** Contain the core business logic of the application. Services should be stateless and independent of HTTP concerns. + - **Repositories:** Handle data access and persistence. Use Spring Data JPA repositories to simplify database interactions. + - **Data Transfer Objects (DTOs):** Use DTOs for request and response payloads to separate API contracts from domain models. This allows for flexibility in evolving the API without affecting internal data structures. + - **Code Splitting Strategies** + - **By Feature:** Split code into modules based on features, improving maintainability and scalability. + - **By Layer:** Split code into distinct layers (presentation, business, data) to separate concerns and improve testability. + +- **Common Patterns and Anti-Patterns** + - **Design Patterns** + - **Dependency Injection:** Core to Spring. Use constructor injection for required dependencies and setter injection for optional dependencies. Avoid field injection. + - **Inversion of Control (IoC):** Let the Spring container manage the creation and lifecycle of beans. + - **Aspect-Oriented Programming (AOP):** Use AOP for cross-cutting concerns like logging, security, and transaction management. Use `@Aspect` annotation. + - **Template Method:** Apply when different implementations of an algorithm are needed. Spring's `JdbcTemplate` is a classical example. + - **Factory Pattern:** Can be used for abstracting object creation. + - **Singleton Pattern:** Spring beans are singletons by default, but be mindful when dealing with stateful beans. + - **Recommended Approaches** + - **Configuration Properties:** Externalize configuration using `@ConfigurationProperties` for type-safe access to properties. + - **Event Handling:** Use Spring's event publishing and listening mechanism for decoupling components. Use `@EventListener` annotation. + - **Transaction Management:** Use `@Transactional` annotation for declarative transaction management. + - **Validation:** Use `@Validated` and JSR-303 annotations for request parameter and body validation. + - **Anti-Patterns and Code Smells** + - **God Classes:** Avoid classes with too many responsibilities. Apply the Single Responsibility Principle. + - **Long Methods:** Break down long methods into smaller, more focused methods. + - **Primitive Obsession:** Avoid using primitive types excessively. Encapsulate related data into value objects. + - **Magic Numbers/Strings:** Replace hardcoded values with constants or configuration properties. + - **Tight Coupling:** Minimize dependencies between components through dependency injection and interfaces. + - **State Management** + - **Stateless Services:** Services should be stateless to improve scalability and concurrency. + - **Session Management:** Use Spring Session for managing user sessions in a distributed environment. Consider using Redis or other external storage for session data. + - **Caching:** Implement caching strategically to reduce database load and improve response times. Use Spring Cache abstraction with appropriate cache providers. + - **Error Handling Patterns** + - **Global Exception Handling:** Use `@ControllerAdvice` and `@ExceptionHandler` to handle exceptions globally and provide consistent error responses. + - **Custom Exceptions:** Create custom exception classes to represent application-specific errors. + - **Logging:** Log exceptions with appropriate levels (e.g., `error`, `warn`, `info`). + - **Return Meaningful Error Responses:** Return well-structured error responses with clear error codes and messages. Use a consistent format for error responses. + +- **Performance Considerations** + - **Optimization Techniques** + - **Database Connection Pooling:** Use a connection pool (e.g., HikariCP) to reuse database connections and reduce overhead. + - **Caching:** Implement caching using Spring Cache abstraction to store frequently accessed data. + - **Asynchronous Processing:** Use `@Async` for long-running tasks that don't need to be executed synchronously. + - **Lazy Loading:** Use lazy loading for entities and relationships in JPA to avoid loading unnecessary data. + - **Profiling:** Use profiling tools (e.g., VisualVM, JProfiler) to identify performance bottlenecks. + - **Memory Management** + - **Garbage Collection Tuning:** Tune JVM garbage collection settings for optimal performance. Monitor GC performance and adjust parameters as needed. + - **Object Pooling:** Consider using object pooling for frequently created and destroyed objects. + - **Avoid Memory Leaks:** Be careful to release resources properly to prevent memory leaks. Use try-with-resources for closing streams and other resources. + - **Rendering Optimization** + - *(Not Directly Applicable to Spring Backend - Focus on API Response times and efficiency)* + - **Minimize Data Transfer:** Only return the data needed by the client. + - **Use Efficient Data Structures:** Use appropriate data structures for API responses (e.g., lists, maps). + - **Compression:** Enable GZIP compression for API responses to reduce the size of the data transferred over the network. + - **Bundle Size Optimization** + - *(Not Directly Applicable to Spring Backend - Although Dependency Management is crucial)* + - **Minimize Dependencies:** Only include necessary dependencies. + - **Use Code Analysis Tools:** Use tools like SonarQube to identify unused code and dependencies. + - **Lazy Loading** + - **JPA Lazy Loading:** Use lazy loading for JPA relationships to avoid loading unnecessary data. Use the `@OneToMany(fetch = FetchType.LAZY)` annotation. + +- **Security Best Practices** + - **Common Vulnerabilities and Prevention** + - **SQL Injection:** Use parameterized queries or ORM frameworks (e.g., Spring Data JPA) to prevent SQL injection. + - **Cross-Site Scripting (XSS):** Sanitize user input to prevent XSS attacks. + - **Cross-Site Request Forgery (CSRF):** Use CSRF protection mechanisms (e.g., Spring Security's CSRF support). + - **Authentication/Authorization Flaws:** Implement robust authentication and authorization mechanisms using Spring Security. + - **Dependency Vulnerabilities:** Regularly scan dependencies for known vulnerabilities and update them. + - **Input Validation** + - **Server-Side Validation:** Always perform input validation on the server-side. + - **JSR-303 Validation:** Use JSR-303 annotations for validating request parameters and body. + - **Custom Validation:** Implement custom validation logic for complex validation rules. + - **Authentication and Authorization** + - **Spring Security:** Use Spring Security for authentication and authorization. + - **OAuth 2.0/OpenID Connect:** Use OAuth 2.0/OpenID Connect for delegating authentication and authorization to external providers. + - **Role-Based Access Control (RBAC):** Implement RBAC to control access to resources based on user roles. + - **Data Protection** + - **Encryption:** Encrypt sensitive data at rest and in transit. + - **Hashing:** Hash passwords using strong hashing algorithms (e.g., bcrypt, Argon2). + - **Data Masking:** Mask sensitive data when displaying or logging it. + - **Secure API Communication** + - **HTTPS:** Use HTTPS for all API communication. + - **API Keys:** Use API keys for authenticating clients. + - **Rate Limiting:** Implement rate limiting to prevent abuse. + +- **Testing Approaches** + - **Unit Testing** + - **JUnit:** Use JUnit for writing unit tests. + - **Mockito:** Use Mockito for mocking dependencies. + - **Test-Driven Development (TDD):** Write tests before writing code. + - **Focus on Business Logic:** Focus unit tests on the business logic of the application. + - **Integration Testing** + - **Spring Boot Test:** Use `@SpringBootTest` annotation for integration tests. + - **TestRestTemplate:** Use `TestRestTemplate` for testing REST endpoints. + - **Database Testing:** Use an in-memory database (e.g., H2, embedded database) or a test database for integration tests. + - **Verify Component Interactions:** Verify that different components of the application work together correctly. + - **End-to-End Testing** + - **Selenium:** Use Selenium for end-to-end testing of web applications. + - **Cypress:** Use Cypress as another alternative for end-to-end testing. + - **Test the Entire System:** Test the entire system from the user interface to the database. + - **Test Organization** + - **Separate Test Directory:** Keep tests in a separate directory from the source code. + - **Test Naming Conventions:** Use clear and consistent naming conventions for tests (e.g., `ClassNameTest`). + - **Organize Tests by Component:** Organize tests by component or feature. + - **Mocking and Stubbing** + - **Mockito Annotations:** Use Mockito annotations (e.g., `@Mock`, `@InjectMocks`) to simplify mocking. + - **When/Then:** Use the `when()` and `thenReturn()` methods to define mock behavior. + - **Verify Interactions:** Use the `verify()` method to verify that mock interactions occurred. + +- **Common Pitfalls and Gotchas** + - **Frequent Mistakes** + - **Over-Engineering:** Avoid over-engineering solutions. Keep it simple and only add complexity when needed. + - **Ignoring Exceptions:** Don't catch exceptions and ignore them. Always log exceptions and handle them appropriately. + - **Hardcoding Values:** Avoid hardcoding values. Use configuration properties instead. + - **Not Using Dependency Injection Properly:** Use dependency injection to decouple components and improve testability. + - **Edge Cases** + - **Null Values:** Handle null values gracefully. + - **Concurrency Issues:** Be aware of concurrency issues when dealing with shared resources. + - **Resource Exhaustion:** Handle resource exhaustion gracefully (e.g., database connections, memory). + - **Version-Specific Issues** + - **Check Release Notes:** Review release notes for each version of Spring Boot and its dependencies to identify potential issues. + - **Test Upgrades:** Thoroughly test upgrades to ensure compatibility. + - **Compatibility Concerns** + - **Dependency Conflicts:** Manage dependency conflicts using Maven or Gradle. + - **Java Version Compatibility:** Ensure compatibility between Spring Boot and the Java version. + - **Debugging Strategies** + - **Logging:** Use logging liberally to track the flow of execution and identify issues. + - **Debuggers:** Use debuggers to step through code and inspect variables. + - **Remote Debugging:** Use remote debugging to debug applications running in remote environments. + +- **Tooling and Environment** + - **Recommended Development Tools** + - **IntelliJ IDEA:** Popular IDE with excellent Spring Boot support. + - **Eclipse:** Another popular IDE with Spring Tool Suite plugin. + - **Spring Initializr:** Web-based tool for generating Spring Boot projects. + - **Build Configuration** + - **Maven:** Use Maven for dependency management and building projects. + - **Gradle:** Use Gradle as another alternative to Maven. + - **Spring Boot Plugin:** Use the Spring Boot Maven or Gradle plugin for building executable JARs. + - **Linting and Formatting** + - **Checkstyle:** Use Checkstyle for enforcing coding standards. + - **PMD:** Use PMD for finding potential bugs and code smells. + - **SpotBugs:** Use SpotBugs to find potential bug patterns in the code. + - **Prettier:** Use Prettier for formatting code consistently. + - **Deployment Best Practices** + - **Containerization:** Use Docker for containerizing applications. + - **Cloud Platforms:** Deploy applications to cloud platforms (e.g., AWS, Azure, GCP). + - **Immutable Infrastructure:** Use immutable infrastructure for deploying applications. + - **CI/CD Integration** + - **Jenkins:** Use Jenkins for continuous integration and continuous deployment. + - **GitLab CI:** Use GitLab CI as another CI/CD alternative. + - **GitHub Actions:** Use GitHub Actions for automating build, test, and deployment workflows. + +- **Additional Tips** + - **Use Spring Boot DevTools:** These tools provide automatic application restarts, live reloading, and other useful features during development. + - **Monitor Application Health:** Use Spring Boot Actuator to monitor application health and performance metrics. + - **Stay Up-to-Date:** Keep up-to-date with the latest Spring Boot releases and best practices. \ No newline at end of file diff --git a/.cursor/rules/springboot.mdc b/.cursor/rules/springboot.mdc new file mode 100644 index 000000000..01b7f6344 --- /dev/null +++ b/.cursor/rules/springboot.mdc @@ -0,0 +1,339 @@ +--- +description: This rule provides comprehensive best practices and coding standards for developing robust, maintainable, and performant Spring Boot applications, covering code structure, performance, security, testing, and common pitfalls. +globs: *.java +--- +# Spring Boot Best Practices and Coding Standards + +This document outlines best practices and coding standards for developing applications with Spring Boot. Following these guidelines will help ensure that your applications are robust, maintainable, performant, and secure. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure + +Adopt a layered architecture to separate concerns and improve maintainability. A recommended directory structure is: + + +src/ + ├── main/ + │ ├── java/ + │ │ └── com/example/app/ + │ │ ├── Application.java (Main entry point) + │ │ ├── config/ (Configuration classes) + │ │ ├── controller/ (REST controllers) + │ │ ├── service/ (Business logic services) + │ │ ├── repository/ (Data access repositories) + │ │ ├── model/ (Data transfer objects (DTOs), entities) + │ │ ├── exception/ (Custom exceptions) + │ │ ├── util/ (Utility classes) + │ │ └── security/ (Security-related classes) + │ └── resources/ + │ ├── application.properties/application.yml (Application configuration) + │ ├── static/ (Static resources like HTML, CSS, JavaScript) + │ └── templates/ (View templates, e.g., Thymeleaf) + └── test/ + ├── java/ + │ └── com/example/app/ + │ ├── controller/ (Controller tests) + │ ├── service/ (Service tests) + │ └── repository/ (Repository tests) + └── resources/ + ├── application.properties/application.yml (Test-specific configuration) + + +* **Root Package:** Choose a meaningful root package name (e.g., `com.yourcompany.appname`). +* **Modularization:** For larger applications, consider breaking down the application into modules (e.g., using Maven or Gradle modules) based on business domains or features. + +### 1.2 File Naming Conventions + +* **Classes:** Use PascalCase (e.g., `UserController`, `ProductService`). +* **Interfaces:** Use PascalCase, often prefixed with `I` (e.g., `ProductRepository`, `IOrderService`). Consider omitting the `I` prefix if it doesn't add value. +* **Methods:** Use camelCase (e.g., `getUserById`, `calculateTotal`). +* **Variables:** Use camelCase (e.g., `userName`, `productPrice`). +* **Constants:** Use UPPER_SNAKE_CASE (e.g., `MAX_RETRIES`, `DEFAULT_TIMEOUT`). +* **Configuration Files:** Use lowercase with hyphens (e.g., `application.properties`, `bootstrap.yml`). + +### 1.3 Module Organization + +For larger projects, break down the application into modules. Each module should represent a distinct business domain or feature. + +* **Maven/Gradle Modules:** Use Maven or Gradle to manage module dependencies and build processes. +* **Clear Boundaries:** Define clear interfaces between modules to promote loose coupling. +* **Independent Deployments:** Design modules to be independently deployable, if possible. + +### 1.4 Component Architecture + +* **Controllers:** Handle incoming requests and delegate to services. Keep controllers thin. +* **Services:** Implement business logic. Services should be transactional. +* **Repositories:** Provide data access abstraction. Use Spring Data JPA or other data access technologies. +* **Models:** Represent data structures. Use DTOs for transferring data between layers and entities for persistence. + +### 1.5 Code Splitting Strategies + +* **Feature-Based Splitting:** Group code related to a specific feature into its own package or module. +* **Layer-Based Splitting:** Separate code based on layers (e.g., presentation, business logic, data access). +* **Horizontal vs. Vertical Slicing:** Consider horizontal slicing (grouping similar functionalities across features) or vertical slicing (grouping all functionalities for a specific feature) based on project needs. + +## 2. Common Patterns and Anti-Patterns + +### 2.1 Design Patterns Specific to Spring Boot + +* **Dependency Injection (DI):** Use constructor injection for required dependencies and setter injection for optional dependencies. +* **Inversion of Control (IoC):** Let the Spring container manage the lifecycle and dependencies of your beans. +* **Aspect-Oriented Programming (AOP):** Use AOP for cross-cutting concerns like logging, security, and transaction management. +* **Repository Pattern:** Use Spring Data repositories for simplified data access. +* **Service Layer Pattern:** Decouple controllers from business logic by introducing a service layer. +* **Template Method Pattern:** Use `JdbcTemplate` or `RestTemplate` for consistent data access or external API calls. +* **Factory Pattern:** Use `@Configuration` classes and `@Bean` methods to define and configure beans. + +### 2.2 Recommended Approaches for Common Tasks + +* **Configuration:** Use `application.properties` or `application.yml` for externalized configuration. Use `@ConfigurationProperties` to bind configuration properties to a class. +* **Logging:** Use SLF4J for logging abstraction and a suitable logging implementation (e.g., Logback or Log4j2). +* **Exception Handling:** Use `@ControllerAdvice` to handle exceptions globally. Create custom exception classes for specific error scenarios. +* **Validation:** Use JSR-303 Bean Validation for validating request parameters and request bodies. Use `@Validated` annotation with appropriate groups. +* **Data Transfer:** Use DTOs to transfer data between layers to avoid exposing internal data structures. +* **Asynchronous Operations:** Use `@Async` annotation and `TaskExecutor` to perform asynchronous operations. +* **Caching:** Use Spring's caching abstraction with implementations like Ehcache, Caffeine, or Redis. +* **Scheduling:** Use `@Scheduled` annotation to schedule tasks. +* **Transaction Management:** Use `@Transactional` annotation for managing transactions. + +### 2.3 Anti-Patterns and Code Smells to Avoid + +* **God Class:** A class that does too much. Break it down into smaller, more focused classes. +* **Long Method:** A method that is too long and complex. Extract smaller methods. +* **Feature Envy:** A method that accesses data from another object more than its own. Move the method to the other object. +* **Data Clumps:** Groups of data that frequently appear together. Create a new class to encapsulate the data clump. +* **Primitive Obsession:** Using primitive data types instead of creating meaningful domain objects. +* **Shotgun Surgery:** Making small changes in many different places. Refactor the code to centralize the changes. +* **Spaghetti Code:** Code that is difficult to understand and maintain due to its tangled structure. +* **Copy-Paste Programming:** Duplicating code instead of reusing existing code. Create reusable components or methods. +* **Field Injection:** Use constructor injection instead for required dependencies. +* **Tight Coupling:** Classes that are highly dependent on each other. Decouple the classes using interfaces or abstract classes. +* **Ignoring Exceptions:** Catching exceptions but not handling them properly. Log the exception and take appropriate action. +* **Over-Engineering:** Making the code too complex for the problem it solves. Keep it simple and only add complexity when needed. + +### 2.4 State Management Best Practices + +* **Stateless Services:** Design services to be stateless whenever possible. This improves scalability and testability. +* **Session Management:** Use Spring Session to manage user sessions in a distributed environment. Store session data in a persistent store like Redis or a database. +* **Caching:** Use caching to store frequently accessed data. Choose a suitable caching strategy (e.g., LRU, FIFO). +* **Database:** Use a relational database or a NoSQL database to persist data. +* **Distributed Transactions:** Use distributed transaction management techniques like two-phase commit (2PC) or Saga pattern for transactions spanning multiple services. + +### 2.5 Error Handling Patterns + +* **Global Exception Handling:** Use `@ControllerAdvice` and `@ExceptionHandler` to handle exceptions globally. +* **Custom Exceptions:** Create custom exception classes for specific error scenarios. +* **Logging:** Log exceptions with sufficient context information (e.g., request parameters, user ID). +* **Error Responses:** Return meaningful error responses with appropriate HTTP status codes and error messages. +* **Retry Mechanism:** Implement a retry mechanism for transient errors. +* **Circuit Breaker:** Use a circuit breaker pattern to prevent cascading failures. +* **Dead Letter Queue:** Use a dead letter queue to handle messages that cannot be processed. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +* **Database Query Optimization:** Use indexes, optimize queries, and avoid N+1 queries. +* **Caching:** Use caching to reduce database load and improve response times. +* **Connection Pooling:** Use connection pooling to reuse database connections. +* **Asynchronous Operations:** Use asynchronous operations to offload long-running tasks from the main thread. +* **Load Balancing:** Use load balancing to distribute traffic across multiple instances. +* **Gzip Compression:** Use Gzip compression to reduce the size of HTTP responses. +* **Code Profiling:** Use profiling tools to identify performance bottlenecks. + +### 3.2 Memory Management + +* **Object Pooling:** Use object pooling to reuse objects and reduce object creation overhead. +* **Avoid Memory Leaks:** Ensure that objects are properly garbage collected. +* **Use Appropriate Data Structures:** Choose data structures that are efficient for the operations you perform. +* **Optimize Collections:** Use appropriate collection types (e.g., `ArrayList` vs. `LinkedList`) based on usage patterns. +* **Lazy Loading:** Use lazy loading to load data only when it is needed. + +### 3.3 Rendering Optimization + +* **Template Caching:** Cache frequently used templates to reduce rendering time. +* **Minimize DOM Manipulations:** Minimize DOM manipulations in the view layer. +* **Use CDN:** Use a Content Delivery Network (CDN) to serve static resources. + +### 3.4 Bundle Size Optimization + +* **Code Splitting:** Split the code into smaller bundles to reduce the initial load time. +* **Tree Shaking:** Remove unused code from the bundles. +* **Minification:** Minify the code to reduce the bundle size. +* **Compression:** Compress the bundles to reduce the transfer size. + +### 3.5 Lazy Loading Strategies + +* **Lazy Initialization:** Initialize objects only when they are first accessed. +* **Virtual Proxy:** Use a virtual proxy to delay the loading of an object until it is needed. +* **Database Lazy Loading:** Use lazy loading features provided by JPA or other data access technologies. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and How to Prevent Them + +* **SQL Injection:** Use parameterized queries or ORM frameworks to prevent SQL injection attacks. +* **Cross-Site Scripting (XSS):** Sanitize user input and use output encoding to prevent XSS attacks. +* **Cross-Site Request Forgery (CSRF):** Use CSRF tokens to prevent CSRF attacks. +* **Authentication and Authorization:** Implement strong authentication and authorization mechanisms. +* **Session Management:** Secure session management to prevent session hijacking. +* **Denial of Service (DoS):** Implement rate limiting and other measures to prevent DoS attacks. +* **Insecure Direct Object References (IDOR):** Implement access control checks to prevent unauthorized access to objects. +* **Security Misconfiguration:** Properly configure security settings to prevent misconfigurations. +* **Using Components with Known Vulnerabilities:** Keep dependencies up-to-date to address known vulnerabilities. +* **Insufficient Logging and Monitoring:** Implement sufficient logging and monitoring to detect and respond to security incidents. + +### 4.2 Input Validation + +* **Whitelisting:** Validate input against a whitelist of allowed values. +* **Regular Expressions:** Use regular expressions to validate input format. +* **Data Type Validation:** Validate that input is of the expected data type. +* **Length Validation:** Validate that input is within the allowed length limits. +* **Encoding Validation:** Validate that input is properly encoded. + +### 4.3 Authentication and Authorization Patterns + +* **OAuth 2.0:** Use OAuth 2.0 for delegated authorization. +* **JWT (JSON Web Token):** Use JWT for stateless authentication. +* **Role-Based Access Control (RBAC):** Implement RBAC to control access to resources based on user roles. +* **Attribute-Based Access Control (ABAC):** Implement ABAC for fine-grained access control based on attributes. +* **Spring Security:** Leverage Spring Security for authentication and authorization. + +### 4.4 Data Protection Strategies + +* **Encryption:** Encrypt sensitive data at rest and in transit. +* **Hashing:** Hash passwords and other sensitive data using strong hashing algorithms. +* **Salting:** Use salting to protect against rainbow table attacks. +* **Data Masking:** Mask sensitive data when it is displayed or used for non-production purposes. +* **Tokenization:** Tokenize sensitive data to replace it with non-sensitive tokens. + +### 4.5 Secure API Communication + +* **HTTPS:** Use HTTPS for secure communication. +* **TLS/SSL:** Use TLS/SSL to encrypt data in transit. +* **API Keys:** Use API keys to authenticate API clients. +* **Rate Limiting:** Implement rate limiting to prevent abuse. +* **Input Validation:** Validate all input to prevent injection attacks. +* **Output Encoding:** Encode output to prevent XSS attacks. + +## 5. Testing Approaches + +### 5.1 Unit Testing Strategies + +* **Test-Driven Development (TDD):** Write tests before writing the code. +* **Mocking:** Use mocking frameworks (e.g., Mockito) to isolate the unit under test. +* **Assertion Libraries:** Use assertion libraries (e.g., AssertJ) for expressive assertions. +* **Code Coverage:** Aim for high code coverage. +* **Test Naming:** Use clear and descriptive test names. +* **Arrange-Act-Assert:** Structure tests using the Arrange-Act-Assert pattern. + +### 5.2 Integration Testing + +* **Test Slices:** Use Spring Boot's test slices (e.g., `@WebMvcTest`, `@DataJpaTest`) to test specific parts of the application. +* **TestContainers:** Use Testcontainers to run integration tests with real dependencies (e.g., databases, message queues). +* **Spring Test:** Use Spring's testing support for integration tests. +* **Database Testing:** Use an in-memory database or a test database for database integration tests. + +### 5.3 End-to-End Testing + +* **Selenium:** Use Selenium to automate browser-based end-to-end tests. +* **REST Assured:** Use REST Assured to test REST APIs. +* **Headless Browser:** Use a headless browser for faster end-to-end tests. + +### 5.4 Test Organization + +* **Test Packages:** Create separate packages for unit tests, integration tests, and end-to-end tests. +* **Test Classes:** Create test classes that correspond to the classes under test. +* **Test Suites:** Use test suites to group related tests. + +### 5.5 Mocking and Stubbing + +* **Mockito:** Use Mockito for mocking dependencies. +* **Spring MockMvc:** Use Spring MockMvc for testing controllers. +* **WireMock:** Use WireMock for stubbing external services. +* **Avoid Over-Mocking:** Mock only the dependencies that are necessary to isolate the unit under test. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes Developers Make + +* **Not Understanding Spring Boot Concepts:** Jumping into Spring Boot without a solid understanding of Spring and Dependency Injection. +* **Overusing `@Autowired`:** Using `@Autowired` for field injection instead of constructor injection. +* **Not Using Spring Boot Starters:** Manually adding dependencies instead of using Spring Boot Starters. +* **Not Externalizing Configuration:** Hardcoding configuration values instead of using `application.properties` or `application.yml`. +* **Not Handling Exceptions Properly:** Ignoring exceptions or not providing meaningful error responses. +* **Not Writing Tests:** Neglecting to write unit tests and integration tests. +* **Using `System.out.println` for Logging:** Using `System.out.println` instead of a proper logging framework. +* **Not Securing the Application:** Failing to implement proper security measures. +* **Not Monitoring the Application:** Not setting up proper monitoring and alerting. + +### 6.2 Edge Cases to Be Aware Of + +* **Null Values:** Handle null values gracefully. +* **Empty Collections:** Handle empty collections properly. +* **Large Datasets:** Optimize performance for large datasets. +* **Concurrency Issues:** Handle concurrency issues properly. +* **Network Errors:** Handle network errors gracefully. + +### 6.3 Version-Specific Issues + +* **Spring Boot Version Compatibility:** Ensure that dependencies are compatible with the Spring Boot version. +* **Java Version Compatibility:** Ensure that the Java version is compatible with the Spring Boot version. +* **Third-Party Library Compatibility:** Ensure that third-party libraries are compatible with the Spring Boot version. + +### 6.4 Compatibility Concerns + +* **Browser Compatibility:** Ensure that the application is compatible with different browsers. +* **Operating System Compatibility:** Ensure that the application is compatible with different operating systems. +* **Device Compatibility:** Ensure that the application is compatible with different devices. + +### 6.5 Debugging Strategies + +* **Logging:** Use logging to trace the execution flow and identify errors. +* **Debuggers:** Use debuggers to step through the code and inspect variables. +* **Profiling Tools:** Use profiling tools to identify performance bottlenecks. +* **Remote Debugging:** Use remote debugging to debug applications running on remote servers. +* **Heap Dumps:** Use heap dumps to analyze memory usage. +* **Thread Dumps:** Use thread dumps to analyze thread activity. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +* **IDE:** IntelliJ IDEA, Eclipse, or Visual Studio Code. +* **Build Tool:** Maven or Gradle. +* **Version Control:** Git. +* **Database Client:** DBeaver or SQL Developer. +* **API Testing Tool:** Postman or Insomnia. + +### 7.2 Build Configuration + +* **Maven:** Use `pom.xml` to define dependencies and build configuration. +* **Gradle:** Use `build.gradle` to define dependencies and build configuration. +* **Spring Boot Maven Plugin:** Use the Spring Boot Maven Plugin for packaging and running the application. +* **Spring Boot Gradle Plugin:** Use the Spring Boot Gradle Plugin for packaging and running the application. + +### 7.3 Linting and Formatting + +* **Checkstyle:** Use Checkstyle to enforce coding style guidelines. +* **PMD:** Use PMD to find potential code defects. +* **FindBugs/SpotBugs:** Use FindBugs/SpotBugs to find potential bugs. +* **EditorConfig:** Use EditorConfig to maintain consistent coding styles across different editors. +* **IntelliJ IDEA Code Style:** Configure IntelliJ IDEA's code style settings to match the project's coding style. + +### 7.4 Deployment Best Practices + +* **Containerization:** Use Docker to containerize the application. +* **Orchestration:** Use Kubernetes or Docker Swarm to orchestrate containers. +* **Cloud Deployment:** Deploy the application to a cloud platform (e.g., AWS, Azure, Google Cloud). +* **Configuration Management:** Use configuration management tools (e.g., Spring Cloud Config) to manage configuration in a distributed environment. +* **Monitoring:** Set up monitoring to track application performance and health. +* **Logging:** Aggregate logs to a central location for analysis. + +### 7.5 CI/CD Integration + +* **Continuous Integration (CI):** Use a CI server (e.g., Jenkins, Travis CI, CircleCI) to automatically build and test the application. +* **Continuous Delivery (CD):** Use a CD pipeline to automatically deploy the application to production. +* **Automated Testing:** Automate unit tests, integration tests, and end-to-end tests. +* **Code Quality Checks:** Integrate code quality checks (e.g., Checkstyle, PMD, FindBugs/SpotBugs) into the CI pipeline. \ No newline at end of file diff --git a/.cursor/rules/sqlalchemy.mdc b/.cursor/rules/sqlalchemy.mdc new file mode 100644 index 000000000..d0e5a32d0 --- /dev/null +++ b/.cursor/rules/sqlalchemy.mdc @@ -0,0 +1,261 @@ +--- +description: Enforces best practices for SQLAlchemy, covering code organization, performance, security, testing, and common pitfalls to ensure maintainable, efficient, and secure database interactions. +globs: *.py +--- +# SQLAlchemy Best Practices and Coding Standards + +This document outlines the best practices and coding standards for using SQLAlchemy in Python projects. Following these guidelines will help you write maintainable, efficient, and secure code. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure + +A well-organized directory structure improves code readability and maintainability. Here's a recommended structure for SQLAlchemy-based projects: + + +project_name/ + ├── app/ + │ ├── __init__.py + │ ├── models/ + │ │ ├── __init__.py + │ │ ├── user.py + │ │ ├── product.py + │ │ └── ... + │ ├── database.py # SQLAlchemy engine and session setup + │ ├── routes/ + │ │ ├── __init__.py + │ │ ├── user_routes.py + │ │ ├── product_routes.py + │ │ └── ... + │ ├── schemas/ + │ │ ├── __init__.py + │ │ ├── user_schema.py + │ │ ├── product_schema.py + │ │ └── ... + │ ├── utils.py + │ └── main.py # Entry point for the application + ├── tests/ + │ ├── __init__.py + │ ├── conftest.py # Fixtures for testing + │ ├── test_models.py + │ ├── test_routes.py + │ └── ... + ├── migrations/ + │ ├── versions/ + │ │ ├── ... (Alembic migration scripts) + │ ├── alembic.ini + │ └── env.py + ├── .env # Environment variables + ├── requirements.txt + ├── pyproject.toml # Define project dependencies + └── README.md + + +### 1.2 File Naming Conventions + +* **Models:** Use descriptive names for model files (e.g., `user.py`, `product.py`). +* **Schemas:** Use `_schema.py` suffix for schema files (e.g., `user_schema.py`). +* **Routes/Controllers:** Use `_routes.py` or `_controllers.py` suffix (e.g., `user_routes.py`). +* **Database:** A central `database.py` or `db.py` file is standard. +* **Migrations:** Alembic manages migration script names automatically. + +### 1.3 Module Organization + +* **Models:** Group related models into separate modules for clarity. +* **Schemas:** Define schemas in separate modules for serialization/deserialization. +* **Routes/Controllers:** Organize API endpoints into logical modules. + +### 1.4 Component Architecture + +* **Data Access Layer (DAL):** Abstract database interactions into a separate layer using the Repository Pattern to decouple the application logic from the database implementation. +* **Service Layer:** Implement business logic in a service layer that utilizes the DAL. +* **Presentation Layer:** (Routes/Controllers) Handle request processing and response generation. + +### 1.5 Code Splitting + +* **Model Definition:** Split large models into smaller, manageable classes. +* **Query Logic:** Move complex query logic into reusable functions or methods. +* **Configuration:** Externalize configuration settings using environment variables. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns + +* **Repository Pattern:** Centralizes data access logic, improving testability and maintainability. Example: + + python + class UserRepository: + def __init__(self, session: Session): + self.session = session + + def get_user_by_id(self, user_id: int) -> User | None: + return self.session.get(User, user_id) + +* **Unit of Work Pattern:** Tracks changes to multiple entities and commits them as a single transaction, ensuring data consistency. +* **Data Mapper Pattern:** Provides a layer of indirection between the database and domain objects, allowing for independent evolution. + +### 2.2 Recommended Approaches + +* **Declarative Base:** Use `declarative_base()` to define models. +* **Context Managers:** Use context managers for session management to ensure sessions are properly closed. +* **Parameterized Queries:** Always use parameterized queries to prevent SQL injection. +* **Eager Loading:** Use `joinedload()`, `subqueryload()`, or `selectinload()` to optimize query performance and avoid the N+1 problem. +* **Alembic:** Use Alembic for database migrations. + +### 2.3 Anti-patterns and Code Smells + +* **Raw SQL:** Avoid writing raw SQL queries whenever possible; leverage SQLAlchemy's ORM or Core features. +* **Global Sessions:** Avoid using global session objects; create sessions within request/transaction scopes. +* **Long-Lived Sessions:** Keep sessions short-lived to prevent stale data and concurrency issues. +* **Over-Fetching:** Avoid retrieving more data than necessary; use targeted queries. +* **N+1 Query Problem:** Identify and address the N+1 query problem using eager loading. + +### 2.4 State Management + +* **Session Scope:** Manage the SQLAlchemy session within the scope of a request or transaction. +* **Thread Safety:** Ensure thread safety when using SQLAlchemy in multi-threaded environments. +* **Asynchronous Sessions:** Use asynchronous sessions for non-blocking database operations in asynchronous applications. + +### 2.5 Error Handling + +* **Exception Handling:** Implement robust exception handling to catch database errors and prevent application crashes. +* **Rollbacks:** Use `session.rollback()` to revert changes in case of errors. +* **Logging:** Log database errors and queries for debugging and monitoring purposes. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +* **Indexing:** Add indexes to frequently queried columns to improve query performance. +* **Query Optimization:** Analyze query execution plans and optimize queries accordingly. +* **Connection Pooling:** Configure connection pooling to reuse database connections and reduce overhead. +* **Caching:** Implement caching strategies to reduce database load. +* **Batch Operations:** Use batch operations for bulk inserts, updates, and deletes. + +### 3.2 Memory Management + +* **Session Management:** Close sessions promptly to release resources. +* **Result Set Size:** Limit the size of result sets to prevent memory exhaustion. +* **Streaming Results:** Use streaming results for large datasets to reduce memory usage. + +### 3.3 Lazy Loading Strategies + +* **Joined Loading**: Load related entities in a single query using a JOIN. +* **Subquery Loading**: Load related entities using a subquery, suitable for complex relationships. +* **Selectin Loading**: Load related entities using a separate SELECT IN query, efficient for collections. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities + +* **SQL Injection:** Prevent SQL injection by using parameterized queries and avoiding string concatenation. +* **Data Exposure:** Protect sensitive data by encrypting it at rest and in transit. +* **Authentication Bypass:** Implement robust authentication and authorization mechanisms to prevent unauthorized access. + +### 4.2 Input Validation + +* **Schema Validation:** Use schemas to validate input data and ensure it conforms to the expected format. +* **Sanitization:** Sanitize input data to remove malicious characters and prevent cross-site scripting (XSS) attacks. + +### 4.3 Authentication and Authorization + +* **Authentication:** Use secure authentication protocols such as OAuth 2.0 or JWT (JSON Web Tokens). +* **Authorization:** Implement role-based access control (RBAC) or attribute-based access control (ABAC) to restrict access to resources. + +### 4.4 Data Protection + +* **Encryption:** Encrypt sensitive data at rest and in transit using strong encryption algorithms. +* **Hashing:** Hash passwords and other sensitive data using strong hashing algorithms. +* **Data Masking:** Mask sensitive data in non-production environments to prevent data breaches. + +### 4.5 Secure API Communication + +* **HTTPS:** Use HTTPS to encrypt communication between the client and the server. +* **API Keys:** Use API keys to authenticate API requests. +* **Rate Limiting:** Implement rate limiting to prevent denial-of-service (DoS) attacks. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +* **Model Testing:** Test model methods and properties. +* **Repository Testing:** Test repository methods in isolation. +* **Service Testing:** Test service layer logic. + +### 5.2 Integration Testing + +* **Database Integration:** Test database interactions and ensure data integrity. +* **API Integration:** Test API endpoints and ensure they function correctly. + +### 5.3 End-to-End Testing + +* **Full Application Testing:** Test the entire application workflow to ensure all components work together seamlessly. + +### 5.4 Test Organization + +* **Test Directory:** Organize tests into a separate `tests` directory. +* **Test Modules:** Create separate test modules for each component. +* **Test Fixtures:** Use test fixtures to set up test data and dependencies. + +### 5.5 Mocking and Stubbing + +* **Mocking Databases**: Use `unittest.mock` or `pytest-mock` to mock the SQLAlchemy engine and session during testing. +* **Patching External Dependencies**: Patch external dependencies to isolate the component under test. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +* **Forgetting to Commit:** Always commit changes to the database after making modifications. +* **Incorrect Relationship Configuration:** Ensure relationships are configured correctly to avoid data integrity issues. +* **Not Handling Exceptions:** Always handle exceptions to prevent application crashes. +* **Lack of Query Optimization:** Neglecting to optimize queries can lead to performance bottlenecks. + +### 6.2 Edge Cases + +* **Concurrency Issues:** Be aware of concurrency issues when multiple users access the database simultaneously. +* **Data Type Mismatches:** Ensure data types in the application and the database are compatible. +* **Large Result Sets:** Handle large result sets efficiently to avoid memory issues. + +### 6.3 Version-Specific Issues + +* **API Changes:** Be aware of API changes between different SQLAlchemy versions. +* **Compatibility Issues:** Ensure compatibility between SQLAlchemy and other libraries. + +### 6.4 Debugging Strategies + +* **Logging:** Use logging to track database queries and errors. +* **Debugging Tools:** Use debugging tools to step through code and inspect variables. +* **Query Analysis:** Analyze query execution plans to identify performance bottlenecks. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +* **IDE:** Use a good IDE such as VS Code, PyCharm, or Spyder. +* **Database Client:** Use a database client such as pgAdmin, Dbeaver, or MySQL Workbench. +* **SQLAlchemy Profiler:** Use an SQLAlchemy profiler to analyze query performance. + +### 7.2 Build Configuration + +* **Dependencies:** Use `requirements.txt` or `pyproject.toml` to manage dependencies. +* **Environment Variables:** Use environment variables to configure the application. + +### 7.3 Linting and Formatting + +* **Linting:** Use linters such as pylint or flake8 to enforce code style. +* **Formatting:** Use formatters such as black or autopep8 to automatically format code. + +### 7.4 Deployment Best Practices + +* **Database Configuration:** Configure the database connection settings correctly. +* **Security Hardening:** Harden the server and database to prevent security breaches. +* **Monitoring:** Implement monitoring to track application performance and errors. + +### 7.5 CI/CD Integration + +* **Automated Testing:** Run automated tests during the CI/CD pipeline. +* **Database Migrations:** Apply database migrations during deployment. +* **Rollbacks:** Implement rollbacks in case of deployment failures. + +By adhering to these best practices, you can build robust, scalable, and maintainable applications with SQLAlchemy. Remember to adapt these guidelines to your specific project requirements and context. \ No newline at end of file diff --git a/.cursor/rules/sqlite.mdc b/.cursor/rules/sqlite.mdc new file mode 100644 index 000000000..9953c658d --- /dev/null +++ b/.cursor/rules/sqlite.mdc @@ -0,0 +1,342 @@ +--- +description: This rule provides comprehensive guidance for SQLite development, covering best practices for schema design, performance optimization, security, testing, and more. It aims to ensure efficient, secure, and maintainable SQLite database applications. +globs: *.db,*.sqlite,*.sqlite3,*.sql +--- +# SQLite Best Practices and Coding Standards + +This document outlines the best practices and coding standards for developing with SQLite. It covers various aspects, including schema design, performance optimization, security considerations, testing strategies, common pitfalls, and tooling. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure + +A well-organized directory structure improves code maintainability and readability. + + +project_root/ +├── data/ +│ ├── production.db # Production database +│ ├── development.db # Development database +│ ├── test.db # Test database +│ └── migrations/ # Directory for database schema migrations +│ ├── 001_initial_schema.sql +│ └── 002_add_indexes.sql +├── src/ +│ ├── database.py # Database connection and helper functions +│ ├── models.py # Data models (if using an ORM) +│ ├── queries.py # Reusable SQL queries +│ └── utils.py # Utility functions +├── tests/ +│ ├── conftest.py # pytest configuration +│ ├── test_database.py # Tests for database operations +│ └── test_models.py # Tests for data models +├── .env # Environment variables +├── README.md # Project documentation +└── requirements.txt # Python dependencies + + +### 1.2. File Naming Conventions + +* Database files: Use `.db`, `.sqlite`, or `.sqlite3` extensions (e.g., `mydatabase.db`). +* SQL migration files: Prefix with a sequence number and use descriptive names (e.g., `001_create_users_table.sql`). +* Python modules: Use lowercase with underscores (e.g., `database.py`, `models.py`). + +### 1.3. Module Organization + +* **Database Connection Module:** Encapsulate database connection logic within a dedicated module. This promotes reusability and simplifies connection management. The module should handle connection establishment, cursor creation, and resource cleanup. + + python + # database.py + import sqlite3 + + DATABASE_PATH = "./data/mydatabase.db" + + def get_db_connection(): + conn = sqlite3.connect(DATABASE_PATH) + conn.row_factory = sqlite3.Row # Access columns by name + return conn + + def close_db_connection(conn): + if conn: + conn.close() + + +* **Data Models Module:** Define Python classes or data structures that represent database tables. This abstraction simplifies data access and manipulation, especially when using an ORM. Data models can include validation and serialization logic. + + python + # models.py + class User: + def __init__(self, id, username, email): + self.id = id + self.username = username + self.email = email + + +* **Queries Module:** Store reusable SQL queries in a separate module. This promotes code reuse, reduces redundancy, and makes it easier to maintain queries. Queries can be parameterized to prevent SQL injection vulnerabilities. + + python + # queries.py + CREATE_USER = """INSERT INTO users (username, email) VALUES (?, ?);""" + GET_USER_BY_ID = """SELECT id, username, email FROM users WHERE id = ?;""" + + +### 1.4. Component Architecture + +A layered architecture helps separate concerns and improve testability. + +* **Data Access Layer:** Manages interaction with the database. Contains functions for executing queries, retrieving data, and updating records. Uses the database connection and queries modules. +* **Business Logic Layer:** Contains the application's core logic. Processes data from the data access layer and implements business rules. Interacts with data models. +* **Presentation Layer:** Handles user interface and input/output. Communicates with the business logic layer to display data and receive user input. + +### 1.5. Code Splitting Strategies + +* **Feature-based Splitting:** Group related code into modules or packages based on application features (e.g., user management, product catalog). This improves modularity and simplifies navigation. +* **Functional Splitting:** Separate code based on functionality (e.g., data access, business logic, UI components). This promotes reuse and simplifies testing. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns + +* **Repository Pattern:** Abstract the data access layer behind an interface. Allows you to easily switch data sources or implement caching. +* **Data Mapper Pattern:** Transfer data between domain objects and the database. Useful when working with complex data structures. +* **Unit of Work Pattern:** Track changes to domain objects within a transaction. Ensures data consistency and simplifies transaction management. + +### 2.2. Recommended Approaches for Common Tasks + +* **Connecting to the database:** Use a connection pool to efficiently manage database connections, especially in multi-threaded environments. See example above in Module Organization. +* **Executing queries:** Use parameterized queries to prevent SQL injection. Always close cursors and connections when finished. +* **Fetching data:** Use `fetchall()` or iterate over the cursor to retrieve all results. Use `fetchone()` to retrieve a single row. +* **Handling transactions:** Use `conn.commit()` to save changes and `conn.rollback()` to undo changes in case of errors. Use `with conn:` to ensure transactions are handled properly. + +### 2.3. Anti-patterns and Code Smells + +* **Hardcoding SQL queries:** Avoid embedding SQL queries directly within application code. Use parameterized queries and store queries in a separate module. +* **Ignoring errors:** Always check for errors after executing SQL statements. Use try-except blocks to handle exceptions gracefully. +* **Leaking database connections:** Ensure that connections are closed properly after use. Use context managers (`with conn:`) to automate connection management. +* **Over-fetching data:** Only retrieve the columns that are needed for a particular operation. Use `EXPLAIN QUERY PLAN` to optimize queries. +* **Using string concatenation for SQL queries:** **Never** use string concatenation to build SQL queries. This is a major security vulnerability that can lead to SQL injection attacks. Always use parameterized queries. + +### 2.4. State Management + +* **Connection State:** Manage database connections at the application level. Use a single connection object for the entire application or connection pooling to efficiently manage multiple connections. +* **Transaction State:** Use transactions to ensure data consistency. Commit changes only when all operations are successful. Rollback changes in case of errors. +* **Data Caching:** Cache frequently accessed data in memory to improve performance. Use a caching library or implement a custom caching mechanism. + +### 2.5. Error Handling + +* **Use try-except blocks:** Wrap database operations in try-except blocks to catch potential exceptions. +* **Log errors:** Log error messages to a file or console for debugging purposes. +* **Rollback transactions:** If an error occurs during a transaction, rollback the transaction to prevent data corruption. +* **Raise custom exceptions:** Define custom exceptions to represent specific database errors. This allows for more fine-grained error handling in higher-level code. + + python + class DatabaseError(Exception): + pass + + try: + with conn: + cursor.execute("INSERT INTO users (username) VALUES (?)", ("invalid username",)) + except sqlite3.IntegrityError as e: + raise DatabaseError(f"Failed to insert user: {e}") + + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +* **Schema Design:** Keep the database schema simple and avoid unnecessary generalization. Use strict tables to enforce type checking and ensure primary keys are non-nullable. +* **Indexing:** Create indexes on columns that are frequently used in `WHERE` clauses or `JOIN` conditions. +* **Prepared Statements:** Prepare SQL statements and bind values to reduce parsing overhead. Use transactions for batch operations, and prefer temporary tables for intermediate results. Avoid excessive use of indices and sub-queries, as they can degrade performance. +* **Transactions:** Use transactions to group multiple operations into a single atomic unit. +* **Appropriate Data Types:** Choosing the right data type can significantly impact performance. For numeric primary keys, always use the `INTEGER` type for `rowid` aliasing. Limit the size of TEXT columns to prevent excessive storage usage. +* **Avoid `SELECT *`:** Instead, specify only the required columns. This reduces the amount of data transferred from the database. +* **Use `EXPLAIN QUERY PLAN`:** Analyze query execution plans to identify performance bottlenecks. This allows you to optimize queries by adding indexes or rewriting the query. +* **Consider `WITHOUT ROWID` tables:** For tables where the `rowid` is not needed and all columns are part of the primary key, using `WITHOUT ROWID` can save space and improve performance. +* **Optimize table order in joins**: Place tables without indexes on join columns at the far left of the SELECT statement. +* **Use `pragma optimize`**: Run this statement to optimize the database after making many changes. + +### 3.2. Memory Management + +* **Connection Pooling:** Use a connection pool to reuse database connections and reduce connection overhead. +* **Cursor Management:** Close cursors after use to release resources. Use context managers to automatically close cursors. +* **Result Set Size:** Limit the size of result sets to prevent excessive memory usage. Use pagination or filtering to reduce the amount of data retrieved. +* **Bulk operations:** Use `executemany` for bulk inserts or updates instead of looping and executing individual statements. +* **Large Blobs:** If storing large binary data (BLOBs), consider storing them as separate files and only store the file path in the database. + +### 3.3. Bundle Size Optimization (Applicable for applications bundling SQLite library) + +* **Strip Debug Symbols:** Remove debug symbols from the SQLite library to reduce its size. +* **Enable Compile-Time Options:** Disable unused SQLite features to reduce binary size. Compile with options like `-DSQLITE_OMIT_...` to remove features that are not needed. +* **Use a Minimal Build:** If possible, use a pre-built SQLite library that is optimized for size. + +### 3.4. Lazy Loading + +* **Lazy Loading Relationships:** Load related data only when it is needed. This can improve performance, especially when dealing with complex relationships. +* **Virtual Tables:** Use virtual tables to access data from external sources on demand. This can reduce memory usage and improve performance. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities and Prevention + +* **SQL Injection:** The most common vulnerability. Prevent it by using parameterized queries (also known as prepared statements) exclusively. Never construct SQL queries using string concatenation with user-provided input. +* **Database File Access:** Restrict access to the SQLite database file to only the necessary users and processes. Use appropriate file system permissions to prevent unauthorized access. +* **Denial of Service (DoS):** Limit the amount of memory and CPU time that SQLite can consume to prevent DoS attacks. Use the `sqlite3_limit()` and `sqlite3_hard_heap_limit64()` functions. +* **Malicious Database Files:** Be cautious when reading SQLite database files from untrusted sources. A malicious database file can contain triggers or virtual tables that execute arbitrary code. Use `PRAGMA integrity_check` and `PRAGMA quick_check` to verify the database file's integrity. + +### 4.2. Input Validation + +* **Sanitize User Input:** Always sanitize user input before using it in SQL queries. This includes escaping special characters and validating data types. +* **Use Constraints:** Define constraints on database columns to enforce data integrity. This can help prevent invalid data from being inserted into the database. +* **Limit Input Lengths:** Limit the length of text fields to prevent buffer overflows and DoS attacks. Always define a maximum length on `TEXT` columns. + +### 4.3. Authentication and Authorization + +* **SQLite Limitations:** SQLite itself does not provide built-in authentication or authorization mechanisms. +* **Application-Level Implementation:** Implement authentication and authorization logic in the application code that interacts with the database. This can involve checking user credentials against a stored hash or using a role-based access control system. +* **Encryption:** Encrypt the SQLite database file to protect sensitive data. Use a library like SQLCipher to encrypt the database file. + +### 4.4. Data Protection + +* **Encryption at Rest:** Encrypt the database file to protect sensitive data when it is stored on disk. This protects against unauthorized access to the database file itself. +* **Encryption in Transit:** Use SSL/TLS to encrypt communication between the application and the database, especially if the database is accessed over a network (though rare with SQLite). +* **Data Masking:** Mask sensitive data in query results or log files to prevent accidental disclosure. Use SQL functions or application-level logic to mask data. +* **Backup and Recovery:** Regularly back up the SQLite database file to prevent data loss. Store backups in a secure location. + +### 4.5. Secure API Communication + +* **HTTPS:** Use HTTPS for all API communication to protect data in transit. +* **API Keys:** Use API keys to authenticate API requests. +* **Rate Limiting:** Implement rate limiting to prevent DoS attacks. +* **Input Validation:** Validate all API inputs to prevent injection attacks. + +## 5. Testing Approaches + +### 5.1. Unit Testing + +* **Test Individual Components:** Unit tests should focus on testing individual components of the application in isolation. This includes testing data models, data access functions, and business logic. +* **Mock Database:** Use a mock database (e.g., an in-memory SQLite database) to isolate tests from the real database. This allows you to run tests quickly and reliably without affecting the production database. +* **Verify Queries:** Assert that the correct SQL queries are executed by the data access functions. This can be done by capturing the queries executed by the cursor and comparing them to expected values. +* **Test Edge Cases:** Test edge cases and error conditions to ensure that the application handles them gracefully. This includes testing invalid data inputs, database connection errors, and transaction failures. +* **Use `pytest` fixtures:** Employ `pytest` fixtures to set up and tear down the test environment (e.g., create and populate the mock database). + + python + # tests/conftest.py + import pytest + import sqlite3 + from src.database import get_db_connection, close_db_connection + + @pytest.fixture + def test_db(): + conn = sqlite3.connect(':memory:') # In-memory database + conn.row_factory = sqlite3.Row + cursor = conn.cursor() + cursor.execute("CREATE TABLE users (id INTEGER PRIMARY KEY, username TEXT)") + cursor.execute("INSERT INTO users (username) VALUES (?)", ("testuser",)) + conn.commit() + yield conn + conn.close() + + # tests/test_database.py + def test_get_user(test_db): + cursor = test_db.cursor() + cursor.execute("SELECT * FROM users WHERE username = ?", ("testuser",)) + user = cursor.fetchone() + assert user['username'] == "testuser" + + +### 5.2. Integration Testing + +* **Test Interactions:** Integration tests should focus on testing the interactions between different components of the application. This includes testing the interactions between the data access layer, the business logic layer, and the presentation layer. +* **Use a Test Database:** Use a separate test database for integration tests. This prevents integration tests from affecting the production database. +* **Verify Data Integrity:** Verify that data is correctly stored and retrieved from the database. This includes testing data validation, data transformations, and data relationships. + +### 5.3. End-to-End Testing + +* **Test the Entire Application:** End-to-end tests should focus on testing the entire application from the user interface to the database. This includes testing user workflows, data flows, and system integrations. +* **Use a Production-Like Environment:** Use a production-like environment for end-to-end tests. This ensures that the tests are run in an environment that is similar to the production environment. +* **Automate Tests:** Automate end-to-end tests to ensure that they are run regularly and consistently. + +### 5.4. Test Organization + +* **Separate Test Files:** Create separate test files for each component or module of the application. This makes it easier to organize and maintain tests. +* **Descriptive Test Names:** Use descriptive test names that clearly indicate what is being tested. This makes it easier to understand test results and debug failures. +* **Follow Arrange-Act-Assert Pattern:** Structure tests according to the Arrange-Act-Assert pattern. This makes tests more readable and maintainable. +* **Keep Tests Independent:** Ensure that tests are independent of each other. This prevents tests from interfering with each other and makes it easier to run tests in parallel. + +### 5.5. Mocking and Stubbing + +* **Mock External Dependencies:** Use mocking to isolate tests from external dependencies, such as network services or file systems. This allows you to test components in isolation without relying on external resources. +* **Stub Database Calls:** Use stubs to replace database calls with pre-defined results. This allows you to test components that interact with the database without actually accessing the database. +* **Verify Interactions:** Verify that the correct methods are called on mock objects or stubs. This ensures that components are interacting with each other as expected. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes + +* **SQL Injection:** Failing to use parameterized queries. +* **Data Type Mismatches:** Inserting data with incorrect data types into database columns. Strict mode and explicit type definitions can help mitigate this. +* **Concurrency Issues:** Not handling concurrency correctly in multi-threaded environments. Use connection pooling and proper transaction management. +* **Deadlocks:** Creating deadlocks by acquiring locks in the wrong order. Be aware of transaction isolation levels and lock contention. +* **Schema Evolution:** Failing to plan for schema evolution. Use database migrations to manage schema changes. + +### 6.2. Edge Cases + +* **Large Databases:** Handling databases with large amounts of data. Use pagination, indexing, and query optimization techniques. +* **Concurrent Access:** Handling concurrent access to the database. Use connection pooling, transactions, and locking mechanisms. +* **Corrupted Database Files:** Handling corrupted database files. Use `PRAGMA integrity_check` and implement a backup and recovery strategy. +* **Full Disk:** Handle situations where disk space is exhausted and database writes fail. + +### 6.3. Version-Specific Issues + +* **Compatibility Changes:** Be aware of compatibility changes between different versions of SQLite. Consult the SQLite documentation for details. +* **New Features:** Take advantage of new features in newer versions of SQLite. This can improve performance, security, or functionality. + +### 6.4. Compatibility Concerns + +* **Cross-Platform Compatibility:** Ensure that the application is compatible with different operating systems and architectures. Test the application on different platforms. +* **Data Type Differences:** Be aware of data type differences between SQLite and other database systems. Use compatible data types or perform data conversions as needed. + +### 6.5. Debugging Strategies + +* **Logging:** Use logging to track database operations and errors. Log SQL queries, execution times, and error messages. +* **Debugging Tools:** Use debugging tools to inspect the database schema, data, and query execution plans. Tools like DB Browser for SQLite are very useful. +* **Error Messages:** Pay attention to error messages. Error messages can provide valuable clues about the cause of the problem. +* **Simplify Queries:** Simplify complex queries to isolate the source of the problem. +* **Replicate the Issue:** Try to replicate the issue in a controlled environment. This can help you identify the root cause of the problem. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +* **DB Browser for SQLite:** A graphical tool for managing SQLite databases. +* **SQLite command-line shell:** A command-line tool for interacting with SQLite databases. Available as `sqlite3`. +* **Python sqlite3 module:** Python's built-in module for working with SQLite databases. +* **SQLCipher:** An open-source extension to SQLite that provides transparent encryption. +* **ORM (Object-Relational Mapper):** Libraries like SQLAlchemy or PonyORM can simplify database interactions, but use caution and understand the underlying SQL generated, as ORMs can sometimes lead to inefficient queries. + +### 7.2. Build Configuration + +* **Dependencies:** Manage dependencies using a dependency management tool (e.g., `pip`, `poetry`, `conda`). +* **Compiler Options:** Use appropriate compiler options to optimize the SQLite library for performance. +* **Linking:** Link the SQLite library statically or dynamically depending on the application requirements. + +### 7.3. Linting and Formatting + +* **SQL Linters:** Use SQL linters to enforce coding standards and identify potential errors in SQL queries. Use tools like `sqlfluff`. +* **Code Formatters:** Use code formatters to automatically format SQL queries and code. Use formatters available through IDE extensions or command-line tools. +* **Consistency:** Maintain consistency in code formatting and style across the project. + +### 7.4. Deployment + +* **Database File Location:** Store the SQLite database file in a secure location on the server. +* **File Permissions:** Set appropriate file permissions on the database file to prevent unauthorized access. +* **Backup Strategy:** Implement a backup and recovery strategy to protect against data loss. +* **Connection Limits:** Configure connection limits to prevent denial-of-service attacks. + +### 7.5. CI/CD Integration + +* **Automated Tests:** Integrate automated tests into the CI/CD pipeline to ensure that code changes do not break existing functionality. +* **Database Migrations:** Automate database migrations as part of the deployment process. +* **Rollback Strategy:** Implement a rollback strategy to revert to a previous version of the application in case of deployment failures. + +By following these best practices and coding standards, developers can create efficient, secure, and maintainable SQLite database applications. \ No newline at end of file diff --git a/.cursor/rules/statsmodels.mdc b/.cursor/rules/statsmodels.mdc new file mode 100644 index 000000000..a9d96b164 --- /dev/null +++ b/.cursor/rules/statsmodels.mdc @@ -0,0 +1,289 @@ +--- +description: A comprehensive guide to best practices for using the statsmodels library in Python, covering code organization, performance, testing, and common pitfalls. These guidelines promote maintainable, reliable, and efficient statsmodels code. +globs: *.py +--- +# Statsmodels Library Best Practices + +This document outlines best practices and coding standards for effectively using the statsmodels library in Python for statistical modeling, machine learning, and data science applications. Following these guidelines will help ensure code that is readable, maintainable, efficient, and statistically sound. + +## Library Information: +- Name: statsmodels +- Tags: ai, ml, data-science, python, statistics + +## 1. Code Organization and Structure + +### 1.1 Directory Structure + +Adopt a clear and organized directory structure for your projects: + + +project_root/ +├── data/ # Raw and processed datasets +├── models/ # Saved model artifacts +├── scripts/ # Data processing, model training, evaluation scripts +├── notebooks/ # Exploratory data analysis and prototyping (use sparingly for final code) +├── tests/ # Unit, integration, and end-to-end tests +├── docs/ # Project documentation +├── requirements.txt # Project dependencies +└── main.py # Entry point for the application (if applicable) + + +### 1.2 File Naming Conventions + +- Use descriptive and consistent file names. +- Data files: `data_description.csv`, `data_description.parquet` +- Script files: `process_data.py`, `train_model.py`, `evaluate_model.py` +- Model files: `model_name.pkl` (if pickling, but consider other serialization methods) +- Test files: `test_module.py` + +### 1.3 Module Organization + +- Break down your code into reusable modules. +- `data_loading.py`: Functions for loading and preprocessing data. +- `model_definition.py`: Classes or functions for defining statsmodels models. +- `model_training.py`: Functions for training models. +- `model_evaluation.py`: Functions for evaluating model performance. +- `utils.py`: Utility functions used throughout the project. + +### 1.4 Component Architecture + +- Employ a modular architecture to separate concerns. +- **Data Layer:** Handles data loading, cleaning, and transformation. +- **Model Layer:** Defines and trains statsmodels models. +- **Evaluation Layer:** Assesses model performance using appropriate metrics. +- **Application Layer:** Integrates the model into an application (if applicable). + +### 1.5 Code Splitting Strategies + +- Split large files into smaller, more manageable modules. +- Group related functions and classes into separate modules. +- Use clear and concise function and class names to indicate their purpose. +- Consider a `config.py` file for global project settings. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns + +- **Factory Pattern:** Use a factory pattern to create different statsmodels models based on configuration. +- **Strategy Pattern:** Implement different evaluation strategies using the strategy pattern. +- **Observer Pattern:** If changes in the underlying data need to trigger model retraining, consider the observer pattern. + +### 2.2 Recommended Approaches for Common Tasks + +- **Data Preprocessing:** Always use Pandas DataFrames for data manipulation before feeding data into statsmodels. +- **Model Selection:** Choose models based on the statistical properties of your data and the research question. +- **Model Fitting:** Use `statsmodels.api` to fit models, and carefully interpret the output. +- **Result Interpretation:** Focus on coefficients, p-values, confidence intervals, and model diagnostics. +- **Visualization:** Utilize Matplotlib and Seaborn to visualize data, model results, and diagnostics. + +### 2.3 Anti-patterns and Code Smells + +- **Magic Numbers:** Avoid hardcoding constants directly in your code; define them with descriptive names. +- **Copy-Pasted Code:** Refactor duplicated code into reusable functions or classes. +- **Overly Long Functions:** Break down long functions into smaller, more manageable units. +- **Lack of Documentation:** Always document your code with docstrings to explain its purpose and usage. +- **Ignoring Warnings:** Pay attention to warnings generated by statsmodels; they often indicate potential issues. + +### 2.4 State Management + +- Avoid global state as much as possible. Pass data and model parameters explicitly. +- If you need to persist model state, use appropriate serialization techniques (e.g., pickling, but with caution due to security risks). Consider alternatives like ONNX or joblib. +- For complex applications, use dependency injection frameworks to manage dependencies and state. + +### 2.5 Error Handling + +- Use try-except blocks to handle potential errors gracefully. +- Log errors and warnings using the `logging` module. +- Raise exceptions with informative error messages to help with debugging. +- Consider custom exception classes for specific statsmodels-related errors. + +python +import logging + +logger = logging.getLogger(__name__) + +try: + model = sm.OLS(y, X).fit() +except Exception as e: + logger.error(f"Error fitting model: {e}") + raise # Re-raise the exception for higher-level handling + + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +- **Vectorization:** Utilize NumPy's vectorized operations whenever possible to speed up computations. +- **Profiling:** Use profiling tools like `cProfile` to identify performance bottlenecks. +- **Caching:** Cache frequently used results to avoid redundant computations (use `functools.lru_cache` for example). +- **Algorithm Selection:** Choose the most efficient algorithms for your specific task (e.g., different optimization methods in statsmodels). + +### 3.2 Memory Management + +- **Data Types:** Use appropriate data types to minimize memory usage (e.g., `np.int32` instead of `np.int64` if possible). +- **Lazy Loading:** Load large datasets in chunks to avoid loading the entire dataset into memory at once. +- **Garbage Collection:** Explicitly release unused memory using `del` or `gc.collect()` if necessary. + +### 3.3 Parallelization + +- Explore parallelization options using libraries like `multiprocessing` or `joblib` for computationally intensive tasks. +- Statsmodels may leverage underlying NumPy and SciPy functions that support parallel execution. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities + +- **Pickle Deserialization:** Avoid deserializing untrusted pickle files, as they can execute arbitrary code. Use safer serialization formats like JSON or ONNX. +- **Injection Attacks:** Sanitize user inputs to prevent injection attacks if your application takes user-provided data and uses it in statsmodels models. +- **Denial of Service (DoS):** Implement rate limiting and resource constraints to prevent DoS attacks on your statsmodels-based services. + +### 4.2 Input Validation + +- Validate all input data to ensure it conforms to the expected format and range. +- Use schemas (e.g., using `jsonschema` or `pydantic`) to define and enforce data validation rules. +- Check for missing values, outliers, and inconsistencies in the data. + +### 4.3 Authentication and Authorization + +- Implement authentication and authorization mechanisms to control access to your statsmodels-based services. +- Use secure authentication protocols like OAuth 2.0 or JWT. +- Enforce role-based access control (RBAC) to restrict access to sensitive data and operations. + +### 4.4 Data Protection + +- Encrypt sensitive data at rest and in transit. +- Use secure communication protocols like HTTPS. +- Implement data masking and anonymization techniques to protect user privacy. + +### 4.5 Secure API Communication + +- Use secure APIs (e.g., REST APIs with HTTPS) to communicate with your statsmodels services. +- Implement input validation and output sanitization to prevent injection attacks. +- Use API keys or other authentication mechanisms to secure your APIs. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +- Write unit tests for individual functions and classes. +- Use the `unittest` or `pytest` framework. +- Test edge cases and boundary conditions. +- Mock external dependencies to isolate the code being tested. + +python +import unittest +import statsmodels.api as sm +import numpy as np + +class TestOLS(unittest.TestCase): + def test_ols_fit(self): + # Create some sample data + X = np.array([[1, 1], [1, 2], [1, 3]]) + y = np.array([2, 4, 5]) + + # Fit an OLS model + model = sm.OLS(y, X).fit() + + # Assert that the model converged + self.assertTrue(model.converged) + + # Assert that the coefficients are close to the expected values + expected_coefs = np.array([0.5, 1.5]) + np.testing.assert_allclose(model.params, expected_coefs, rtol=1e-5) + +if __name__ == '__main__': + unittest.main() + + +### 5.2 Integration Testing + +- Write integration tests to verify the interaction between different components. +- Test the data pipeline from data loading to model evaluation. +- Verify that the model produces correct results on sample datasets. + +### 5.3 End-to-End Testing + +- Write end-to-end tests to simulate real-world usage scenarios. +- Test the entire application from start to finish. +- Use tools like Selenium or Cypress to automate browser-based testing (if applicable). + +### 5.4 Test Organization + +- Organize your tests in a separate `tests` directory. +- Use a consistent naming convention for test files (e.g., `test_module.py`). +- Group related tests into test classes. + +### 5.5 Mocking and Stubbing + +- Use mocking libraries like `unittest.mock` or `pytest-mock` to isolate the code being tested. +- Mock external dependencies like databases or APIs. +- Stub out complex functions to simplify testing. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +- **Incorrect Model Specification:** Choosing the wrong model for the data. +- **Ignoring Data Assumptions:** Failing to check the assumptions of the statistical tests or models being used. +- **Overfitting:** Training a model that performs well on the training data but poorly on unseen data. +- **Misinterpreting Results:** Drawing incorrect conclusions from the model output. +- **Not Scaling Features**: Some models will perform poorly if the data is not scaled or normalized + +### 6.2 Edge Cases + +- **Multicollinearity:** Independent variables being highly correlated +- **Missing Data:** Handling missing values appropriately. +- **Outliers:** Identifying and handling outliers in the data. +- **Non-Normal Data:** Dealing with data that doesn't follow a normal distribution. + +### 6.3 Version-Specific Issues + +- Be aware of changes in statsmodels API between versions. +- Check the release notes for any breaking changes or bug fixes. +- Use a virtual environment to manage dependencies and ensure compatibility. + +### 6.4 Compatibility Concerns + +- Ensure compatibility between statsmodels and other libraries like NumPy, SciPy, and Pandas. +- Check the documentation for any known compatibility issues. + +### 6.5 Debugging Strategies + +- Use a debugger (e.g., `pdb`) to step through the code and inspect variables. +- Add logging statements to track the execution flow and identify potential issues. +- Use assertions to verify that the code is behaving as expected. +- Consult the statsmodels documentation and community forums for help. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +- **IDE:** VS Code, PyCharm, or other Python IDE. +- **Virtual Environment Manager:** `venv`, `conda`. +- **Package Manager:** `pip`, `conda`. +- **Debugger:** `pdb`, `ipdb`. +- **Profiler:** `cProfile`. + +### 7.2 Build Configuration + +- Use `setuptools` or `poetry` to manage project dependencies and build configurations. +- Create a `requirements.txt` file to specify project dependencies. +- Use a `setup.py` file to define the project metadata and build process. + +### 7.3 Linting and Formatting + +- Use linters like `flake8` or `pylint` to enforce code style and identify potential errors. +- Use formatters like `black` or `autopep8` to automatically format your code. +- Configure your IDE to run linters and formatters automatically on save. + +### 7.4 Deployment + +- Containerize your application using Docker. +- Use a deployment platform like AWS, Azure, or Google Cloud. +- Monitor your application for performance and errors. + +### 7.5 CI/CD + +- Use a CI/CD platform like GitHub Actions, Jenkins, or CircleCI to automate the build, test, and deployment process. +- Run unit tests, integration tests, and end-to-end tests as part of the CI/CD pipeline. +- Deploy your application automatically to a staging or production environment after successful testing. \ No newline at end of file diff --git a/.cursor/rules/streamlit.mdc b/.cursor/rules/streamlit.mdc new file mode 100644 index 000000000..7add946bd --- /dev/null +++ b/.cursor/rules/streamlit.mdc @@ -0,0 +1,74 @@ +--- +description: This rule provides guidelines and best practices for developing maintainable, performant, and secure Streamlit applications. It covers code organization, performance optimization, security considerations, testing strategies, and common pitfalls to avoid. +globs: *.py +--- +- **Code Organization and Structure**: + - **Directory Structure**: Structure your Streamlit application using a modular directory structure to improve maintainability and collaboration. A suggested layout includes folders for pages, services, models, components, and utilities. + + my_streamlit_app/ + ├── app.py # Main entry point for the Streamlit app + ├── pages/ # Directory for individual app pages + │ ├── dashboard.py # Dashboard page + │ ├── reports.py # Reports page + │ └── settings.py # Settings page + ├── services/ # Directory for business logic and data operations + │ ├── data_loader.py # Handles data loading + │ ├── analysis.py # Contains data analysis functions + │ └── utils.py # Utility functions + ├── models/ # Directory for data models and schemas + │ └── data_model.py # Defines data structures + ├── components/ # Directory for reusable UI components + │ ├── button.py # Custom button component + │ └── chart.py # Custom chart component + ├── utils/ # Miscellaneous utility functions + │ └── helpers.py # Helper functions + ├── requirements.txt # List of Python dependencies + ├── .gitignore # Specifies intentionally untracked files that Git should ignore + └── .venv/ # Python virtual environment (optional) + + - **File Naming Conventions**: Use descriptive and consistent file names for your Streamlit components, services, and pages. For example, `data_loader.py`, `dashboard.py`, and `button.py`. + - **Module Organization**: Organize your Streamlit application into logical modules to promote code reuse and reduce complexity. Create separate modules for data loading, data processing, UI components, and utility functions. Use relative imports to maintain a clear module structure. + - **Component Architecture**: Build reusable UI components using Streamlit's `st.components.v1` API or custom functions. Encapsulate component logic and styling to promote consistency and maintainability. Use props or parameters to customize component behavior and appearance. + - **Code Splitting**: Split large Streamlit applications into multiple pages using the `st.page_config` and `st.switch_page` functions. This enhances navigation and reduces initial loading times. Consider lazy loading strategies using the `secrets` feature to control which parts of the application are loaded when. + +- **Common Patterns and Anti-patterns**: + - **Design Patterns**: Employ the Observer pattern for reactive UIs, Facade pattern for simplifying complex interactions, and Strategy pattern to manage different data sources or algorithms. + - **Recommended Approaches**: Utilize Streamlit's `st.form` for handling user input and submission. Use `st.session_state` to manage application state across reruns. Leverage Streamlit's caching mechanisms (`@st.cache_data`, `@st.cache_resource`) to optimize performance. + - **Anti-patterns**: Avoid performing expensive computations or data loading operations directly within the main Streamlit application loop. Do not store large datasets in `st.session_state`. Avoid using global variables, as they can lead to unexpected behavior. + - **State Management**: Use `st.session_state` for storing and managing application state across reruns. Initialize session state variables using a conditional statement (`if 'key' not in st.session_state:`) to prevent overwriting values on subsequent reruns. Consider implementing a state management class or module to encapsulate and manage complex state logic. + - **Error Handling**: Implement comprehensive error handling throughout your Streamlit application. Use `try...except` blocks to catch exceptions and display informative error messages to the user using `st.error` or `st.exception`. Log errors to a file or monitoring system for debugging and analysis. + +- **Performance Considerations**: + - **Optimization Techniques**: Use caching (`@st.cache_data`, `@st.cache_resource`) to store the results of expensive computations or data loading operations. Optimize data loading by using appropriate data formats (e.g., Parquet, Feather) and filtering data at the source. Debounce user interactions to reduce the frequency of reruns. + - **Memory Management**: Avoid loading large datasets into memory unnecessarily. Use data streaming techniques or chunking to process large datasets in smaller portions. Delete unused variables and data structures to release memory. + - **Rendering Optimization**: Use Streamlit's built-in rendering optimizations, such as delta generation, to minimize the amount of data that needs to be sent to the browser. Avoid creating complex layouts with deeply nested components, as this can impact rendering performance. + - **Bundle Size**: Minimize external dependencies to reduce the bundle size and improve loading times. Use a `.streamlitignore` file to exclude unnecessary files and directories from the Streamlit deployment bundle. Consider using a CDN to serve static assets. + - **Lazy Loading**: Implement lazy loading for expensive components or sections of your Streamlit application. Load components only when they are needed, using conditional rendering or callbacks. Utilize the `secrets` functionality to control the conditional loading of certain modules if access to a given secret is available. + +- **Security Best Practices**: + - **Common Vulnerabilities**: Be aware of common vulnerabilities, such as cross-site scripting (XSS), SQL injection, and command injection. Prevent these vulnerabilities by implementing proper input validation and sanitization, and by avoiding the execution of untrusted code. + - **Input Validation**: Validate all user inputs to prevent malicious code from being injected into your Streamlit application. Use regular expressions or custom validation functions to ensure that inputs conform to expected formats and ranges. Sanitize user inputs by encoding or escaping special characters. + - **Authentication and Authorization**: Implement authentication and authorization mechanisms to control access to sensitive data and functionality. Use Streamlit's `secrets` API to store API keys, passwords, and other sensitive information securely. Consider using a third-party authentication provider, such as Auth0 or Google Identity Platform. + - **Data Protection**: Protect sensitive data by encrypting it at rest and in transit. Use HTTPS to secure communication between the browser and the Streamlit application. Implement data masking or redaction to prevent sensitive data from being displayed to unauthorized users. + - **Secure API Communication**: Always use HTTPS for API communications. Validate the origin and authenticity of data from external APIs. Protect API keys using Streamlit secrets and environment variables. + +- **Testing Approaches**: + - **Unit Testing**: Unit test individual Streamlit components, services, and utility functions using the `pytest` framework. Mock external dependencies to isolate the code being tested. Write test cases that cover a range of inputs and edge cases. + - **Integration Testing**: Integrate test Streamlit applications to verify that components and modules work together correctly. Use Streamlit's testing utilities to simulate user interactions and verify the output. + - **End-to-end Testing**: Perform end-to-end tests to verify that the entire Streamlit application works as expected. Use Selenium or Cypress to automate browser interactions and verify the UI and functionality. + - **Test Organization**: Organize your tests into a separate `tests` directory within your Streamlit project. Use descriptive test names and docstrings to explain the purpose of each test. Follow a consistent naming convention for test files and functions. + - **Mocking and Stubbing**: Use mocking and stubbing techniques to isolate the code being tested and to simulate external dependencies. Use the `unittest.mock` module or a third-party mocking library to create mock objects and stubs. + +- **Common Pitfalls and Gotchas**: + - **Frequent Mistakes**: Forgetting to use `@st.cache_data` or `@st.cache_resource` for expensive operations, not managing `st.session_state` properly, and neglecting error handling are common mistakes. + - **Edge Cases**: Be aware of edge cases, such as handling missing data, dealing with invalid inputs, and managing concurrent user sessions. + - **Version-Specific Issues**: Check the Streamlit documentation and release notes for version-specific issues and compatibility concerns. Be aware of breaking changes when upgrading to a newer version of Streamlit. + - **Compatibility Concerns**: Ensure that your Streamlit application is compatible with the browsers and operating systems that your users will be using. Test your application on different devices and browsers to identify and resolve compatibility issues. + - **Debugging Strategies**: Use Streamlit's debugging tools, such as the debugger and the console, to identify and resolve issues. Print debug messages to the console to track the execution flow and to inspect variables. Use `st.experimental_rerun` to programmatically trigger reruns during debugging. + +- **Tooling and Environment**: + - **Recommended Tools**: Use VS Code with the Python extension for development. Use `pip` or `uv` for dependency management, and Docker for containerization. + - **Build Configuration**: Create a `requirements.txt` file to list your Streamlit application's dependencies. Use a virtual environment to isolate your project's dependencies from the system-level Python installation. Consider using a build tool, such as Make or Poetry, to automate the build process. + - **Linting and Formatting**: Use linters and formatters, such as `flake8` and `black`, to enforce code style guidelines and to identify potential errors. Configure your editor to automatically run linters and formatters on save. + - **Deployment**: Deploy your Streamlit application to Streamlit Cloud, Heroku, AWS, or another cloud platform. Use Docker to containerize your application and to ensure consistent deployment across different environments. + - **CI/CD Integration**: Integrate your Streamlit project with a CI/CD pipeline to automate the testing, building, and deployment processes. Use GitHub Actions, GitLab CI, or another CI/CD platform to configure your pipeline. \ No newline at end of file diff --git a/.cursor/rules/stripe.mdc b/.cursor/rules/stripe.mdc new file mode 100644 index 000000000..adc7af465 --- /dev/null +++ b/.cursor/rules/stripe.mdc @@ -0,0 +1,284 @@ +--- +description: This rule file outlines best practices for integrating Stripe's payment processing services into web and mobile applications, focusing on security, performance, and maintainability. It provides guidelines on coding standards, error handling, and testing to ensure a robust and reliable Stripe integration. +globs: *.js,*.jsx,*.ts,*.tsx,*.py,*.rb,*.php,*.java,*.go +--- +# Stripe Integration Best Practices + +This document provides a comprehensive guide to best practices for integrating Stripe's payment processing services. It covers code organization, common patterns, performance considerations, security, testing, common pitfalls, and tooling. + +## Library Information: +- Name: stripe +- Tags: payments, api, e-commerce, financial + +## 1. Code Organization and Structure + +### 1.1 Directory Structure Best Practices + +Organizing your project effectively enhances maintainability and readability. Consider the following structure: + + +project-root/ +├── src/ +│ ├── components/ # Reusable UI components (if applicable) +│ ├── services/ +│ │ └── stripe/ # Stripe-related services +│ │ ├── payments.js/ts # Payment processing logic +│ │ ├── subscriptions.js/ts # Subscription management +│ │ ├── webhooks.js/ts # Webhook handling +│ │ ├── utils.js/ts # Utility functions +│ ├── models/ # Data models and types +│ ├── config/ # Configuration files (API keys, etc.) +│ ├── utils/ # General utility functions +├── tests/ # Test suites +├── .env # Environment variables +└── ... + + +* `components`: If you're building a UI, this directory holds reusable UI components. This might include components that display payment forms or payment status indicators. +* `services/stripe`: This directory encapsulates all Stripe-related interactions. This promotes separation of concerns and makes it easier to update or replace the Stripe integration in the future. +* `payments.js/ts`: Contains functions for creating charges, handling payment intents, and processing refunds. +* `subscriptions.js/ts`: Includes logic for creating, updating, and canceling subscriptions. +* `webhooks.js/ts`: Handles incoming Stripe webhook events. Crucially important for asynchronous payment flows. +* `utils.js/ts`: Contains utility functions, such as formatting currency or validating data. +* `models`: Defines data models for Stripe objects (e.g., Customer, Charge, PaymentIntent). This can be useful for type checking and data validation. +* `config`: Stores configuration settings, including your Stripe API keys. Never commit your secret keys to version control. Use environment variables. +* `utils`: General utility functions used throughout your application. +* `tests`: Contains unit and integration tests for your Stripe integration. + +### 1.2 File Naming Conventions + +* Use descriptive names for files and functions. +* Follow the naming conventions of your chosen language (e.g., camelCase in JavaScript, snake_case in Python). +* Example: + * `stripePayments.js` (JavaScript) + * `stripe_payments.py` (Python) + * `StripePayments.java` (Java) + +### 1.3 Module Organization + +* Break down your Stripe integration into smaller, reusable modules. +* Use clear and consistent naming conventions for modules. +* Export only the necessary functions and classes from each module to maintain a clean API. + +### 1.4 Component Architecture (If Applicable) + +* If building a UI, design reusable components for payment forms, payment status displays, and other Stripe-related elements. +* Use a component-based framework (e.g., React, Vue.js, Angular) to manage UI complexity. +* Consider using a state management library (e.g., Redux, Zustand, Vuex) to manage Stripe-related state. + +### 1.5 Code Splitting Strategies + +* Lazy-load Stripe-related code to reduce the initial bundle size of your application. +* Dynamically import modules containing Stripe-specific logic only when they are needed. +* Use code splitting features provided by your bundler (e.g., Webpack, Parcel) to separate Stripe code into its own chunk. + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns + +* **Facade:** Create a facade to simplify the Stripe API and provide a higher-level interface for your application. +* **Adapter:** Use an adapter to map data between your application's data model and Stripe's data model. +* **Strategy:** Implement a strategy pattern to handle different payment methods. +* **Observer:** Implement an observer pattern with webhooks. When certain events occur in Stripe, your application gets notified by the Observer (Stripe's Webhooks). + +### 2.2 Recommended Approaches for Common Tasks + +* **Creating a Charge:** Use Payment Intents for new integrations, and migrate from Charges if possible. Payment Intents handle SCA and other authentication requirements more gracefully. +* **Handling Webhooks:** Implement robust webhook handling with retries and proper error handling. +* **Managing Subscriptions:** Use Stripe's Subscription API to manage recurring payments. +* **Storing Card Details:** Use Stripe Elements to securely collect and tokenize card details. +* **Handling Errors:** Implement comprehensive error handling to gracefully handle API errors. + +### 2.3 Anti-patterns and Code Smells + +* **Storing API Keys in Client-Side Code:** Never expose your secret API keys in client-side code. All server side interaction only. +* **Directly Accessing the Stripe API from UI Components:** Introduce a service layer to decouple UI components from Stripe API calls. +* **Ignoring Webhook Events:** Always handle webhook events to ensure that your application is synchronized with Stripe's state. +* **Not Validating Input:** Always validate user input to prevent security vulnerabilities. +* **Creating CHARGES directly:** Prefer PaymentIntents which abstract away much of the complexity and make the code future-proof. + +### 2.4 State Management + +* Use a state management library (e.g., Redux, Zustand, Vuex) to manage Stripe-related state in UI applications. +* Store only the necessary data in the state to minimize memory usage. +* Use immutable data structures to simplify state updates and prevent unexpected side effects. + +### 2.5 Error Handling + +* Implement comprehensive error handling to gracefully handle API errors. +* Log errors to a central location for monitoring and debugging. +* Provide informative error messages to the user. +* Implement retry logic for transient errors. +* Handle specific Stripe errors (e.g., card declined, invalid API key) appropriately. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +* Cache frequently accessed data to reduce API calls. +* Use Stripe's pagination features to retrieve large datasets in smaller chunks. +* Optimize database queries to improve performance. +* Minimize the number of API calls required to complete a task. +* Use webhooks to avoid polling the Stripe API. + +### 3.2 Memory Management + +* Avoid storing large amounts of Stripe data in memory. +* Use pagination to retrieve large datasets in smaller chunks. +* Release resources when they are no longer needed. + +### 3.3 Rendering Optimization (If Applicable) + +* Optimize UI components to minimize rendering time. +* Use techniques such as virtualization and memoization to improve rendering performance. +* Avoid re-rendering components unnecessarily. + +### 3.4 Bundle Size Optimization + +* Use code splitting to reduce the initial bundle size of your application. +* Remove unused code from your Stripe integration. +* Use a minifier to reduce the size of your code. + +### 3.5 Lazy Loading + +* Lazy-load Stripe-related code to reduce the initial bundle size of your application. +* Dynamically import modules containing Stripe-specific logic only when they are needed. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and Prevention + +* **Cross-Site Scripting (XSS):** Sanitize user input to prevent XSS attacks. Use Stripe Elements to securely collect card details. +* **Cross-Site Request Forgery (CSRF):** Protect against CSRF attacks by using anti-CSRF tokens. +* **SQL Injection:** Use parameterized queries to prevent SQL injection attacks. +* **API Key Exposure:** Never expose your secret API keys in client-side code. Store API Keys as environment variables on a secure server. +* **Man-in-the-Middle Attacks:** Enforce HTTPS to protect against man-in-the-middle attacks. + +### 4.2 Input Validation + +* Validate all user input to prevent security vulnerabilities. +* Use strong validation rules to ensure that data is valid. +* Sanitize user input to remove malicious characters. + +### 4.3 Authentication and Authorization + +* Use Stripe's API keys to authenticate requests. +* Use restricted API keys for granular permissions. +* Implement role-based access control to restrict access to sensitive data. + +### 4.4 Data Protection + +* Use Stripe Elements to securely collect and tokenize card details. +* Encrypt sensitive data at rest and in transit. +* Follow PCI DSS compliance guidelines. +* Regularly review your security practices to mitigate risks. +* Implement tokenization to protect sensitive payment information. + +### 4.5 Secure API Communication + +* Enforce HTTPS for all API communication. +* Use TLS 1.2 or higher. +* Verify the authenticity of Stripe's servers using SSL/TLS certificates. + +## 5. Testing Approaches + +### 5.1 Unit Testing + +* Write unit tests for individual components and functions in your Stripe integration. +* Use mocking and stubbing to isolate components from external dependencies. +* Test edge cases and error conditions. + +### 5.2 Integration Testing + +* Write integration tests to verify that different parts of your Stripe integration work together correctly. +* Test the interaction between your application and the Stripe API. +* Use a test Stripe account to avoid affecting live data. + +### 5.3 End-to-End Testing + +* Write end-to-end tests to verify that your Stripe integration works correctly from the user's perspective. +* Use a testing framework such as Cypress or Selenium to automate end-to-end tests. + +### 5.4 Test Organization + +* Organize your tests into logical suites based on functionality. +* Use clear and consistent naming conventions for tests. +* Run tests automatically as part of your CI/CD pipeline. + +### 5.5 Mocking and Stubbing + +* Use mocking and stubbing to isolate components from external dependencies. +* Mock the Stripe API to simulate different scenarios and error conditions. +* Use a mocking library such as Jest or Sinon.js. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes + +* **Not Handling Webhooks:** Failing to handle webhook events can lead to inconsistent data and missed events. +* **Storing API Keys in Client-Side Code:** Exposing your secret API keys in client-side code can compromise your account. +* **Not Validating Input:** Failing to validate user input can lead to security vulnerabilities. +* **Using Test Data in Production:** Using test data in a live environment can lead to incorrect transactions and data corruption. +* **Ignoring Error Messages:** Treat stripe error messages as a core part of the functionality. Do not display generic error messages. + +### 6.2 Edge Cases + +* **Idempotency:** Handle requests carefully by using idempotency keys. These keys ensure that accidental duplicate requests do not result in unexpected behavior, like multiple charges for the same transaction. +* **Network Errors:** Handle network errors gracefully by implementing retry logic. +* **Rate Limiting:** Be aware of Stripe's rate limits and implement a backoff strategy. +* **Concurrency:** Handle concurrent requests carefully to avoid race conditions. +* **SCA (Strong Customer Authentication):** Handle SCA requirements for European customers. + +### 6.3 Version-Specific Issues + +* Be aware of breaking changes in new versions of the Stripe API. +* Test your integration thoroughly after upgrading to a new version of the Stripe API. +* Use Stripe's API versioning feature to maintain compatibility with older versions of the API. + +### 6.4 Compatibility Concerns + +* Ensure that your Stripe integration is compatible with all supported browsers and devices. +* Test your integration on different platforms to identify compatibility issues. + +### 6.5 Debugging Strategies + +* Use Stripe's logging features to track API requests and responses. +* Use a debugger to step through your code and identify errors. +* Use Stripe's API explorer to test API requests. +* Consult Stripe's documentation and support resources. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +* **IDE:** Visual Studio Code, IntelliJ IDEA, or similar. +* **Testing Framework:** Jest, Mocha, or similar. +* **Mocking Library:** Jest, Sinon.js, or similar. +* **Bundler:** Webpack, Parcel, or similar. +* **Linting:** ESLint, Pylint, or similar. +* **Formatter:** Prettier, Black, or similar. +* **Stripe CLI:** Stripe Command Line Interface is incredibly valuable. + +### 7.2 Build Configuration + +* Use a build tool to automate the build process. +* Configure your build tool to minify and optimize your code. +* Use environment variables to store configuration settings. + +### 7.3 Linting and Formatting + +* Use a linter to enforce code style and identify potential errors. +* Use a formatter to automatically format your code. +* Configure your editor to automatically lint and format your code on save. + +### 7.4 Deployment + +* Deploy your application to a secure server. +* Use HTTPS to encrypt communication between your application and the server. +* Monitor your application for errors and performance issues. + +### 7.5 CI/CD Integration + +* Use a CI/CD pipeline to automate the build, test, and deployment process. +* Run tests automatically as part of your CI/CD pipeline. +* Use a deployment tool such as Docker to package and deploy your application. \ No newline at end of file diff --git a/.cursor/rules/supabase.mdc b/.cursor/rules/supabase.mdc new file mode 100644 index 000000000..1f969596e --- /dev/null +++ b/.cursor/rules/supabase.mdc @@ -0,0 +1,283 @@ +--- +description: Comprehensive best practices for developing with Supabase, covering code organization, performance, security, testing, and common pitfalls. This rule provides actionable guidance for developers to build robust and scalable applications using Supabase. +globs: *.js,*.jsx,*.ts,*.tsx,*.sql +--- +# Supabase Best Practices: A Comprehensive Guide + +This document outlines best practices for developing with Supabase, covering various aspects from code organization to security and performance. + +## 1. Code Organization and Structure + +A well-organized codebase is crucial for maintainability and scalability. Here's how to structure your Supabase project: + +### 1.1. Directory Structure Best Practices + +Adopt a modular structure that separates concerns: + + +project-root/ +├── src/ +│ ├── components/ # Reusable UI components +│ ├── pages/ # Page-level components (routes) +│ ├── services/ # Supabase client initialization and data fetching +│ │ ├── supabase.ts # Supabase client initialization +│ │ ├── auth.ts # Authentication-related functions +│ │ ├── database.ts # Database interaction functions +│ ├── utils/ # Utility functions (e.g., date formatting) +│ ├── types/ # TypeScript types and interfaces +│ ├── hooks/ # Custom React hooks +│ ├── styles/ # Global styles and theme +│ └── App.tsx # Main application component +├── migrations/ # Database migrations +├── tests/ # Unit and integration tests +├── .env # Environment variables +└── package.json # Project dependencies + + +### 1.2. File Naming Conventions + +* **Components:** Use PascalCase (e.g., `UserProfile.tsx`). +* **Functions:** Use camelCase (e.g., `fetchUserData`). +* **Variables:** Use camelCase (e.g., `userName`). +* **Files:** Use kebab-case (e.g., `user-profile.tsx`). + +### 1.3. Module Organization + +* Group related functionalities into modules. +* Use clear and descriptive module names. +* Export only what is necessary from each module to minimize the API surface. + +### 1.4. Component Architecture + +* Favor small, reusable components. +* Use a component-based approach (e.g., React, Vue.js) to build UIs. +* Separate presentational components from container components to improve reusability and testability. + +### 1.5. Code Splitting Strategies + +* Use dynamic imports to lazy-load modules. +* Split large components into smaller chunks. +* Implement route-based code splitting to load only the necessary code for each route. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns Specific to Supabase + +* **Repository Pattern:** Abstract database interactions behind a repository interface for better testability and maintainability. +* **Observer Pattern:** Utilize Supabase's real-time capabilities to implement reactive UIs. + +### 2.2. Recommended Approaches for Common Tasks + +* **Data Fetching:** Create reusable functions for fetching data from Supabase tables. +* **Authentication:** Use Supabase Auth for user authentication and authorization. +* **Real-time Updates:** Leverage Supabase Realtime for real-time data synchronization. +* **File Storage:** Utilize Supabase Storage for managing file uploads and downloads. + +### 2.3. Anti-patterns and Code Smells to Avoid + +* **Direct Database Access from UI Components:** This tightly couples the UI to the database and makes testing difficult. Use a service layer to abstract database interactions. +* **Overusing Supabase Functions and Policies:** Keep business logic in your application code whenever possible to avoid vendor lock-in and improve testability. +* **Manually Creating Tables Without an ORM:** Always use an ORM to manage your database schema and migrations. +* **Ignoring Error Handling:** Implement proper error handling to prevent unexpected crashes and provide informative error messages to users. +* **Storing Sensitive Data in Plain Text:** Never store sensitive data like passwords or API keys in plain text. Use encryption and secure storage mechanisms. + +### 2.4. State Management Best Practices + +* Choose a state management library (e.g., Redux, Zustand, Recoil) based on your project's complexity. +* Use local component state for simple UI state. +* Centralize application state in a global store for complex data dependencies. +* Use asynchronous actions to handle data fetching and updates. + +### 2.5. Error Handling Patterns + +* Use try-catch blocks to handle exceptions. +* Implement a global error handler to catch unhandled exceptions. +* Log errors to a monitoring service for tracking and analysis. +* Display user-friendly error messages to the user. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +* **Indexing:** Add indexes to frequently queried columns to improve query performance. +* **Query Optimization:** Use efficient SQL queries and avoid N+1 query problems. +* **Caching:** Implement caching strategies to reduce database load. +* **Connection Pooling:** Use connection pooling to reuse database connections and reduce overhead. + +### 3.2. Memory Management + +* Avoid memory leaks by properly cleaning up resources. +* Use garbage collection to reclaim unused memory. +* Optimize data structures to reduce memory usage. + +### 3.3. Rendering Optimization + +* Use memoization techniques to prevent unnecessary re-renders. +* Virtualize long lists to improve rendering performance. +* Optimize images and other assets to reduce file sizes. + +### 3.4. Bundle Size Optimization + +* Use tree shaking to remove unused code. +* Minify code to reduce file sizes. +* Compress assets to reduce transfer times. + +### 3.5. Lazy Loading Strategies + +* Lazy load images and other assets that are not immediately visible. +* Implement infinite scrolling to load data in chunks as the user scrolls. +* Use code splitting to load only the necessary code for each route. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities and How to Prevent Them + +* **SQL Injection:** Prevent SQL injection by using parameterized queries and prepared statements. +* **Cross-Site Scripting (XSS):** Sanitize user input to prevent XSS attacks. +* **Cross-Site Request Forgery (CSRF):** Implement CSRF protection to prevent unauthorized requests. +* **Authentication and Authorization Issues:** Secure your authentication and authorization mechanisms to prevent unauthorized access. + +### 4.2. Input Validation + +* Validate all user input to prevent malicious data from entering your system. +* Use server-side validation in addition to client-side validation. +* Sanitize user input to remove potentially harmful characters. + +### 4.3. Authentication and Authorization Patterns + +* Use Supabase Auth for user authentication and authorization. +* Implement role-based access control (RBAC) to manage user permissions. +* Use row-level security (RLS) to control data access at the row level. + +### 4.4. Data Protection Strategies + +* Encrypt sensitive data at rest and in transit. +* Use secure storage mechanisms to store API keys and other secrets. +* Implement data masking to protect sensitive data from unauthorized access. + +### 4.5. Secure API Communication + +* Use HTTPS to encrypt API traffic. +* Implement API rate limiting to prevent abuse. +* Validate API requests to prevent malicious input. + +## 5. Testing Approaches + +### 5.1. Unit Testing Strategies + +* Write unit tests for individual functions and components. +* Use mocking and stubbing to isolate units of code. +* Aim for high code coverage. + +### 5.2. Integration Testing + +* Write integration tests to verify the interaction between different parts of your system. +* Test the integration between your application and Supabase. + +### 5.3. End-to-end Testing + +* Write end-to-end tests to simulate user interactions and verify the overall functionality of your application. +* Use tools like Cypress or Playwright to automate end-to-end tests. + +### 5.4. Test Organization + +* Organize tests into separate directories based on functionality. +* Use descriptive test names. +* Keep tests concise and focused. + +### 5.5. Mocking and Stubbing + +* Use mocking to replace external dependencies with controlled substitutes. +* Use stubbing to replace specific method calls with predefined values. +* Avoid over-mocking, as it can make tests less reliable. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes Developers Make + +* Not using an ORM for database schema management. +* Over-relying on Supabase functions and policies for business logic. +* Using Supabase-only features without considering open-source alternatives. +* Ignoring error handling and security best practices. + +### 6.2. Edge Cases to Be Aware Of + +* Handling large datasets efficiently. +* Dealing with real-time data synchronization conflicts. +* Managing user sessions and authentication tokens securely. + +### 6.3. Version-Specific Issues + +* Be aware of breaking changes in Supabase and its dependencies. +* Test your application thoroughly after upgrading Supabase or its dependencies. +* Refer to the Supabase documentation for migration guides. + +### 6.4. Compatibility Concerns + +* Ensure compatibility between your application and the Supabase client library. +* Test your application on different browsers and devices. + +### 6.5. Debugging Strategies + +* Use browser developer tools to debug client-side code. +* Use server-side logging to track errors and performance issues. +* Use the Supabase dashboard to monitor database activity. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +* Supabase CLI: For local development and database management. +* VS Code: For code editing and debugging. +* Drizzle ORM: For database schema management. +* Postman/Insomnia: For testing API endpoints. + +### 7.2. Build Configuration + +* Use a build tool like Webpack or Parcel to bundle your code. +* Configure your build tool to optimize for production. +* Use environment variables to configure your application. + +### 7.3. Linting and Formatting + +* Use ESLint to enforce code style and prevent errors. +* Use Prettier to format your code automatically. +* Integrate linting and formatting into your development workflow. + +### 7.4. Deployment Best Practices + +* Use a CI/CD pipeline to automate deployments. +* Deploy your application to a production-ready environment. +* Monitor your application for errors and performance issues. + +### 7.5. CI/CD Integration + +* Use a CI/CD tool like GitHub Actions or GitLab CI to automate your build, test, and deployment processes. +* Configure your CI/CD pipeline to run tests and linting before deployment. +* Use environment variables to configure your application in the CI/CD environment. + +## 8. Database Workflow Design + +* **Avoid Direct Changes in Production**: Once your application is live, refrain from making database changes using the Supabase Dashboard. Instead, utilize migration tools and enforce access control to prevent unauthorized modifications. +* **Multiple Environments**: Adopt a multi-stage development workflow (`local -> staging -> prod`). This approach allows for thorough testing and validation at each stage before changes are deployed to production. +* **Point-in-Time Recovery**: As your database grows, consider moving to Point-in-Time Recovery (PITR) to minimize the impact on database performance during maintenance and ensure data safety. +* **Database Migrations**: Use database migration tools to manage schema changes. This practice helps maintain consistency across different environments and simplifies version control. +* **Access Control**: Be mindful of who has access to your production environment. Limit access to experienced team members and set clear internal workflows to mitigate risks. +* **Security**: Regularly review and update your security measures. Ensure that tables with sensitive data have appropriate access levels and that database secrets and API keys are stored securely. +* **Performance Monitoring**: Utilize Supabase's observability tooling to monitor database performance and optimize queries, indexes, and connection management. + +## 9. Additional Best Practices + +* **Understand Shared Responsibilities:** When using Supabase, be aware of the shared responsibilities model. Supabase manages infrastructure, but you are responsible for application architecture, security, and data management. +* **Proper Indexing:** Essential for query optimization. Indices should be created based on common query patterns to reduce search time. +* **Load Testing**: Before deploying changes to production, perform load testing in a staging environment. Tools such as `k6` can simulate traffic and help identify potential bottlenecks. +* **Resource Upgrades**: Monitor resource usage and upgrade your database when necessary. For significant traffic surges, contact support in advance for assistance. +* **Database Optimization**: Regularly optimize your database by adding filters on large queries, using caching strategies, and managing connections efficiently. +* **Regular Backups:** Schedule regular backups of your database to protect against data loss. +* **Use of Postgres Features**: As Supabase is built around Postgres, understand and leverage Postgres features for performance and scalability. + +## 10. Conclusion + +By following these best practices, you can build robust, scalable, and secure applications with Supabase. Remember to stay up-to-date with the latest Supabase documentation and community resources to continuously improve your development practices. + +@file ./supabase_code_examples.mdc \ No newline at end of file diff --git a/.cursor/rules/svelte.mdc b/.cursor/rules/svelte.mdc new file mode 100644 index 000000000..e22ba75ad --- /dev/null +++ b/.cursor/rules/svelte.mdc @@ -0,0 +1,157 @@ +--- +description: Comprehensive Svelte best practices covering code structure, performance, security, testing, and common pitfalls. This rule provides guidance on writing maintainable, efficient, and secure Svelte applications. +globs: *.svelte +--- +- **Manipulate the DOM with HTML:** Utilize Svelte's HTML-centric approach to manipulate the DOM. +- **Solve challenges with HTML/CSS first:** Prioritize solving problems with HTML and CSS before resorting to JavaScript. +- **Write short components:** Aim for concise and focused components to improve readability and maintainability. +- **Write concise reactive `$:` blocks:** Keep reactive statements short and easy to understand. +- **Scope CSS to the actual component:** Ensure that CSS styles are scoped to the component to avoid conflicts. +- **Avoid hard coding data or dimensions:** Use dynamic data and responsive design techniques instead of hard-coded values. +- **Keep objects immutable:** Treat objects as immutable to prevent unexpected side effects and improve performance. +- **Use two-way binding effectively:** Leverage two-way binding for efficient data synchronization but be mindful of its potential impact on performance for complex scenarios. + +## 1. Code Organization and Structure: + + - **Directory Structure Best Practices:** + - `src/`: Contains all the application source code. + - `src/components/`: Holds reusable Svelte components, categorized further by feature or domain (e.g., `src/components/Button/`, `src/components/Form/`). + - `src/lib/`: Contains utility functions, helper modules, and reusable logic that is not specific to a component. + - `src/routes/`: For SvelteKit applications, this directory defines the application's routes. + - `src/stores/`: Stores for global state management (if using Svelte's stores). + - `static/`: Static assets like images, fonts, and other resources. + - **File Naming Conventions:** + - Components: Use PascalCase (e.g., `MyComponent.svelte`). + - Utility functions: Use camelCase (e.g., `formatDate.js`). + - Stores: Use descriptive names related to the data they hold (e.g., `userStore.js`). + - **Module Organization:** + - Group related components and utilities into modules within their respective directories. + - Use `index.js` or `index.svelte` files to export multiple components or functions from a directory, providing a cleaner import experience. + - **Component Architecture:** + - Favor a component-based architecture where UI is broken down into small, reusable components. + - Consider using a composition pattern where complex components are built by composing simpler ones. + - Separate concerns: Keep components focused on presentation logic and delegate data fetching or business logic to services or stores. + - **Code Splitting Strategies:** + - Use dynamic imports (`import()`) to load components or modules on demand, reducing the initial bundle size. + - Leverage SvelteKit's built-in code splitting capabilities for route-based splitting. + - Consider splitting large components into smaller, lazily loaded sub-components. + +## 2. Common Patterns and Anti-patterns: + + - **Design Patterns Specific to Svelte:** + - **Store pattern:** Use Svelte's stores for managing application state and reactivity. + - **Action pattern:** Use Svelte's actions for manipulating DOM elements or integrating with third-party libraries. + - **Component composition:** Build complex UIs by composing smaller, reusable components. + - **Recommended Approaches for Common Tasks:** + - **Form handling:** Use Svelte's two-way binding (`bind:value`) for simple forms. For more complex scenarios, consider libraries like Formik or Svelte Formly. + - **Data fetching:** Use `fetch` or libraries like Axios for fetching data from APIs. Handle loading and error states appropriately. + - **Event handling:** Use Svelte's event directives (e.g., `on:click`) for handling DOM events. + - **Anti-patterns and Code Smells to Avoid:** + - **Overusing global state:** Avoid putting everything in a global store. Use component-level state when appropriate. + - **Directly manipulating the DOM outside of actions:** Rely on Svelte's reactivity system to update the DOM. + - **Writing overly complex components:** Break down large components into smaller, more manageable ones. + - **State Management Best Practices:** + - Use Svelte's built-in stores for managing global or application-level state. + - Consider using reactive declarations (`$:`) to derive state from other state variables. + - Keep state updates predictable and avoid modifying state directly. + - For complex state management needs, explore libraries like Redux or Zustand (though often unnecessary in Svelte). + - **Error Handling Patterns:** + - Use try-catch blocks to handle potential errors during data fetching or other asynchronous operations. + - Display user-friendly error messages in the UI. + - Log errors to the console or a logging service for debugging purposes. + - Implement global error handling to catch unhandled exceptions. + +## 3. Performance Considerations: + + - **Optimization Techniques:** + - **Minimize DOM updates:** Svelte is very efficient at updating the DOM, but unnecessary updates can still impact performance. Use reactive declarations judiciously. + - **Use `{#each}` blocks efficiently:** Key your `{#each}` blocks with unique identifiers to help Svelte efficiently update the list. + - **Avoid unnecessary component re-renders:** Use the `$:` syntax effectively to only update components when necessary. + - **Memory Management:** + - Avoid memory leaks by properly cleaning up event listeners and subscriptions when components are destroyed (using `onDestroy`). + - Be mindful of large data structures in stores, and consider using techniques like pagination or virtualization to manage them efficiently. + - **Rendering Optimization:** + - Use the `svelte:options` tag with the `immutable` option for components that receive immutable data as props. + - Use `shouldUpdate` to prevent rendering for unchanged immutable props. + - **Bundle Size Optimization:** + - Use production builds with minification and tree shaking to remove unused code. + - Use dynamic imports for code splitting. + - Optimize images and other assets. + - **Lazy Loading Strategies:** + - Use dynamic imports to lazy load components or modules that are not immediately needed. + - Implement lazy loading for images or other resources that are below the fold. + +## 4. Security Best Practices: + + - **Common Vulnerabilities and How to Prevent Them:** + - **Cross-Site Scripting (XSS):** Sanitize user input before displaying it in the UI. Svelte automatically escapes HTML entities, but be careful when using `@html` or `{@debug}`. + - **Cross-Site Request Forgery (CSRF):** Use CSRF tokens to protect against CSRF attacks. + - **SQL Injection:** If interacting with a database, use parameterized queries to prevent SQL injection attacks. + - **Input Validation:** + - Validate user input on both the client-side and server-side. + - Use appropriate data types and validation rules to prevent invalid data from being processed. + - **Authentication and Authorization Patterns:** + - Use secure authentication mechanisms like OAuth or JWT. + - Implement proper authorization checks to ensure that users only have access to the resources they are allowed to access. + - **Data Protection Strategies:** + - Encrypt sensitive data at rest and in transit. + - Use secure storage mechanisms for storing sensitive data. + - **Secure API Communication:** + - Use HTTPS for all API communication. + - Validate API responses to ensure that they are valid and not malicious. + +## 5. Testing Approaches: + + - **Unit Testing Strategies:** + - Use a testing framework like Jest or Mocha to write unit tests for individual components and utility functions. + - Mock external dependencies to isolate the code being tested. + - **Integration Testing:** + - Use a testing framework like Cypress or Playwright to write integration tests that verify the interaction between different components or modules. + - Test the integration with APIs and other external services. + - **End-to-End Testing:** + - Use a testing framework like Cypress or Playwright to write end-to-end tests that simulate user interactions with the application. + - Test the entire application flow from start to finish. + - **Test Organization:** + - Organize tests into directories that mirror the application's directory structure. + - Use descriptive names for test files and test cases. + - **Mocking and Stubbing:** + - Use mocking libraries like Jest's `jest.fn()` or Sinon.js to mock external dependencies. + - Use stubbing to replace complex or slow dependencies with simpler implementations for testing purposes. + +## 6. Common Pitfalls and Gotchas: + + - **Frequent Mistakes Developers Make:** + - **Incorrect reactivity usage:** Failing to understand how Svelte's reactivity system works can lead to unexpected behavior. + - **Over-reliance on `$: `:** Although reactive declarations are powerful, overusing them can make code harder to read and debug. Consider if a regular variable or function would be more appropriate. + - **Mutating props directly:** Modifying props directly inside a component can lead to unexpected side effects. + - **Edge Cases to Be Aware Of:** + - **Transition and animation behavior:** Understanding how transitions and animations interact with component updates is important for creating smooth user experiences. + - **Server-side rendering (SSR) considerations:** When using SvelteKit, be aware of the differences between client-side and server-side execution environments. + - **Version-Specific Issues:** + - Consult the Svelte changelog and migration guides when upgrading to a new version to avoid compatibility issues. + - **Compatibility Concerns:** + - Ensure that your code is compatible with the target browsers and devices. + - Use polyfills or transpilers to support older browsers if necessary. + - **Debugging Strategies:** + - Use the Svelte Devtools browser extension for inspecting component state and reactivity. + - Use `console.log` or the `debugger` statement to debug code execution. + - Use SvelteKit's built-in error handling and logging features. + +## 7. Tooling and Environment: + + - **Recommended Development Tools:** + - **VS Code with the Svelte extension:** Provides syntax highlighting, code completion, and other useful features. + - **Svelte Devtools:** A browser extension for inspecting Svelte component state and reactivity. + - **ESLint and Prettier:** For linting and formatting code. + - **Build Configuration:** + - Use a build tool like Vite or Rollup to bundle your Svelte code for production. + - Configure the build tool to perform optimizations like minification and tree shaking. + - **Linting and Formatting:** + - Use ESLint with the `eslint-plugin-svelte3` plugin to lint your Svelte code. + - Use Prettier with the `prettier-plugin-svelte` plugin to format your Svelte code. + - **Deployment Best Practices:** + - Deploy your Svelte application to a CDN or a serverless platform like Netlify or Vercel. + - Configure your server to serve the application's static assets with proper caching headers. + - **CI/CD Integration:** + - Use a CI/CD platform like GitHub Actions or GitLab CI to automate the build, testing, and deployment process. + - Run tests and linting checks on every commit to ensure code quality. \ No newline at end of file diff --git a/.cursor/rules/sveltekit.mdc b/.cursor/rules/sveltekit.mdc new file mode 100644 index 000000000..b0d05ee63 --- /dev/null +++ b/.cursor/rules/sveltekit.mdc @@ -0,0 +1,306 @@ +--- +description: This rule provides comprehensive best practices and coding standards for SvelteKit development, covering code structure, performance, security, testing, and common pitfalls. It aims to improve code quality, maintainability, and overall project health. +globs: *.svelte +--- +# SvelteKit Best Practices + +This guide provides comprehensive best practices for developing SvelteKit applications. It covers various aspects, including code organization, performance considerations, security measures, and common pitfalls to avoid. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure Best Practices + +* **`src/lib`**: Reusable components, utilities, and stores. Organize by feature or domain. +* **`src/routes`**: Defines the application's routes. Each directory represents a route segment. +* **`src/routes/+page.svelte`**: Represents a page component for a route. +* **`src/routes/+layout.svelte`**: Layout component that wraps the page. Useful for consistent UI elements. +* **`src/routes/+page.server.ts`**: Server-side code for handling data fetching and form submissions for a page. +* **`src/routes/+layout.server.ts`**: Server-side code for layout data, loaded before any +page.server.ts data. +* **`src/hooks.server.ts`**: Handle server-side requests and responses. Useful for authentication and session management. +* **`static`**: Static assets like images, fonts, and other files served directly. +* **`tests`**: Directory for all tests, mirroring the source structure for clarity. + +Example: + + +src/ +├── lib/ +│ ├── components/ +│ │ ├── Button.svelte +│ │ └── Card.svelte +│ ├── utils/ +│ │ ├── api.ts +│ │ └── helpers.ts +│ └── stores/ +│ └── user.ts +├── routes/ +│ ├── + +│ │ └── page.svelte +│ ├── about/ +│ │ ├── + +│ │ │ └── page.svelte +│ │ └── +page.server.ts +│ └── blog/ +│ ├── [slug]/ +│ │ ├── + +│ │ │ └── page.svelte +│ │ └── +page.server.ts +│ └── + +│ └── page.svelte +├── hooks.server.ts +└── app.d.ts + + +### 1.2. File Naming Conventions + +* **Components**: PascalCase (e.g., `MyComponent.svelte`). +* **Utility functions**: camelCase (e.g., `formatDate.ts`). +* **Stores**: camelCase (e.g., `userStore.ts`). +* **Route files**: Follow SvelteKit's routing conventions (`+page.svelte`, `+layout.svelte`, etc.). + +### 1.3. Module Organization + +* Group related functionality into modules. +* Use `src/lib` for reusable modules. +* Employ clear and descriptive module names. +* Leverage SvelteKit's `$lib` alias for importing modules from `src/lib`. + +### 1.4. Component Architecture + +* **Atomic Design**: Break down UI into small, reusable components (atoms, molecules, organisms, templates, pages). +* **Component Composition**: Compose complex UIs from simpler components. +* **Props and Events**: Use props for data input and events for component output. +* **Slots**: Allow parent components to inject content into child components. + +### 1.5. Code Splitting Strategies + +* **Dynamic Imports**: Use dynamic imports (`import()`) to load modules on demand. +* **Route-Based Splitting**: SvelteKit automatically splits code based on routes. +* **Component-Level Splitting**: Split large components into smaller chunks that can be loaded independently. +* **Lazy Loading**: Load non-critical components or modules only when needed. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns Specific to SvelteKit + +* **Form Actions**: Use SvelteKit's form actions for handling form submissions on the server. +* **Load Functions**: Utilize `load` functions in `+page.server.ts` and `+layout.server.ts` for data fetching. +* **Stores for State Management**: Employ Svelte stores for managing application state. +* **Server-Side Rendering (SSR)**: Leverage SSR for improved SEO and initial load performance. +* **Progressive Enhancement**: Design applications to work even with JavaScript disabled. + +### 2.2. Recommended Approaches for Common Tasks + +* **Data Fetching**: Use `fetch` API or libraries like `axios` within `load` functions. +* **Form Handling**: Use SvelteKit's form actions for server-side validation and processing. +* **Authentication**: Implement authentication using libraries like `lucia-auth` or integrate with OAuth providers. +* **Authorization**: Implement role-based access control using stores or server-side checks. +* **Error Handling**: Use `try...catch` blocks and SvelteKit's `handleError` hook for global error handling. + +### 2.3. Anti-patterns and Code Smells to Avoid + +* **Over-reliance on Global State**: Avoid using global stores for component-specific state. +* **Direct DOM Manipulation**: Avoid directly manipulating the DOM; use Svelte's reactivity system instead. +* **Large, Unmaintainable Components**: Break down large components into smaller, reusable ones. +* **Ignoring Accessibility**: Ensure components are accessible to users with disabilities (ARIA attributes, semantic HTML). +* **Unnecessary Re-renders**: Optimize components to avoid unnecessary re-renders. + +### 2.4. State Management Best Practices + +* **Use Svelte Stores**: Svelte stores are the recommended way to manage application state. +* **Divide stores according to feature**: Split stores based on logical domain or feature (e.g. UserStore, CartStore). +* **Readable and Writable Stores**: Use readable stores for derived state and writable stores for mutable state. +* **Custom Stores**: Create custom stores for complex state management logic. + +### 2.5. Error Handling Patterns + +* **`try...catch` Blocks**: Use `try...catch` blocks to handle errors within components and functions. +* **`handleError` Hook**: Implement the `handleError` hook in `src/hooks.server.ts` for global error handling. +* **Error Boundaries**: Use error boundaries to catch errors in component trees and display fallback UI. +* **Logging**: Log errors to a server for monitoring and debugging. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +* **Lazy Loading**: Load non-critical resources (images, components) only when needed. +* **Code Splitting**: Split code into smaller chunks to reduce initial load time. +* **Image Optimization**: Optimize images using tools like `squoosh` or `tinypng`. +* **Server-Side Rendering (SSR)**: Use SSR for improved initial load performance and SEO. +* **Caching**: Implement caching strategies for data and assets. +* **Preload critical assets**: use `` to fetch critical assets earlier + +### 3.2. Memory Management + +* **Avoid Memory Leaks**: Properly clean up resources when components unmount. +* **Use WeakRefs**: Use `WeakRefs` for managing objects that might be garbage collected. +* **Minimize Object Creation**: Avoid creating unnecessary objects and arrays. +* **Properly unsubscrive from stores:** Properly unsubscribe to avoid re-renders when the component is unmounted. + +### 3.3. Rendering Optimization + +* **Tracked values**: Use the `$:` syntax to track dependencies of reactive statements. +* **Svelte Compiler**: Utilize Svelte's compiler to optimize component rendering. +* **Virtual DOM**: Svelte doesn't use a virtual DOM, so updates are generally very efficient. +* **Avoid Unnecessary Re-renders**: Use `{#each}` blocks with keys to optimize list rendering. +* **Debouncing and Throttling**: Use debouncing and throttling to limit the frequency of updates. + +### 3.4. Bundle Size Optimization + +* **Tree Shaking**: Ensure unused code is removed during the build process. +* **Minification**: Minify JavaScript and CSS files to reduce bundle size. +* **Compression**: Compress assets using gzip or Brotli. +* **Dependency Analysis**: Analyze dependencies to identify and remove unused libraries. + +### 3.5. Lazy Loading Strategies + +* **Intersection Observer**: Use the Intersection Observer API to detect when elements are visible and load them lazily. +* **Dynamic Imports**: Use dynamic imports to load components and modules on demand. +* **Placeholder UI**: Display a placeholder UI while resources are loading. + +## 4. Security Best Practices + +### 4.1. Common Vulnerabilities and How to Prevent Them + +* **Cross-Site Scripting (XSS)**: Sanitize user input to prevent XSS attacks. +* **Cross-Site Request Forgery (CSRF)**: Use CSRF tokens to protect against CSRF attacks. +* **SQL Injection**: Use parameterized queries or ORMs to prevent SQL injection. +* **Authentication and Authorization**: Implement secure authentication and authorization mechanisms. +* **Insecure Direct Object References (IDOR)**: Implement proper access controls to prevent unauthorized access to resources. + +### 4.2. Input Validation + +* **Server-Side Validation**: Always validate user input on the server. +* **Client-Side Validation**: Use client-side validation for immediate feedback, but never rely on it solely. +* **Escape User Input**: Escape user input before rendering it in the DOM. +* **Use Validation Libraries**: Use libraries like `zod` or `yup` for input validation. + +### 4.3. Authentication and Authorization Patterns + +* **JWT (JSON Web Tokens)**: Use JWTs for authentication and authorization. +* **OAuth 2.0**: Integrate with OAuth providers like Google and Facebook for social login. +* **Role-Based Access Control (RBAC)**: Implement RBAC to control access to resources based on user roles. +* **Multi-Factor Authentication (MFA)**: Implement MFA for enhanced security. + +### 4.4. Data Protection Strategies + +* **Encryption**: Encrypt sensitive data at rest and in transit. +* **Hashing**: Hash passwords using strong hashing algorithms like bcrypt or Argon2. +* **Data Masking**: Mask sensitive data in logs and reports. +* **Regular Security Audits**: Conduct regular security audits to identify and address vulnerabilities. + +### 4.5. Secure API Communication + +* **HTTPS**: Use HTTPS for all API communication. +* **API Keys**: Protect API keys and secrets. +* **Rate Limiting**: Implement rate limiting to prevent abuse. +* **Input Validation**: Always validate input to your APIs. +* **Output Encoding**: Always encode output from your API to prevent injection attacks. + +## 5. Testing Approaches + +### 5.1. Unit Testing Strategies + +* **Test Individual Components**: Write unit tests for individual components in isolation. +* **Test Utility Functions**: Write unit tests for utility functions and modules. +* **Use Mock Data**: Use mock data for testing components and functions. +* **Test Edge Cases**: Test edge cases and error conditions. +* **Utilize Svelte Testing Library**: Employ `svelte-testing-library` for testing Svelte components. + +### 5.2. Integration Testing + +* **Test Component Interactions**: Write integration tests to ensure components interact correctly. +* **Test Data Flow**: Test the flow of data between components and modules. +* **Test API Integrations**: Test integrations with APIs and external services. + +### 5.3. End-to-End Testing + +* **Simulate User Interactions**: Write end-to-end tests to simulate user interactions. +* **Test Full Workflows**: Test complete user workflows from start to finish. +* **Use Playwright or Cypress**: Use tools like Playwright or Cypress for end-to-end testing. + +### 5.4. Test Organization + +* **Mirror Source Structure**: Organize tests in a directory structure that mirrors the source code. +* **Use Descriptive Names**: Use descriptive names for test files and functions. +* **Separate Test Environments**: Use separate test environments for unit, integration, and end-to-end tests. + +### 5.5. Mocking and Stubbing + +* **Mock API Calls**: Mock API calls to isolate components during testing. +* **Stub External Dependencies**: Stub external dependencies to control their behavior. +* **Use Mocking Libraries**: Use libraries like `jest.mock` or `sinon` for mocking and stubbing. + +## 6. Common Pitfalls and Gotchas + +### 6.1. Frequent Mistakes Developers Make + +* **Incorrectly Using Reactive Statements**: Not understanding how Svelte's reactivity system works. +* **Ignoring Accessibility**: Neglecting accessibility considerations when building components. +* **Over-complicating State Management**: Using complex state management solutions when simpler ones would suffice. +* **Not Handling Errors Properly**: Ignoring errors or not providing informative error messages. +* **Premature Optimization**: Optimizing code before it is necessary. + +### 6.2. Edge Cases to Be Aware Of + +* **Server-Side Rendering Differences**: Be aware of differences between server-side and client-side rendering. +* **Browser Compatibility**: Test applications in different browsers to ensure compatibility. +* **Network Latency**: Design applications to handle network latency and unreliable connections. +* **Memory Constraints**: Consider memory constraints when building large applications. + +### 6.3. Version-Specific Issues + +* **Breaking Changes**: Be aware of breaking changes in new versions of SvelteKit. +* **Deprecated Features**: Avoid using deprecated features. +* **Upgrade Guides**: Follow upgrade guides when migrating to new versions of SvelteKit. + +### 6.4. Compatibility Concerns + +* **Third-Party Libraries**: Ensure third-party libraries are compatible with SvelteKit. +* **Browser Support**: Check browser support for new features and APIs. +* **Node.js Version**: Ensure the Node.js version is compatible with SvelteKit. + +### 6.5. Debugging Strategies + +* **Browser Developer Tools**: Use browser developer tools for debugging JavaScript and CSS. +* **Svelte Devtools**: Use the Svelte Devtools extension for inspecting Svelte components. +* **Logging**: Use `console.log` statements for debugging. +* **Debuggers**: Use debuggers like VS Code's debugger for step-by-step debugging. + +## 7. Tooling and Environment + +### 7.1. Recommended Development Tools + +* **VS Code**: VS Code is a popular code editor with excellent Svelte and TypeScript support. +* **Svelte Devtools**: The Svelte Devtools browser extension allows you to inspect Svelte components. +* **ESLint**: ESLint is a linter that helps enforce code style and best practices. +* **Prettier**: Prettier is a code formatter that automatically formats code to a consistent style. +* **TypeScript**: Using typescript is recommended for type safety. + +### 7.2. Build Configuration + +* **`svelte.config.js`**: Configure SvelteKit using the `svelte.config.js` file. +* **Vite**: SvelteKit uses Vite as its build tool. +* **Environment Variables**: Use environment variables for configuration settings. +* **Adapters**: Use adapters to deploy SvelteKit applications to different environments. + +### 7.3. Linting and Formatting + +* **ESLint**: Configure ESLint to enforce code style and best practices. +* **Prettier**: Configure Prettier to automatically format code. +* **Husky**: Use Husky to run linters and formatters before committing code. +* **Lint Staged**: Use lint-staged to run linters and formatters only on staged files. + +### 7.4. Deployment Best Practices + +* **Choose an Adapter**: Use an adapter like `@sveltejs/adapter-node` or `@sveltejs/adapter-static` to deploy the application. +* **Configure Environment Variables**: Configure environment variables for production environments. +* **Set Up HTTPS**: Set up HTTPS for secure communication. +* **Monitor the Application**: Monitor the application for errors and performance issues. + +### 7.5. CI/CD Integration + +* **Use a CI/CD Pipeline**: Use a CI/CD pipeline to automate the build, test, and deployment process. +* **Automated Tests**: Run automated tests in the CI/CD pipeline. +* **Automated Deployments**: Automate deployments to production environments. +* **Use tools like GitHub Actions, GitLab CI, or CircleCI** \ No newline at end of file diff --git a/.cursor/rules/tailwind.mdc b/.cursor/rules/tailwind.mdc new file mode 100644 index 000000000..637a9b019 --- /dev/null +++ b/.cursor/rules/tailwind.mdc @@ -0,0 +1,101 @@ +--- +description: Comprehensive guide for Tailwind CSS best practices, covering code organization, performance optimization, security considerations, and common pitfalls. This rule provides actionable guidance for developers to build scalable and maintainable Tailwind CSS projects. +globs: *.html,*.js,*.jsx,*.ts,*.tsx,*.vue +--- +- Leverage Tailwind's PurgeCSS to remove unused styles in production. Configure `purge` in `tailwind.config.js` to specify files to scan for Tailwind classes. Example: + javascript + module.exports = { + purge: ['./src/*.js,*.jsx,*.ts,*.tsx', './public/index.html'], + darkMode: false, // or 'media' or 'class' + theme: { + extend: {}, + }, + variants: { + extend: {}, + }, + plugins: [], + } + +- Use Tailwind's Configuration File (`tailwind.config.js`) to customize the theme, colors, spacing, breakpoints, and other design tokens. This promotes consistency and maintainability across the project. Avoid hardcoding values directly in the HTML or components. +- Adopt a Mobile-First Approach: Design for smaller screens first and then use Tailwind's responsive modifiers (e.g., `md:`, `lg:`) to adapt the layout and styles for larger screens. This improves the user experience on mobile devices and reduces the amount of CSS needed. +- Utilize Tailwind UI or other component libraries built with Tailwind CSS to accelerate development and maintain a consistent design language. Customize the components as needed to fit the specific requirements of the project. Alternatively, create your own reusable components. +- Optimize for Performance: Be mindful of the number of Tailwind classes used on each element. Consider using `@apply` in CSS or extracting common styles into custom CSS classes to reduce duplication and improve performance. Use `content` variants when needed instead of the `DEFAULT` to control when generated classes are added. For instance, use `content-[url(..)]` instead of `content-[url(..)]`. Configure the JIT mode for faster build times during development. +- Stay Organized with Components: Break down the UI into smaller, reusable components. Use a consistent naming convention for components and Tailwind classes. Consider using a component library like Storybook to document and showcase the components. +- Integrate with a Design System: Define a clear design system with consistent colors, typography, and spacing. Map the design system tokens to Tailwind's configuration file. This ensures that the UI is consistent and aligned with the brand guidelines. +- Use semantic class names: While Tailwind promotes utility-first CSS, consider using semantic class names in conjunction with Tailwind classes to improve readability and maintainability. For example, instead of `; + }; + + +- **ESLint Configuration (Example):** + javascript + module.exports = { + env: { + browser: true, + es2021: true, + node: true, + }, + extends: [ + 'eslint:recommended', + 'plugin:react/recommended', + 'plugin:@typescript-eslint/recommended', + 'prettier', + ], + parser: '@typescript-eslint/parser', + parserOptions: { + ecmaFeatures: { + jsx: true, + }, + ecmaVersion: 12, + sourceType: 'module', + }, + plugins: ['react', '@typescript-eslint', 'prettier'], + rules: { + 'prettier/prettier': 'error', + 'react/react-in-jsx-scope': 'off', + '@typescript-eslint/explicit-function-return-type': 'off', + '@typescript-eslint/no-explicit-any': 'warn', + }, + }; + + +- **Conclusion:** + - Following these best practices will help you build efficient, maintainable, and secure applications with Vite. Continuously review and update your practices as the library and ecosystem evolve. \ No newline at end of file diff --git a/.cursor/rules/vitest.mdc b/.cursor/rules/vitest.mdc new file mode 100644 index 000000000..a6990c72a --- /dev/null +++ b/.cursor/rules/vitest.mdc @@ -0,0 +1,363 @@ +--- +description: This rule file provides comprehensive best practices for using Vitest, covering code organization, testing strategies, performance, and security within Vitest projects. These guidelines ensure clean, maintainable, and reliable test suites. +globs: *.js,*.jsx,*.ts,*.tsx,*.vue,*.svelte,*.spec,*.test.* +--- +# Vitest Best Practices and Coding Standards + +This document outlines best practices for using Vitest to create reliable, maintainable, and performant test suites. It covers various aspects of testing, from code organization to performance considerations and security measures. + +## 1. Code Organization and Structure + +### 1.1 Directory Structure + +* **Keep tests close to the source code:** Place test files in the same directory as the components or modules they test. This improves discoverability and maintainability. + + + src/ + components/ + MyComponent.vue + MyComponent.spec.ts + MyComponent.test.ts # Alternative naming + utils/ + math.ts + math.test.ts + App.vue + App.spec.ts + + +* **Use a dedicated `tests` directory for end-to-end tests or shared utilities:** For larger projects, a `tests` directory at the root level can house end-to-end tests, integration tests that require a specific environment, or shared testing utilities. + + + tests/ + e2e/ + specs/ + home.spec.ts + support/ + commands.ts + unit/ + utils.test.ts # Tests for general utilities + integration/ + db.setup.ts # Setup for integration tests + src/ + ... + + +### 1.2 File Naming Conventions + +* **Use consistent naming:** Adopt a consistent naming scheme for test files. Common conventions include: + * `[component/module].spec.ts` + * `[component/module].test.ts` + * `[component/module].e2e.ts` (for end-to-end tests) + +* **Be descriptive:** Name test files to clearly indicate what they are testing. For example, `MyComponent.props.spec.ts` might test specific props of `MyComponent`. + +### 1.3 Module Organization + +* **Group related tests:** Organize tests into modules using `describe` blocks. This improves readability and helps structure test output. + + typescript + import { describe, it, expect } from 'vitest'; + import { add } from './math'; + + describe('Math functions', () => { + describe('add', () => { + it('should add two numbers correctly', () => { + expect(add(2, 3)).toBe(5); + }); + + it('should handle negative numbers', () => { + expect(add(-1, 1)).toBe(0); + }); + }); + }); + + +### 1.4 Component Architecture + +* **Test component logic separately:** Extract complex logic from components into separate, testable functions or modules. This promotes reusability and simplifies component testing. + +* **Focus on component interactions:** When testing components, concentrate on verifying that the component renders correctly given certain props or state and that it emits the correct events in response to user interactions. + +### 1.5 Code Splitting Strategies + +* **Test code splitting:** If your application uses code splitting, ensure your tests cover different code chunks and lazy-loaded modules. + +* **Mock dynamic imports:** Use Vitest's mocking capabilities to simulate dynamic imports during testing. + + typescript + import { describe, it, expect, vi } from 'vitest'; + + describe('Dynamic import', () => { + it('should mock dynamic import', async () => { + const mockModule = { value: 'mocked' }; + vi.mock('./dynamic-module', () => ({ + default: mockModule, + })); + + const dynamicModule = await import('./dynamic-module'); + expect(dynamicModule.default).toBe(mockModule); + }); + }); + + +## 2. Common Patterns and Anti-patterns + +### 2.1 Design Patterns Specific to Vitest + +* **AAA (Arrange, Act, Assert):** Structure each test case following the AAA pattern for clarity and maintainability. + + typescript + it('should add two numbers correctly', () => { + // Arrange + const a = 2; + const b = 3; + + // Act + const result = add(a, b); + + // Assert + expect(result).toBe(5); + }); + + +* **Page Object Model (POM):** For end-to-end tests, use the Page Object Model to abstract away the details of the user interface and make tests more resilient to UI changes. Define dedicated classes representing different pages or components. + +### 2.2 Recommended Approaches for Common Tasks + +* **Mocking external dependencies:** Use Vitest's mocking capabilities to isolate units of code during testing. Use `vi.mock()` to mock modules and `vi.spyOn()` to spy on specific methods or properties. + +* **Testing asynchronous code:** Utilize `async/await` and Vitest's `expect.resolves` and `expect.rejects` matchers for testing asynchronous functions. + + typescript + it('should resolve with the correct value', async () => { + await expect(Promise.resolve(42)).resolves.toBe(42); + }); + + it('should reject with an error', async () => { + await expect(Promise.reject(new Error('Something went wrong'))).rejects.toThrow('Something went wrong'); + }); + + +### 2.3 Anti-patterns and Code Smells to Avoid + +* **Over-mocking:** Avoid mocking everything. Mock only external dependencies or components that are not under test. Testing against mocks rather than real implementations reduces test confidence. + +* **Flaky tests:** Tests that pass or fail intermittently are a sign of underlying issues. Investigate flaky tests to identify and fix the root cause, such as race conditions or reliance on external resources. + +* **Ignoring edge cases:** Ensure your tests cover all possible scenarios, including edge cases, error conditions, and boundary values. Don't only test the "happy path". + +* **Long test functions:** Break down overly complex test functions into smaller, more focused tests. This improves readability and makes it easier to identify the cause of failures. + +### 2.4 State Management Best Practices + +* **Isolate state:** When testing code that relies on state management libraries (e.g., Vuex, Pinia, Redux), isolate the state and actions being tested. + +* **Mock store actions/getters:** Mock actions and getters to control the state during testing and verify that they are called correctly. + + typescript + import { describe, it, expect, vi } from 'vitest'; + import { useStore } from './store'; + + describe('Store actions', () => { + it('should dispatch the correct action', () => { + const store = useStore(); + const mockAction = vi.fn(); + store.dispatch = mockAction; + + store.commit('increment'); + expect(mockAction).toHaveBeenCalledWith('increment'); + }); + }); + + +### 2.5 Error Handling Patterns + +* **Test error handling:** Ensure your tests cover error handling scenarios. Use `try...catch` blocks or `expect.rejects` to verify that errors are thrown and handled correctly. + +* **Mock error responses:** Mock API responses to simulate error conditions and test how your code handles them. + +## 3. Performance Considerations + +### 3.1 Optimization Techniques + +* **Run tests in parallel:** Vitest supports running tests in parallel. Enable this feature to speed up test execution. + + json + // vitest.config.ts + import { defineConfig } from 'vitest/config' + + export default defineConfig({ + test: { + threads: true, // Enable parallel execution + }, + }) + + +* **Use `--changed` and `--related` flags:** When running tests, use the `--changed` flag to only run tests that have changed since the last commit or the `--related` flag to run tests related to specific files. + + bash + vitest --changed + vitest --related src/components/MyComponent.vue + + +* **Optimize test setup:** Minimize the amount of setup required for each test. Use `beforeAll` and `afterAll` hooks to perform setup and teardown operations once for each test suite, rather than for each test case. + +### 3.2 Memory Management + +* **Clean up after tests:** Ensure that your tests do not leak memory. Use `afterEach` hooks to clean up any resources created during the test, such as mocks or temporary files. + +* **Avoid creating large objects in tests:** Minimize the size of objects created in tests to reduce memory consumption. + +### 3.3 Rendering Optimization + +* **Use shallow rendering:** When testing components, use shallow rendering to avoid rendering the entire component tree. This can significantly improve test performance. + + typescript + import { shallowMount } from '@vue/test-utils'; + import MyComponent from './MyComponent.vue'; + + it('should render correctly', () => { + const wrapper = shallowMount(MyComponent); + expect(wrapper.exists()).toBe(true); + }); + + +### 3.4 Bundle Size Optimization + +* **Keep tests small:** Avoid including unnecessary dependencies in your test files. This can help reduce the bundle size of your tests and improve startup time. + +### 3.5 Lazy Loading Strategies + +* **Mock lazy-loaded modules:** When testing code that uses lazy loading, mock the lazy-loaded modules to avoid loading them during testing. This can improve test performance and reduce dependencies. + +## 4. Security Best Practices + +### 4.1 Common Vulnerabilities and How to Prevent Them + +* **Cross-Site Scripting (XSS):** Prevent XSS vulnerabilities by sanitizing user input and encoding output. Ensure the testing framework is not vulnerable either. Check versions of plugins. + +* **Injection Attacks:** Prevent injection attacks by validating user input and using parameterized queries. + +* **Sensitive Data Exposure:** Avoid storing sensitive data in test files. Use environment variables or secure configuration files to manage sensitive data. + +### 4.2 Input Validation + +* **Test input validation:** Ensure your tests cover input validation scenarios. Verify that your code correctly validates user input and handles invalid input gracefully. + +### 4.3 Authentication and Authorization Patterns + +* **Mock authentication:** When testing code that requires authentication, mock the authentication service to avoid making actual API calls. Verify that your code correctly handles authenticated and unauthenticated states. + +* **Test authorization:** Ensure your tests cover authorization scenarios. Verify that your code correctly enforces access control and prevents unauthorized access to resources. + +### 4.4 Data Protection Strategies + +* **Protect sensitive data in tests:** Avoid including sensitive data in your test files. Use mock data or anonymized data for testing. + +### 4.5 Secure API Communication + +* **Mock API responses:** Mock API responses to avoid making actual API calls during testing. Use HTTPS for secure communication. + +## 5. Testing Approaches + +### 5.1 Unit Testing Strategies + +* **Focus on individual units:** Unit tests should focus on testing individual functions, classes, or modules in isolation. Avoid testing multiple units of code in a single test. + +* **Test all code paths:** Ensure your unit tests cover all possible code paths, including normal execution paths, error conditions, and edge cases. + +* **Use mocks and stubs:** Use mocks and stubs to isolate units of code and control their behavior during testing. + +### 5.2 Integration Testing + +* **Test interactions between units:** Integration tests should focus on testing the interactions between different units of code. Verify that units of code work together correctly. + +* **Use real dependencies:** Use real dependencies in integration tests whenever possible. This can help ensure that your code works correctly in a real-world environment. + +### 5.3 End-to-End Testing + +* **Test the entire application:** End-to-end tests should focus on testing the entire application, from the user interface to the backend services. Verify that the application works correctly from the user's perspective. + +* **Use a real browser:** Use a real browser for end-to-end testing. This can help ensure that your application works correctly in different browsers and environments. + +### 5.4 Test Organization + +* **Group related tests:** Organize tests into modules using `describe` blocks. This improves readability and helps structure test output. (See 1.3 Module Organization) + +* **Use meaningful test names:** Use descriptive test names that clearly indicate what the test is verifying. This makes it easier to understand the purpose of the test and to identify the cause of failures. + +* **Keep tests short and focused:** Keep tests short and focused on a single aspect of the code. This improves readability and makes it easier to maintain the tests. + +### 5.5 Mocking and Stubbing + +* **Use mocks to verify interactions:** Use mocks to verify that functions are called with the correct arguments and that they return the correct values. + +* **Use stubs to control behavior:** Use stubs to control the behavior of functions during testing. This allows you to simulate different scenarios and test how your code handles them. + +* **Avoid over-mocking:** Only mock dependencies that are not under test. Over-mocking can lead to tests that are brittle and do not accurately reflect the behavior of the code. + +## 6. Common Pitfalls and Gotchas + +### 6.1 Frequent Mistakes Developers Make + +* **Not writing enough tests:** Ensure that you have sufficient test coverage to catch bugs and prevent regressions. + +* **Writing brittle tests:** Avoid writing tests that are too tightly coupled to the implementation details of the code. This makes the tests brittle and difficult to maintain. + +* **Ignoring test failures:** Address test failures promptly. Ignoring test failures can lead to regressions and make it more difficult to maintain the code. + +### 6.2 Edge Cases to Be Aware Of + +* **Null and undefined values:** Ensure your tests cover scenarios where values are null or undefined. + +* **Empty strings and arrays:** Ensure your tests cover scenarios where strings or arrays are empty. + +* **Boundary values:** Ensure your tests cover boundary values, such as the minimum and maximum values of numeric types. + +### 6.3 Version-Specific Issues + +* **Keep Vitest up to date:** Stay up-to-date with the latest version of Vitest to benefit from bug fixes, performance improvements, and new features. + +### 6.4 Compatibility Concerns + +* **Test on different browsers and environments:** Ensure your tests cover different browsers and environments to catch compatibility issues. + +### 6.5 Debugging Strategies + +* **Use debugging tools:** Utilize debugging tools to step through your tests and identify the cause of failures. Vitest integrates with popular debuggers. + +* **Write clear and concise tests:** Write clear and concise tests to make it easier to understand the purpose of the test and to identify the cause of failures. + +* **Use logging:** Add logging statements to your tests to help track the flow of execution and identify the source of problems. + +## 7. Tooling and Environment + +### 7.1 Recommended Development Tools + +* **VS Code with the Vitest extension:** VS Code with the Vitest extension provides a rich development experience, including test discovery, execution, and debugging. + +* **Other IDEs with Vitest support:** Many other IDEs, such as WebStorm and IntelliJ IDEA, also offer support for Vitest. + +### 7.2 Build Configuration + +* **Use a build tool:** Use a build tool, such as Vite or esbuild, to bundle your code and optimize it for testing. + +* **Configure Vitest:** Configure Vitest to suit your project's needs. Use the `vitest.config.ts` file to customize Vitest's behavior. + +### 7.3 Linting and Formatting + +* **Use ESLint and Prettier:** Use ESLint and Prettier to enforce consistent coding styles and catch potential errors. Integrate these tools into your development workflow. + +### 7.4 Deployment Best Practices + +* **Run tests before deployment:** Always run your tests before deploying your code to ensure that it is working correctly. Automate this process as part of your CI/CD pipeline. + +### 7.5 CI/CD Integration + +* **Integrate Vitest with your CI/CD pipeline:** Integrate Vitest with your CI/CD pipeline to automatically run tests on every commit. This helps catch bugs early and prevent regressions. + +* **Use a CI/CD service:** Use a CI/CD service, such as GitHub Actions or GitLab CI, to automate your build, test, and deployment processes. + +## Conclusion + +By following these best practices, you can create robust and maintainable test suites with Vitest that ensure the quality and reliability of your code. \ No newline at end of file diff --git a/.cursor/rules/vllm.mdc b/.cursor/rules/vllm.mdc new file mode 100644 index 000000000..550b83418 --- /dev/null +++ b/.cursor/rules/vllm.mdc @@ -0,0 +1,140 @@ +--- +description: This rule outlines the best practices and coding standards for developing with the vllm library, ensuring code quality, performance, and maintainability. It covers code organization, performance considerations, security, testing, and common pitfalls. +globs: *.py +--- +- ## General + - Adhere to the Google Python Style Guide: vllm projects should strictly follow the Google Python Style Guide for consistency and readability. + - Adhere to the Google C++ Style Guide (where applicable, especially for backend components). + - Pass all linter checks: Ensure code passes all configured linter checks (e.g., pylint, flake8) before committing. + - Always use UV when installing dependencies. + - Always use Python 3.12 or higher. + - Prefer classes over bare functions for better code organization and state management (where appropriate). + +- ## Code Organization and Structure: + - **Directory Structure Best Practices:** + - `vllm/` (Root directory): + - `api/`: Contains API endpoints and related logic. + - `core/`: Core functionalities and algorithms. + - `layers/`: Custom layers for the models. + - `models/`: Model definitions and configurations. + - `sampling/`: Sampling algorithms. + - `sequence/`: Sequence management and data structures. + - `utils/`: Utility functions and helper modules. + - `tests/`: Unit and integration tests. + - `examples/`: Example usage and demonstrations. + - `docs/`: Documentation. + - Maintain clear separation of concerns. + - **File Naming Conventions:** + - Use descriptive and meaningful names for files and modules (e.g., `attention_layer.py`, `model_loader.py`). + - Follow snake_case for Python files and variables. + - **Module Organization:** + - Group related functionalities into modules. + - Use `__init__.py` files to define packages and control module imports. + - **Component Architecture:** + - Design components with clear interfaces and responsibilities. + - Favor composition over inheritance to promote flexibility and reusability. + - **Code Splitting Strategies:** + - Break down large modules into smaller, more manageable files. + - Utilize lazy loading or dynamic imports to reduce startup time. + +- ## Common Patterns and Anti-patterns: + - **Design Patterns Specific to vllm:** + - **Model Abstraction:** Decouple the model implementation from the core engine. + - **Sequence Manager:** Centralized management of sequences and their states. + - **Asynchronous Execution:** Use asyncio to handle concurrent requests and I/O operations. + - **Recommended Approaches for Common Tasks:** + - **Model Loading:** Implement a robust model loading mechanism with caching and error handling. + - **Tokenization:** Use a dedicated tokenizer class to handle tokenization and detokenization. + - **Inference:** Design an efficient inference pipeline with batching and optimized tensor operations. + - **Anti-patterns and Code Smells to Avoid:** + - **God Classes:** Avoid creating large classes with too many responsibilities. + - **Code Duplication:** Refactor duplicated code into reusable functions or classes. + - **Magic Numbers:** Use named constants for configuration values. + - **State Management Best Practices:** + - Use immutable data structures to avoid unintended side effects. + - Manage state within dedicated classes or modules. + - Avoid global state where possible. + - **Error Handling Patterns:** + - Use exceptions to handle errors and unexpected conditions. + - Provide informative error messages. + - Implement retry mechanisms for transient errors. + +- ## Performance Considerations: + - **Optimization Techniques:** + - **Tensor Optimization:** Use optimized tensor operations and data layouts (e.g., `torch.compile`). + - **Kernel Fusion:** Fuse multiple operations into a single kernel to reduce overhead. + - **Quantization:** Use quantization techniques to reduce model size and memory footprint. + - **Memory Management:** + - Minimize memory allocations and deallocations. + - Use memory pooling to reuse memory buffers. + - Profile memory usage to identify bottlenecks. + - **Bundle Size Optimization:** + - Remove unused dependencies. + - **Lazy Loading Strategies:** + - Use lazy loading for large modules or resources. + - Implement asynchronous loading for non-critical components. + +- ## Security Best Practices: + - **Common Vulnerabilities and How to Prevent Them:** + - **Injection Attacks:** Sanitize user inputs to prevent injection attacks. + - **Denial of Service (DoS):** Implement rate limiting and resource management to protect against DoS attacks. + - **Data Breaches:** Protect sensitive data with encryption and access controls. + - **Input Validation:** + - Validate all user inputs before processing. + - Use regular expressions or schema validation to enforce input constraints. + - **Authentication and Authorization Patterns:** + - Implement authentication to verify user identities. + - Use authorization to control access to resources. + - **Data Protection Strategies:** + - Encrypt sensitive data at rest and in transit. + - Use secure storage mechanisms. + - **Secure API Communication:** + - Use HTTPS for API communication. + - Implement API authentication and authorization. + +- ## Testing Approaches: + - **Unit Testing Strategies:** + - Write unit tests for individual functions and classes. + - Use mocking and stubbing to isolate units of code. + - **Integration Testing:** + - Test the interaction between different modules and components. + - **End-to-End Testing:** + - Test the entire system from end to end. + - **Test Organization:** + - Organize tests into separate directories. + - Use descriptive names for test files and functions. + - **Mocking and Stubbing:** + - Use mocking to replace external dependencies with controlled substitutes. + - Use stubbing to provide predefined responses for function calls. + +- ## Common Pitfalls and Gotchas: + - **Frequent Mistakes Developers Make:** + - **Incorrect Tensor Shapes:** Ensure tensor shapes are compatible for operations. + - **Memory Leaks:** Properly release allocated memory to prevent memory leaks. + - **Synchronization Issues:** Avoid race conditions and deadlocks in concurrent code. + - **Edge Cases to be Aware Of:** + - **Handling Empty Sequences:** Handle empty sequences gracefully. + - **Dealing with Unknown Tokens:** Implement a mechanism to handle unknown tokens. + - **Version-Specific Issues:** + - Keep dependencies up-to-date and be aware of breaking changes when upgrading. + - **Compatibility Concerns:** + - Ensure compatibility with different hardware platforms (CPU, GPU). + - Consider different versions of Python and PyTorch. + - **Debugging Strategies:** + - Use logging to track program execution. + - Use a debugger to step through code and inspect variables. + - Profile performance to identify bottlenecks. + +- ## Tooling and Environment: + - **Recommended Development Tools:** + - **VS Code:** A popular code editor with Python and C++ support. + - **PyCharm:** An IDE specifically designed for Python development. + - **Build Configuration:** + - Use `setup.py` or `pyproject.toml` to define project dependencies and build configurations. + - **Linting and Formatting:** + - Use pylint, flake8, and black to enforce code style. + - **Deployment Best Practices:** + - Use Docker to containerize the application. + - Deploy to a cloud platform such as AWS, Google Cloud, or Azure. + - **CI/CD Integration:** + - Use a CI/CD pipeline to automate testing and deployment. \ No newline at end of file diff --git a/.cursor/rules/vue.mdc b/.cursor/rules/vue.mdc new file mode 100644 index 000000000..ceb131ae5 --- /dev/null +++ b/.cursor/rules/vue.mdc @@ -0,0 +1,298 @@ +--- +description: Comprehensive guidelines for Vue.js development, covering code structure, performance, security, testing, and tooling best practices. This rule provides actionable guidance to enhance code quality, maintainability, and developer productivity in Vue.js projects. +globs: *.vue, *.js, *.ts +--- +# Vue.js Best Practices and Coding Standards + +This document outlines best practices for Vue.js development, covering various aspects to ensure high-quality, maintainable, and performant code. + +## 1. Code Organization and Structure + +### 1.1. Directory Structure + +* **Component-Based Structure:** Organize components into logical folders based on their functionality or feature. This improves code readability and maintainability. + + src/ + ├── components/ + │ ├── Button/ + │ │ ├── Button.vue + │ │ └── Button.spec.js + │ ├── Input/ + │ │ ├── Input.vue + │ │ └── Input.spec.js + │ └── ... + ├── views/ + │ ├── Home.vue + │ ├── About.vue + │ └── ... + ├── services/ + │ ├── api.js + │ └── auth.js + ├── store/ + │ ├── index.js # Vuex store + │ ├── modules/ + │ │ ├── user.js + │ │ └── ... + ├── App.vue + └── main.js + +* **Feature-Based Structure:** Alternatively, organize files by feature, grouping components, routes, and store modules related to a specific feature. + + src/ + ├── features/ + │ ├── user-profile/ + │ │ ├── components/ + │ │ │ ├── UserProfile.vue + │ │ │ └── ... + │ │ ├── routes.js + │ │ ├── store.js + │ │ └── ... + │ ├── shopping-cart/ + │ │ ├── ... + │ └── ... + ├── App.vue + └── main.js + + +### 1.2. File Naming Conventions + +* **Component Files:** Use PascalCase for component file names (e.g., `MyComponent.vue`). +* **Other Files:** Use camelCase or kebab-case for other JavaScript/TypeScript files (e.g., `apiService.js`, `my-helper.js`). +* **Consistency:** Maintain a consistent naming convention throughout the project. + +### 1.3. Module Organization + +* **ES Modules:** Utilize ES modules (`import`/`export`) for modular code organization. +* **Single Responsibility Principle:** Each module should have a single, well-defined responsibility. +* **Avoid Circular Dependencies:** Prevent circular dependencies between modules to avoid unexpected behavior and improve maintainability. + +### 1.4. Component Architecture + +* **Component Composition:** Favor component composition over inheritance for increased flexibility and reusability. +* **Presentational and Container Components:** Separate presentational (dumb) components from container (smart) components. Presentational components focus on rendering UI, while container components handle data fetching and logic. +* **Single File Components (SFCs):** Leverage Vue's SFCs for encapsulating component logic, template, and styling. + +### 1.5. Code Splitting Strategies + +* **Route-Based Splitting:** Use dynamic imports and Vue's `async` component feature to split the application into chunks based on routes. +* **Component-Based Splitting:** Split large components into smaller, lazy-loaded components to improve initial load time. +* **Vendor Splitting:** Separate vendor dependencies into a separate chunk to allow for browser caching and prevent unnecessary reloads. + +## 2. Common Patterns and Anti-patterns + +### 2.1. Design Patterns Specific to Vue + +* **Provide/Inject:** Use `provide` and `inject` for dependency injection between components, especially when dealing with deeply nested components. +* **Renderless Components:** Create renderless components that encapsulate logic and provide data to be rendered by slot-using components. +* **Higher-Order Components (HOCs):** Use HOCs to reuse component logic or add functionality to existing components. + +### 2.2. Recommended Approaches for Common Tasks + +* **Form Handling:** Use `v-model` for two-way data binding in forms. Consider using a form validation library like Vuelidate or VeeValidate for robust form validation. +* **API Requests:** Use a dedicated service module for handling API requests. Use `async/await` for cleaner asynchronous code. +* **State Management:** Utilize Vuex for centralized state management in larger applications. For simpler applications, consider using Vue's reactivity system directly or a lightweight state management solution like Pinia. +* **Event Handling:** Use component events (`$emit`) for communication between parent and child components. For communication between unrelated components, use a global event bus (with caution) or a state management solution. + +### 2.3. Anti-patterns and Code Smells to Avoid + +* **Mutating Props Directly:** Avoid mutating props directly within a component. Instead, emit an event to the parent component to update the prop value. +* **Overusing Global State:** Avoid storing too much data in global state. Use local component state whenever possible. +* **Direct DOM Manipulation:** Avoid directly manipulating the DOM using `document` APIs. Use Vue's template directives and component APIs to update the DOM reactively. +* **Magic Numbers and Strings:** Avoid using magic numbers and strings directly in the code. Use constants to improve readability and maintainability. +* **Complex Computed Properties:** Keep computed properties simple and focused. Complex computations should be moved to methods or utility functions. + +### 2.4. State Management Best Practices + +* **Single Source of Truth:** Maintain a single source of truth for application state using Vuex or Pinia. +* **Mutations for State Updates:** Only use mutations to update the state in Vuex. Mutations should be synchronous and atomic. +* **Actions for Asynchronous Operations:** Use actions to handle asynchronous operations like API requests. Actions can commit mutations to update the state. +* **Getters for Derived State:** Use getters to derive state from the store. Getters should be pure functions and should not modify the state. +* **Modularity:** Organize the store into modules to improve maintainability and scalability. + +### 2.5. Error Handling Patterns + +* **Centralized Error Handling:** Implement a centralized error handling mechanism to catch and log errors consistently. +* **Error Boundary Components:** Use error boundary components to catch errors within specific parts of the application and prevent crashes. +* **User-Friendly Error Messages:** Provide user-friendly error messages to guide users when errors occur. +* **Logging:** Log errors to a server or error tracking service for monitoring and debugging. +* **Try-Catch Blocks:** Use `try-catch` blocks to handle potential errors in asynchronous operations or complex computations. + +## 3. Performance Considerations + +### 3.1. Optimization Techniques + +* **Virtual DOM Optimization:** Vue's virtual DOM implementation is already highly optimized, but avoid unnecessary re-renders by using `v-if` instead of `v-show` when elements are rarely displayed. +* **Computed Properties and Watchers:** Use computed properties and watchers judiciously. Avoid performing expensive computations in computed properties that are frequently re-evaluated. Debounce or throttle watchers to limit the number of updates. +* **List Rendering Optimization:** Use the `:key` attribute when rendering lists with `v-for` to help Vue track changes efficiently. Ensure the keys are unique and stable. +* **Functional Components:** Use functional components for simple, stateless components to improve rendering performance. +* **Avoid Inline Templates:** Use pre-compiled templates in single-file components instead of inline templates (using `