Files
obsidian/Test/EventKit/07 - Technical Requirements.md

551 lines
30 KiB
Markdown

---
tags:
- eventkit
---
# Technical Requirements
## Purpose
Non-functional requirements, deployment model, API design, and infrastructure considerations for EventKit.
---
## Technology Stack
### Core Decisions
| Layer | Technology | Rationale |
| -------------------- | -------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |
| **Backend language** | **Go** | Fast compilation, single binary deployment, excellent concurrency, strong ecosystem for network services |
| **API protocol** | **ConnectRPC** (gRPC wrapper) | Type-safe, code-generated clients, supports gRPC, gRPC-Web, and Connect protocols — works in browsers natively |
| **API definition** | **Protocol Buffers (Protobuf)** | Schema-first API design, backward-compatible evolution, generates Go server stubs + TypeScript/JS clients |
| **Database** | **PostgreSQL 15+** | Robust, proven, excellent JSON support, full-text search, strong ecosystem |
| **Frontend** | **React** (TanStack Router + TanStack Query) | Team expertise, strong ecosystem, ConnectRPC generates typed TS clients |
| **UI components** | **shadcn/ui** | Accessible, composable, Tailwind-based component library |
| **File storage** | Local filesystem or S3-compatible | Configurable per deployment — local for self-hosted, S3/MinIO for SaaS |
| **Search** | PostgreSQL full-text search | Start with built-in FTS; add dedicated engine (Meilisearch) later if needed |
| **Background jobs** | NATS or Redis | External message broker — suits Kubernetes deployment model |
### Why ConnectRPC
[ConnectRPC](https://connectrpc.com/) wraps gRPC to provide **three protocols in one**:
| Protocol | Transport | Use Case |
| ------------ | ------------------------------------ | ---------------------------------------------------------- |
| **Connect** | HTTP/1.1 or HTTP/2, JSON or Protobuf | Browser clients (no proxy needed), simple `curl` debugging |
| **gRPC** | HTTP/2, Protobuf | Server-to-server, federation, high-performance clients |
| **gRPC-Web** | HTTP/1.1, Protobuf | Legacy browser support via Envoy/proxy |
This means:
- **Web browsers** call the same API using the Connect protocol (plain HTTP + JSON or Protobuf) — no Envoy proxy required
- **[[04 - Federation Architecture|Federation]]** between instances uses gRPC for efficient binary communication
- **Mobile apps** can use gRPC natively
- **One `.proto` definition** generates server code (Go), browser client (TypeScript), and mobile client code
### Frontend Stack
| Component | Technology | Notes |
| ----------------- | ---------------------------- | -------------------------------------------------- |
| **Framework** | React | SPA — SEO is not a requirement |
| **Routing** | @tanstack/router | Type-safe, file-based routing |
| **Data fetching** | @tanstack/query | Caching, background refetching, optimistic updates |
| **UI components** | shadcn/ui | Accessible, composable component library |
| **API client** | ConnectRPC generated TS code | Type-safe from `.proto` definitions |
| **Build tool** | Vite | Fast dev server and optimised builds |
| **Theme** | Dark mode (default) | Light mode + high contrast mode as toggles |
### Mobile App
| Component | Technology | Notes |
| ----------------- | ---------------------------- | ------------------------------------------------ |
| **Framework** | React Native / Expo | Shared knowledge with web React team |
| **Data fetching** | @tanstack/query | Same patterns as web client |
| **API client** | ConnectRPC generated TS code | Same `.proto` definitions as web |
| **Priority** | Planned but not prioritised | PWA / responsive web covers initial mobile needs |
| **Offline** | Nice-to-have | Queue scan operations and sync when reconnected |
### Project Structure (Proposed)
```
eventkit/
├── proto/ # Protobuf definitions
│ ├── eventkit/v1/
│ │ ├── inventory.proto
│ │ ├── planning.proto
│ │ ├── crm.proto
│ │ ├── scanning.proto
│ │ ├── federation.proto
│ │ └── common.proto # Shared types (UUID, timestamps, pagination)
│ └── buf.yaml # Buf schema registry config
├── cmd/
│ └── eventkit/
│ └── main.go # Application entrypoint
├── internal/
│ ├── server/ # ConnectRPC service implementations
│ ├── domain/ # Business logic (pure Go, no framework deps)
│ ├── store/ # Database access (PostgreSQL)
│ ├── federation/ # Federation protocol handler
│ └── scanner/ # Scanning workflow logic
├── gen/ # Generated code (protoc / buf)
│ ├── go/
│ └── ts/ # TypeScript client for frontend
├── web/ # Frontend application
├── migrations/ # SQL migrations
├── docker-compose.yml
├── Dockerfile
└── buf.gen.yaml # Code generation config
```
### Key Go Libraries
| Library | Purpose |
| ---------------------------------- | ------------------------------------------------------ |
| **connectrpc.com/connect** | ConnectRPC server and client |
| **buf.build/gen** | Protobuf code generation (replaces `protoc`) |
| **jackc/pgx** | PostgreSQL driver (high-performance, pure Go) |
| **golang-migrate/migrate** | Database migrations |
| **rs/zerolog** or **slog** | Structured logging (slog is stdlib in Go 1.21+) |
| **go-chi/chi** or **std net/http** | HTTP router (ConnectRPC mounts on standard `net/http`) |
| **prometheus/client_golang** | Metrics |
| **golang-jwt/jwt** | JWT authentication |
| **testcontainers-go** | Integration testing with real PostgreSQL |
---
## Deployment
### Self-Hosted (Primary Model)
Each company runs their own instance. The deployment should be **simple and reproducible**.
### Multi-Tenancy (Managed SaaS)
For the hosted SaaS offering, use a **shared database with tenant isolation**:
| Aspect | Approach |
| -------------------- | ----------------------------------------------------------------------------------------------------------- |
| **Database** | Shared PostgreSQL instance, tenant ID on every table |
| **Isolation** | Row-level security (RLS) policies enforce tenant boundaries — complete isolation, no cross-tenant data ever |
| **Cross-tenant** | No shared data between tenants — use Federation for inter-company workflows |
| **Connection** | Connection pooling shared across tenants |
| **Migration** | Single schema, all tenants upgrade together |
| **Data sovereignty** | Tenants wanting full isolation should use self-hosted or dedicated tier |
| **Onboarding** | Self-service with admin approval, or manual admin creation |
| **Data export** | Full JSON/CSV export at any time (GDPR Article 20) + right to be forgotten |
| **Quotas** | Soft limits with warnings — pricing tiers TBD |
| Approach | Description |
| ---------------------- | -------------------------------------------------------------------- |
| **Docker Compose** | Recommended for single-server deployments — all services in one file |
| **Kubernetes / Helm** | For larger companies wanting HA, auto-scaling, or multi-node |
| **Single binary + DB** | Possible for minimal deployments — app server + PostgreSQL |
### Minimum Infrastructure
| Component | Requirement |
| ---------------------- | ----------------------------------------------------------- |
| **Application server** | Linux VPS, 2+ CPU, 4+ GB RAM |
| **Database** | PostgreSQL 15+ |
| **File storage** | Local filesystem or S3-compatible object storage |
| **Reverse proxy** | Nginx / Caddy / Traefik (TLS termination) |
| **Domain + TLS** | Required for federation (valid certificate needed for mTLS) |
### Docker Compose Example Structure
```yaml
services:
eventkit:
image: eventkit/server:latest
# Single Go binary — serves ConnectRPC API + static frontend
ports:
- "8080:8080" # Connect protocol (HTTP/1.1 + HTTP/2)
- "8081:8081" # gRPC (federation, native clients)
environment:
- DATABASE_URL=postgres://eventkit:secret@db:5432/eventkit?sslmode=disable
- FEDERATION_ENABLED=true
- INSTANCE_URL=https://inventory.company-a.com
- FEDERATION_GRPC_PORT=8081
volumes:
- ./data/uploads:/app/uploads
db:
image: postgres:15
environment:
- POSTGRES_USER=eventkit
- POSTGRES_PASSWORD=secret
- POSTGRES_DB=eventkit
volumes:
- ./data/postgres:/var/lib/postgresql/data
caddy:
image: caddy:latest
ports:
- "443:443"
- "80:80"
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile
```
> [!NOTE]
> Go compiles to a **single static binary** — no runtime, no JVM, no interpreter. The `eventkit` container is typically < 30 MB. Background workers run as goroutines within the same process.
---
## API Design
### Protocol
| Decision | Choice |
| -------------------------- | -------------------------------------------------------------------------------- |
| **API protocol** | ConnectRPC (supports Connect, gRPC, and gRPC-Web protocols simultaneously) |
| **Schema language** | Protocol Buffers v3 (`.proto` files) |
| **Code generation** | `buf` CLI — generates Go server stubs + TypeScript clients |
| **Transport (browsers)** | Connect protocol over HTTP/1.1 or HTTP/2 (JSON or Protobuf encoding) |
| **Transport (federation)** | gRPC over HTTP/2 (Protobuf encoding) |
| **Authentication** | JWT tokens (local users) + mTLS (federation) |
| **Pagination** | Cursor-based using `page_token` / `next_page_token` pattern in Protobuf messages |
| **Rate limiting** | Per-user and per-federation-partner, implemented as ConnectRPC interceptors |
### Protobuf Service Definitions
All APIs are defined as Protobuf services. Example structure:
```protobuf
// proto/eventkit/v1/inventory.proto
syntax = "proto3";
package eventkit.v1;
service InventoryService {
rpc ListProducts(ListProductsRequest) returns (ListProductsResponse);
rpc GetProduct(GetProductRequest) returns (Product);
rpc CreateProduct(CreateProductRequest) returns (Product);
rpc UpdateProduct(UpdateProductRequest) returns (Product);
rpc DeleteProduct(DeleteProductRequest) returns (DeleteProductResponse);
rpc ListAssets(ListAssetsRequest) returns (ListAssetsResponse);
rpc GetAsset(GetAssetRequest) returns (Asset);
rpc CheckAvailability(CheckAvailabilityRequest) returns (CheckAvailabilityResponse);
}
```
### Service Map
| Protobuf Service | Package | Purpose |
| ------------------- | ------------- | ------------------------------------------------ |
| `InventoryService` | `eventkit.v1` | Products, assets, cases, locations, availability |
| `PlanningService` | `eventkit.v1` | Events, pull lists, scheduling |
| `CrewService` | `eventkit.v1` | Crew profiles, assignments, availability |
| `TransportService` | `eventkit.v1` | Vehicles, truck packs, dispatch |
| `CRMService` | `eventkit.v1` | Contacts, companies, deals, pipeline |
| `QuoteService` | `eventkit.v1` | Quote building, versioning, PDF generation |
| `ScanningService` | `eventkit.v1` | Check-in, check-out, stocktake, case packing |
| `FederationService` | `eventkit.v1` | Trust handshake, sub-hire, asset resolution |
| `AuthService` | `eventkit.v1` | Login, token refresh, user management |
| `ReportService` | `eventkit.v1` | Dashboards, analytics, data export |
### How Clients Connect
| Client | Protocol | Encoding | Notes |
| -------------------- | --------------- | ----------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| **Web browser** | Connect | JSON (dev) or Protobuf (prod) | No proxy needed — standard `fetch()` under the hood |
| **Mobile app** | gRPC | Protobuf | Native gRPC client libraries |
| **Federation** | gRPC | Protobuf | Instance-to-instance via mTLS |
| **CLI tool** | Connect or gRPC | Protobuf | For automation and scripting |
| **cURL / debugging** | Connect | JSON | `curl -X POST -H 'Content-Type: application/json' -d '{...}' https://host/eventkit.v1.InventoryService/ListProducts` |
### ConnectRPC Interceptors (Middleware)
| Interceptor | Purpose |
| ------------------- | ------------------------------------------------- |
| **Auth** | Validate JWT, extract user context |
| **Federation Auth** | Validate mTLS certificate, check trust level |
| **Rate Limiter** | Per-user and per-partner rate limiting |
| **Logger** | Structured request/response logging |
| **Recovery** | Panic recovery with error reporting |
| **Validator** | Protobuf message validation (e.g. `buf validate`) |
| **Metrics** | Prometheus request duration, error rates |
---
## Authentication & Authorisation
### Identity Provider — Keycloak
Authentication is delegated to **Keycloak** as the central identity provider:
| Component | Approach |
| ---------------------- | ---------------------------------------------------------------------------------------------------- |
| **Identity provider** | Keycloak (self-hosted alongside EventKit) |
| **Protocol** | Generic OIDC — works with Keycloak, Auth0, or any OIDC provider |
| **Session management** | Handled by Keycloak (tokens, refresh, SSO) |
| **User onboarding** | Admin creates user → email invitation → user sets password |
| **Role management** | Roles defined in Keycloak, permissions mapped in EventKit |
| **Federation auth** | mTLS certificates for instance-to-instance communication |
| **API keys** | Granular scopes (e.g. `inventory:read`, `planning:write`) — start with full access, add scopes later |
### Role-Based Access Control (RBAC)
Roles are managed in Keycloak. EventKit reads roles and maps them to permissions:
| Role | Permissions |
| ------------- | -------------------------------------------------------------------- |
| **Admin** | Full system access, user management, federation management, settings |
| **Manager** | All operational features, reporting, approve quotes and purchases |
| **Planner** | Create/edit events, manage pull lists, assign crew |
| **Warehouse** | Scan in/out, manage stock, stocktakes, maintenance |
| **Sales** | CRM access, create/send quotes, manage pipeline |
| **Crew** | View assigned events, update availability, timesheets |
| **Read-only** | View dashboards and reports, no edit capabilities |
Custom roles with granular permission sets are fully supported — admins create roles in Keycloak, then map permissions in EventKit.
---
## Data Management
### Soft Delete
| Behaviour | Description |
| --------------- | ------------------------------------------------------- |
| **Default** | Soft delete — records marked as deleted, hidden from UI |
| **Recovery** | Admins can restore soft-deleted records |
| **Hard delete** | Admin action after configurable retention period |
| **Cascade** | Related records follow the parent's delete behaviour |
### Audit Trail
| Feature | Description |
| ------------------------ | --------------------------------------------- |
| **Field-level tracking** | Old value → new value for every field change |
| **User attribution** | Who made the change and when |
| **Entity history** | Full change history per record (like Git log) |
| **Immutable log** | Audit records cannot be edited or deleted |
| **Future** | Full diff view (Git-style) for every record |
### Caching
| Component | Strategy |
| ---------- | ----------------------------------------------------- |
| **Layer** | Redis (shared with background job broker) |
| **Scope** | Session cache, query cache, federation response cache |
| **Policy** | TTL-based with manual invalidation on writes |
### Database Migrations
| Environment | Strategy |
| --------------- | ---------------------------------------------------- |
| **Development** | Automatic on startup |
| **Production** | Explicit migration step before deploying new version |
---
## UI & Navigation
| Feature | Description |
| ------------------- | -------------------------------------------------------------------------------- |
| **Navigation** | Sidebar with module sections + command palette (Cmd+K) |
| **Dashboard** | Configurable per user — multiple layouts with simple view switching |
| **Bulk operations** | Bulk status change, assign to event, print labels, import/export |
| **List views** | Table default, card view for specific entities (e.g. assets with photos) |
| **Keyboard** | Full keyboard navigation (like Linear) — Cmd+K, shortcuts for all common actions |
### Dashboard Widgets
| Widget | Description |
| ------------------------------- | ---------------------------------- |
| Upcoming events | Next 7/14/30 days |
| Equipment utilisation overview | Usage rates across categories |
| Open quotes / pending approvals | Awaiting client or internal action |
| Overdue returns / missing items | Equipment past return deadline |
| Revenue summary | Period revenue with trend |
| Low-stock consumable alerts | Below reorder threshold |
| Crew availability overview | Available vs booked crew |
| Recent activity feed | Latest system-wide actions |
---
## Data Model Highlights
### Key Relationships
```mermaid
erDiagram
COMPANY ||--o{ CONTACT : has
COMPANY ||--o{ DEAL : has
DEAL ||--o{ QUOTE : has
DEAL ||--o| EVENT : creates
EVENT ||--o{ PULL_LIST_ITEM : has
EVENT ||--o{ CREW_ASSIGNMENT : has
EVENT ||--o{ LOAD : has
EVENT }o--|| VENUE : at
EVENT }o--o| PROJECT : part_of
PRODUCT ||--o{ ASSET : has_instances
PRODUCT ||--o{ PULL_LIST_ITEM : referenced_by
ASSET }o--o| CASE : packed_in
ASSET }o--|| LOCATION : stored_at
ASSET ||--o{ ASSET_STATUS_LOG : has
ASSET ||--o{ MAINTENANCE_RECORD : has
CASE }o--|| LOCATION : stored_at
CASE ||--o{ ASSET : contains
KIT ||--o{ KIT_ITEM : contains
KIT_ITEM }o--|| PRODUCT : references
```
---
## Performance Requirements
| Metric | Target |
| --------------------- | ------------------------------------ |
| **Page load time** | < 2 seconds |
| **API response time** | < 500ms (p95) |
| **Search response** | < 1 second |
| **Scan processing** | < 500ms (perceived instant) |
| **Concurrent users** | 50+ per instance |
| **Asset count** | Support 100,000+ assets per instance |
| **Event count** | Support 10,000+ events per instance |
---
## Backup & Recovery
| Feature | Description |
| ------------------------- | -------------------------------------------------- |
| **Automated backups** | Daily database backups with configurable retention |
| **Point-in-time restore** | Transaction log backups for precise recovery |
| **File backup** | Uploaded files backed up alongside database |
| **Export** | Full data export in standard formats (JSON, CSV) |
| **Import** | Data import for migration from other systems |
---
## Notifications
| Channel | Supported | Notes |
| --------------- | --------- | --------------------------------------------- |
| **Email** | ✅ Yes | Transactional emails (SMTP or provider API) |
| **Web push** | ✅ Yes | Browser push notifications via service worker |
| **Native push** | ✅ Yes | Via Expo push / APNs / FCM for mobile app |
| **Webhooks** | ✅ Yes | Outbound webhooks for third-party integration |
| **SMS** | ❌ No | Not planned |
### Webhook Events
All major entity changes trigger outbound webhooks:
| Event Category | Examples |
| -------------- | ------------------------------------------------ |
| **Assets** | Status changed (checked out, returned, damaged) |
| **Events** | Status changed (confirmed, completed, cancelled) |
| **Quotes** | Approved / rejected by client |
| **Invoices** | Paid, overdue |
| **Stock** | Low stock alert |
| **Federation** | Sub-hire request, asset transfer |
| **CRUD** | Create / update / delete on any major entity |
---
## Monitoring & Observability
| Feature | Description |
| ------------------------- | ------------------------------------------------------ |
| **Health check endpoint** | `/health` for uptime monitoring |
| **Structured logging** | JSON logs for easy parsing and analysis |
| **Metrics** | Prometheus-compatible metrics endpoint |
| **Error tracking** | Integration with Sentry or similar |
| **Federation health** | Monitor federation connection status with each partner |
---
## Reporting
| Feature | Description |
| ------------------------- | -------------------------------------------------- |
| **Output formats** | In-app, PDF, CSV/Excel, scheduled email delivery |
| **Predefined reports** | Standard reports with filters and grouping |
| **Custom report builder** | Drag-and-drop fields, custom calculations (future) |
| **Roadmap** | Predefined + filters for MVP, full builder later |
---
## Document Generation
| Feature | Description |
| ---------------------- | --------------------------------------------------------- |
| **Branding** | Company logo + name + colours for MVP |
| **Full templates** | Header, footer, fonts, layout customisation (future) |
| **Template engine** | HTML/CSS templates — users can provide their own (future) |
| **Document numbering** | Configurable prefix + sequence per document type |
---
## Data Retention & Files
| Feature | Description |
| -------------------- | ------------------------------------------------------------------- |
| **Retention policy** | Configurable per entity type (e.g. events: 5 years, audit: 7 years) |
| **Legal compliance** | Follow Austrian/EU legal requirements (varies by document type) |
| **File versioning** | Keep version history (v1, v2, v3) when files are updated |
---
## Notifications
| Feature | Description |
| -------------------- | ---------------------------------------------------------------- |
| **User preferences** | Granular per-event-type toggle (future feature) |
| **Email templates** | Tenant-branded for MVP (logo, colours), full customisation later |
---
## API & Extensibility
| Feature | Description |
| --------------------- | ------------------------------------------------------------- |
| **API documentation** | Auto-generated from Protobuf definitions (buf.build registry) |
| **MVP docs** | Minimal — just the `.proto` files |
| **Plugin system** | Design for extensibility now, implement plugin API later |
| **Webhooks** | Primary extensibility mechanism for MVP |
---
## Testing & Environments
| Feature | Description |
| ------------------------ | ----------------------------------------------------------------------- |
| **Testing strategy** | Whatever ships fastest with confidence |
| **Staging environments** | Ephemeral preview environments per PR (Kubernetes-native, not priority) |
| **Local development** | Docker Compose for local dev stack |
---
## Internationalisation
| Feature | Description |
| ----------------------- | ---------------------------------------------------------------------------- |
| **i18n framework** | Built-in from the start — translations added incrementally |
| **MVP languages** | English + German |
| **Multi-currency** | Configurable from the start — currency, tax rates, tax ID formats per tenant |
| **Timezone support** | Per-user timezone, event times stored in UTC |
| **Date/number formats** | Locale-aware formatting (e.g. `dd.MM.yyyy` for DE, `MM/dd/yyyy` for EN) |
| **Right-to-left** | RTL layout support for applicable languages (future) |
---
## Related Documentation
- [[00 - System Overview]] — High-level system overview
- [[01 - Inventory Management]] — Inventory module served by `InventoryService`
- [[02 - Planning Module]] — Planning module served by `PlanningService`
- [[03 - CRM Module]] — CRM module served by `CRMService` and `QuoteService`
- [[04 - Federation Architecture]] — Federation deployment and security
- [[05 - Barcode and QR Scanning]] — Hardware requirements and `ScanningService`
- [[06 - Module Integration]] — Cross-module workflows
- [[08 - Open Questions]] — Remaining technical decisions