Feat: Replace UUID image identifiers with 8-character base62 short IDs
Short IDs become the canonical identifier in URLs (/i/:short_id), MinIO/R2 storage keys, and all API responses. Hash-based deduplication is preserved. Includes two-phase Alembic migration (003 adds nullable column, 004 enforces NOT NULL) with a backfill script to copy storage objects and populate short_id for existing images. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
34
specs/017-short-id-migration/checklists/requirements.md
Normal file
34
specs/017-short-id-migration/checklists/requirements.md
Normal file
@@ -0,0 +1,34 @@
|
||||
# Specification Quality Checklist: Short Image IDs
|
||||
|
||||
**Purpose**: Validate specification completeness and quality before proceeding to planning
|
||||
**Created**: 2026-05-09
|
||||
**Feature**: [spec.md](../spec.md)
|
||||
|
||||
## Content Quality
|
||||
|
||||
- [X] No implementation details (languages, frameworks, APIs)
|
||||
- [X] Focused on user value and business needs
|
||||
- [X] Written for non-technical stakeholders
|
||||
- [X] All mandatory sections completed
|
||||
|
||||
## Requirement Completeness
|
||||
|
||||
- [X] No [NEEDS CLARIFICATION] markers remain
|
||||
- [X] Requirements are testable and unambiguous
|
||||
- [X] Success criteria are measurable
|
||||
- [X] Success criteria are technology-agnostic (no implementation details)
|
||||
- [X] All acceptance scenarios are defined
|
||||
- [X] Edge cases are identified
|
||||
- [X] Scope is clearly bounded
|
||||
- [X] Dependencies and assumptions identified
|
||||
|
||||
## Feature Readiness
|
||||
|
||||
- [X] All functional requirements have clear acceptance criteria
|
||||
- [X] User scenarios cover primary flows
|
||||
- [X] Feature meets measurable outcomes defined in Success Criteria
|
||||
- [X] No implementation details leak into specification
|
||||
|
||||
## Notes
|
||||
|
||||
- All items pass. Ready for /speckit-plan.
|
||||
115
specs/017-short-id-migration/contracts/image-api.md
Normal file
115
specs/017-short-id-migration/contracts/image-api.md
Normal file
@@ -0,0 +1,115 @@
|
||||
# Contract: Image API (Short ID Update)
|
||||
|
||||
## ImageRecord Response Schema
|
||||
|
||||
All image endpoints return this shape. `short_id` is a new field; all other fields are unchanged.
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "7343d164-80bb-473b-b239-717f2842ae4e",
|
||||
"short_id": "xK7mN2pQ",
|
||||
"hash": "163dec08460650439f1e7439721e8e566aff7d8aaad60cf451e7d3518a334a23",
|
||||
"filename": "image.gif",
|
||||
"mime_type": "image/gif",
|
||||
"size_bytes": 1957149,
|
||||
"width": 265,
|
||||
"height": 199,
|
||||
"storage_key": "xK7mN2pQ",
|
||||
"thumbnail_key": "xK7mN2pQ-thumb",
|
||||
"file_url": "https://cdn.reactbin.juggalol.com/xK7mN2pQ",
|
||||
"thumbnail_url": "https://cdn.reactbin.juggalol.com/xK7mN2pQ-thumb",
|
||||
"created_at": "2026-05-09T02:46:29.520296+00:00",
|
||||
"tags": ["kfc"]
|
||||
}
|
||||
```
|
||||
|
||||
**Constraints**:
|
||||
- `short_id`: exactly 8 alphanumeric characters `[a-zA-Z0-9]{8}`
|
||||
- `storage_key`: equals `short_id` (post-migration)
|
||||
- `thumbnail_key`: equals `{short_id}-thumb` or `null` if no thumbnail exists
|
||||
- `file_url`: `{cdn_base}/{short_id}` when CDN is configured; `/api/v1/images/{short_id}/file` otherwise
|
||||
- `thumbnail_url`: `{cdn_base}/{short_id}-thumb` or `null`
|
||||
|
||||
---
|
||||
|
||||
## Route Changes
|
||||
|
||||
All routes that previously accepted `{image_id}` as a UUID now accept `{short_id}` as an 8-character alphanumeric string.
|
||||
|
||||
### GET /api/v1/images/{short_id}
|
||||
|
||||
Fetch a single image by short ID.
|
||||
|
||||
- **Path param**: `short_id` — 8-char alphanumeric string
|
||||
- **Response 200**: ImageRecord
|
||||
- **Response 404**: `{"detail": "Image not found", "code": "image_not_found"}`
|
||||
- **Response 422**: `{"detail": "Invalid image ID", "code": "invalid_short_id"}` if param is not 8 alphanumeric chars
|
||||
|
||||
### PATCH /api/v1/images/{short_id}/tags
|
||||
|
||||
Update tags on an image. Auth required.
|
||||
|
||||
- **Path param**: `short_id` — 8-char alphanumeric string
|
||||
- **Body**: `{"tags": ["tag1", "tag2"]}`
|
||||
- **Response 200**: ImageRecord (updated)
|
||||
- **Response 404/422**: same shape as above
|
||||
|
||||
### DELETE /api/v1/images/{short_id}
|
||||
|
||||
Delete an image and its storage objects. Auth required.
|
||||
|
||||
- **Path param**: `short_id` — 8-char alphanumeric string
|
||||
- **Response 204**: no body
|
||||
- **Response 404**: error envelope
|
||||
|
||||
### GET /api/v1/images/{short_id}/file
|
||||
|
||||
Serve the raw image file (proxy mode, when CDN is not configured).
|
||||
|
||||
- **Path param**: `short_id`
|
||||
- **Response 200**: raw image bytes with correct `Content-Type`
|
||||
|
||||
### GET /api/v1/images/{short_id}/thumbnail
|
||||
|
||||
Serve the thumbnail (proxy mode).
|
||||
|
||||
- **Path param**: `short_id`
|
||||
- **Response 200**: WebP bytes or original image if no thumbnail
|
||||
|
||||
### POST /api/v1/images (upload — unchanged route, updated response)
|
||||
|
||||
- **Response**: ImageRecord with `short_id` populated
|
||||
|
||||
---
|
||||
|
||||
## Frontend Route Change
|
||||
|
||||
| Old route | New route |
|
||||
|-----------------|--------------|
|
||||
| `/images/:id` | `/i/:id` |
|
||||
|
||||
The `:id` segment now contains the `short_id` value (8 alphanumeric chars) rather than a UUID.
|
||||
|
||||
---
|
||||
|
||||
## ImageRecord TypeScript Interface (updated)
|
||||
|
||||
```typescript
|
||||
export interface ImageRecord {
|
||||
id: string; // UUID — retained, not used for routing
|
||||
short_id: string; // NEW — 8-char base62, used for all routing and API calls
|
||||
hash: string;
|
||||
filename: string;
|
||||
mime_type: string;
|
||||
size_bytes: number;
|
||||
width: number;
|
||||
height: number;
|
||||
storage_key: string;
|
||||
thumbnail_key: string | null;
|
||||
file_url: string;
|
||||
thumbnail_url: string | null;
|
||||
created_at: string;
|
||||
tags: string[];
|
||||
duplicate?: boolean;
|
||||
}
|
||||
```
|
||||
77
specs/017-short-id-migration/data-model.md
Normal file
77
specs/017-short-id-migration/data-model.md
Normal file
@@ -0,0 +1,77 @@
|
||||
# Data Model: Short Image IDs
|
||||
|
||||
## Changed Entity: Image
|
||||
|
||||
### New Column
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
|------------|--------------|------------------------------|-------------------------------------------|
|
||||
| `short_id` | VARCHAR(8) | UNIQUE, NOT NULL (post-migration), INDEX | Base62 alphanumeric, 8 characters |
|
||||
|
||||
### Updated Columns (values change, types unchanged)
|
||||
|
||||
| Column | Old values | New values |
|
||||
|-----------------|-----------------------------------------|-----------------------------------|
|
||||
| `storage_key` | SHA-256 hash (64 hex chars) | short_id (8 base62 chars) |
|
||||
| `thumbnail_key` | `{hash}-thumb` (69 chars) | `{short_id}-thumb` (13 chars) |
|
||||
|
||||
### Unchanged Columns
|
||||
|
||||
| Column | Notes |
|
||||
|------------|-----------------------------------------------------------------------|
|
||||
| `id` | UUID primary key — unchanged, retained as internal identifier |
|
||||
| `hash` | SHA-256 content hash — unchanged, still used for deduplication |
|
||||
| `filename` | Unchanged |
|
||||
| `mime_type`| Unchanged |
|
||||
| `size_bytes`, `width`, `height` | Unchanged |
|
||||
| `created_at` | Unchanged |
|
||||
|
||||
### Validation Rules
|
||||
|
||||
- `short_id`: exactly 8 characters, matching `[a-zA-Z0-9]{8}` — generated on insert, never updated
|
||||
- `short_id` must be unique across all image records
|
||||
- On collision (rare), a new value is generated and retried (up to 10 attempts)
|
||||
|
||||
---
|
||||
|
||||
## Alembic Migrations
|
||||
|
||||
### Migration 003 — Add `short_id` column (nullable)
|
||||
|
||||
```
|
||||
ALTER TABLE images ADD COLUMN short_id VARCHAR(8) NULL;
|
||||
CREATE UNIQUE INDEX ix_images_short_id ON images (short_id);
|
||||
```
|
||||
|
||||
Run immediately on deploy. Existing rows get `short_id = NULL`. New uploads will populate `short_id` on insert (application-level).
|
||||
|
||||
### Migration Script — Backfill existing rows
|
||||
|
||||
`api/scripts/migrate_to_short_ids.py`
|
||||
|
||||
For each image where `short_id IS NULL`:
|
||||
1. Generate 8-char base62 short_id (retry on collision)
|
||||
2. Copy storage object: `{hash}` → `{short_id}` (S3 copy)
|
||||
3. Copy thumbnail if present: `{hash}-thumb` → `{short_id}-thumb`
|
||||
4. Verify new objects exist (S3 head_object)
|
||||
5. Update DB row: `short_id = {short_id}`, `storage_key = {short_id}`, `thumbnail_key = {short_id}-thumb` (or NULL)
|
||||
6. Delete old storage objects
|
||||
|
||||
### Migration 004 — Add NOT NULL constraint
|
||||
|
||||
```
|
||||
ALTER TABLE images ALTER COLUMN short_id SET NOT NULL;
|
||||
```
|
||||
|
||||
Run only after the migration script completes successfully with zero `short_id IS NULL` rows remaining.
|
||||
|
||||
---
|
||||
|
||||
## Storage Object Naming Convention
|
||||
|
||||
| Object type | Key pattern | Example |
|
||||
|-------------|---------------------|-------------------|
|
||||
| Original | `{short_id}` | `xK7mN2pQ` |
|
||||
| Thumbnail | `{short_id}-thumb` | `xK7mN2pQ-thumb` |
|
||||
|
||||
No folder structure. Flat bucket layout (unchanged from current convention).
|
||||
198
specs/017-short-id-migration/plan.md
Normal file
198
specs/017-short-id-migration/plan.md
Normal file
@@ -0,0 +1,198 @@
|
||||
# Implementation Plan: Short Image IDs
|
||||
|
||||
**Branch**: `017-short-id-migration` | **Date**: 2026-05-09 | **Spec**: [spec.md](spec.md)
|
||||
**Input**: Feature specification from `specs/017-short-id-migration/spec.md`
|
||||
|
||||
## Summary
|
||||
|
||||
Replace hash-based storage keys and UUID-based URL routing with 8-character base62 short IDs. The short ID becomes the canonical identifier in URLs (`/i/:short_id`), storage keys (`{short_id}` / `{short_id}-thumb`), and all API responses. Hash-based deduplication is preserved unchanged. A Python migration script handles existing images: generates short IDs, copies storage objects to new keys, updates DB records, deletes old keys.
|
||||
|
||||
## Technical Context
|
||||
|
||||
**Language/Version**: Python 3.12+ (API), TypeScript strict (UI)
|
||||
**Primary Dependencies**: FastAPI, SQLAlchemy 2.x async, Alembic, aiobotocore/boto3, Angular (latest stable)
|
||||
**Storage**: PostgreSQL (DB), S3-compatible via boto3 (MinIO local / CDN in prod)
|
||||
**Testing**: pytest + pytest-asyncio (API unit + integration), Karma/Jasmine (Angular)
|
||||
**Target Platform**: Linux server (k3s), browser SPA
|
||||
**Project Type**: Web application (FastAPI API + Angular SPA)
|
||||
**Performance Goals**: Migration script should process all existing images without timeout; no user-facing performance change
|
||||
**Constraints**: Migration must be idempotent; no data loss; copy-before-delete for all storage operations
|
||||
**Scale/Scope**: Personal collection (~hundreds to low thousands of images); collision probability negligible
|
||||
|
||||
## Constitution Check
|
||||
|
||||
*GATE: Must pass before implementation.*
|
||||
|
||||
| Principle | Status | Notes |
|
||||
|-----------|--------|-------|
|
||||
| §2.5 DB abstraction — all queries through repos | ✅ PASS | New `get_by_short_id()` added to `ImageRepository`; no raw SQL outside repo |
|
||||
| §2.6 No speculative abstraction | ✅ PASS | `generate_short_id()` is a concrete utility; no new interfaces |
|
||||
| §3.1 Routes prefixed `/api/v1/` | ✅ PASS | All routes remain under `/api/v1/images/` |
|
||||
| §3.1 Adding fields is non-breaking | ✅ PASS | `short_id` is additive; `id` UUID retained |
|
||||
| §4.2 Images immutable after upload | ✅ PASS | File content is copied, not replaced; the operation changes the storage key, not the bytes |
|
||||
| §4.3 Deduplication by content hash | ✅ PASS | `hash` column retained; `get_by_hash` unchanged |
|
||||
| §5.1 Tests alongside every implementation task | ✅ PASS | Each task includes tests |
|
||||
| §5.2 Integration tests use real PostgreSQL + MinIO | ✅ PASS | Existing integration test infrastructure reused |
|
||||
| §8 Scope boundaries | ✅ PASS | No multi-user, no public sharing feature, no OR/NOT tag logic |
|
||||
|
||||
**No violations. Implementation may proceed.**
|
||||
|
||||
## Project Structure
|
||||
|
||||
### Documentation (this feature)
|
||||
|
||||
```text
|
||||
specs/017-short-id-migration/
|
||||
├── plan.md ← this file
|
||||
├── research.md ← short ID generation, migration strategy
|
||||
├── data-model.md ← Image schema changes, Alembic migrations
|
||||
├── contracts/
|
||||
│ └── image-api.md ← updated ImageRecord schema, route changes
|
||||
├── quickstart.md ← manual test scenarios
|
||||
└── tasks.md ← generated by /speckit-tasks
|
||||
```
|
||||
|
||||
### Source Code Changes
|
||||
|
||||
```text
|
||||
api/
|
||||
├── app/
|
||||
│ ├── models.py # Add Image.short_id column
|
||||
│ ├── utils.py # Add generate_short_id()
|
||||
│ ├── repositories/
|
||||
│ │ └── image_repo.py # Add get_by_short_id(), update create()
|
||||
│ └── routers/
|
||||
│ └── images.py # Path params uuid→str, add short_id to response
|
||||
├── alembic/versions/
|
||||
│ ├── 003_add_short_id.py # ADD COLUMN short_id VARCHAR(8) NULLABLE UNIQUE
|
||||
│ └── 004_short_id_not_null.py # SET NOT NULL (run after migration script)
|
||||
├── scripts/
|
||||
│ └── migrate_to_short_ids.py # Backfill existing images
|
||||
└── tests/
|
||||
├── unit/
|
||||
│ ├── test_hashing.py # Add generate_short_id() tests
|
||||
│ ├── test_url_construction.py # Update mock images to include short_id
|
||||
│ └── test_short_id.py # NEW: collision retry, charset validation
|
||||
└── integration/
|
||||
├── test_upload.py # Assert short_id in response
|
||||
├── test_search.py # Update {id} → {short_id} in route calls
|
||||
├── test_delete.py # Update route params
|
||||
├── test_serving.py # Update route params
|
||||
└── test_tags.py # Update route params
|
||||
|
||||
ui/src/app/
|
||||
├── app.routes.ts # 'images/:id' → 'i/:id'
|
||||
├── services/
|
||||
│ └── image.service.ts # Add short_id to ImageRecord, update service calls
|
||||
├── library/
|
||||
│ └── library.component.ts # Navigate to ['/i', img.short_id]
|
||||
├── upload/
|
||||
│ └── upload.component.ts # Navigate to ['/i', res.short_id] after upload
|
||||
└── detail/
|
||||
└── detail.component.ts # (no route change needed; reads :id param same way)
|
||||
```
|
||||
|
||||
**Structure Decision**: Existing web application layout. API changes are concentrated in models, repository, router, and a new migration script. UI changes are confined to routes, image service interface, and two navigation calls.
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1: Backend — Short ID Infrastructure
|
||||
|
||||
1. Add `generate_short_id()` to `api/app/utils.py`
|
||||
- Base62 charset: `string.ascii_letters + string.digits`
|
||||
- Uses `secrets.choice` for cryptographic randomness
|
||||
- Returns 8-character string
|
||||
|
||||
2. Add Alembic migration `003_add_short_id.py`
|
||||
- `ADD COLUMN short_id VARCHAR(8) NULL`
|
||||
- `CREATE UNIQUE INDEX ix_images_short_id ON images (short_id)`
|
||||
|
||||
3. Update `api/app/models.py`
|
||||
- Add `short_id: Mapped[str | None] = mapped_column(String(8), unique=True, nullable=True, index=True)`
|
||||
|
||||
4. Update `api/app/repositories/image_repo.py`
|
||||
- Add `get_by_short_id(short_id: str) -> Image | None`
|
||||
- Update `create()` to accept and persist `short_id` parameter
|
||||
|
||||
5. Update `api/app/routers/images.py`
|
||||
- Change all `image_id: uuid.UUID` path params to `short_id: str`
|
||||
- Add `_validate_short_id(short_id: str)` helper (8 alphanumeric chars, else 422)
|
||||
- Replace `get_by_id` calls with `get_by_short_id`
|
||||
- Update `_image_to_dict` to include `"short_id": image.short_id` in response
|
||||
- Update upload handler: generate `short_id` with collision retry, use as storage key
|
||||
|
||||
### Phase 2: Migration Script
|
||||
|
||||
`api/scripts/migrate_to_short_ids.py`:
|
||||
|
||||
```
|
||||
for each image where short_id IS NULL:
|
||||
generate short_id (retry on DB collision)
|
||||
copy {hash} → {short_id} in storage
|
||||
if thumbnail_key IS NOT NULL:
|
||||
copy {hash}-thumb → {short_id}-thumb in storage
|
||||
verify new objects exist (head_object)
|
||||
UPDATE images SET short_id={sid}, storage_key={sid}, thumbnail_key={sid}-thumb WHERE id={id}
|
||||
delete {hash} from storage
|
||||
if thumbnail_key was not null:
|
||||
delete {hash}-thumb from storage
|
||||
log: "migrated {id} → {short_id}"
|
||||
|
||||
print summary: N migrated, M skipped (already had short_id)
|
||||
```
|
||||
|
||||
After script runs with 0 remaining `NULL` short_ids, apply migration `004_short_id_not_null.py`.
|
||||
|
||||
### Phase 3: Frontend
|
||||
|
||||
1. `app.routes.ts`: `path: 'images/:id'` → `path: 'i/:id'`
|
||||
2. `image.service.ts`: add `short_id: string` to `ImageRecord`
|
||||
3. `library.component.ts`: `router.navigate(['/images', img.id])` → `router.navigate(['/i', img.short_id])`
|
||||
4. `upload.component.ts`: `router.navigate(['/images', res.id])` → `router.navigate(['/i', res.short_id])`
|
||||
|
||||
### Phase 4: Polish
|
||||
|
||||
- Update all existing API integration tests to use `short_id` in route paths
|
||||
- Run `ng lint` and `ruff check` across modified files
|
||||
- Verify `ng build --configuration production` succeeds
|
||||
- Run full test suites: `make test-unit && make test-integration`
|
||||
|
||||
## Key Implementation Notes
|
||||
|
||||
### Collision Retry Pattern (upload)
|
||||
|
||||
```python
|
||||
MAX_RETRIES = 10
|
||||
for attempt in range(MAX_RETRIES):
|
||||
short_id = generate_short_id()
|
||||
try:
|
||||
image = await image_repo.create(..., short_id=short_id)
|
||||
break
|
||||
except IntegrityError: # short_id collision
|
||||
await db.rollback()
|
||||
if attempt == MAX_RETRIES - 1:
|
||||
raise RuntimeError("Could not generate unique short_id")
|
||||
```
|
||||
|
||||
### Route Validation
|
||||
|
||||
```python
|
||||
import re
|
||||
_SHORT_ID_RE = re.compile(r'^[a-zA-Z0-9]{8}$')
|
||||
|
||||
def _validate_short_id(short_id: str) -> None:
|
||||
if not _SHORT_ID_RE.match(short_id):
|
||||
raise HTTPException(422, detail={"detail": "Invalid image ID", "code": "invalid_short_id"})
|
||||
```
|
||||
|
||||
### `_image_to_dict` Update
|
||||
|
||||
Add `"short_id": image.short_id` to the returned dict. The `file_url` and `thumbnail_url` generation already uses `image.storage_key` which will now equal `image.short_id` — no formula change needed.
|
||||
|
||||
### Migration Script Entry Point
|
||||
|
||||
```bash
|
||||
cd api && python -m scripts.migrate_to_short_ids
|
||||
```
|
||||
|
||||
Reads DB URL and storage config from environment variables (same as the application).
|
||||
73
specs/017-short-id-migration/quickstart.md
Normal file
73
specs/017-short-id-migration/quickstart.md
Normal file
@@ -0,0 +1,73 @@
|
||||
# Quickstart: Short Image IDs
|
||||
|
||||
## Scenario 1 — Happy Path: New Upload Gets Short ID
|
||||
|
||||
1. Log in and navigate to Upload.
|
||||
2. Upload any image.
|
||||
3. Observe: browser navigates to `/i/AbCdEfGh` (8-char short ID, not a UUID).
|
||||
4. Copy the URL from the address bar and paste in a new tab — image loads correctly.
|
||||
5. Open the URL in a private/incognito window (not logged in) — image still loads.
|
||||
|
||||
**Pass criteria**: URL is `/i/{8 alphanumeric chars}`, image loads authenticated and unauthenticated.
|
||||
|
||||
---
|
||||
|
||||
## Scenario 2 — Deduplication Still Works
|
||||
|
||||
1. Upload any image — note the short ID in the URL.
|
||||
2. Upload the exact same file again.
|
||||
3. Observe: API returns `duplicate: true`, browser navigates to the same short ID URL as step 1.
|
||||
|
||||
**Pass criteria**: No second record created, same short ID returned.
|
||||
|
||||
---
|
||||
|
||||
## Scenario 3 — Library Navigation Uses Short IDs
|
||||
|
||||
1. Open the library (`/`).
|
||||
2. Click any image card.
|
||||
3. Observe: navigated to `/i/{short_id}`, not `/images/{uuid}`.
|
||||
|
||||
**Pass criteria**: All image card clicks navigate to `/i/` routes.
|
||||
|
||||
---
|
||||
|
||||
## Scenario 4 — Tag and Delete Operations Work via Short ID
|
||||
|
||||
1. Open an image detail page at `/i/{short_id}`.
|
||||
2. If logged in: add a tag, remove a tag — confirm both succeed.
|
||||
3. If logged in: delete the image — confirm navigates back to library, image no longer appears.
|
||||
|
||||
**Pass criteria**: Tag updates and delete work correctly when the route uses a short ID.
|
||||
|
||||
---
|
||||
|
||||
## Scenario 5 — Migration: All Existing Images Accessible
|
||||
|
||||
1. After running the migration script: open the library.
|
||||
2. Click through several images from before the migration.
|
||||
3. Observe: all navigate to `/i/{short_id}` URLs, all images and thumbnails load.
|
||||
4. No broken image placeholders visible.
|
||||
|
||||
**Pass criteria**: 100% of pre-migration images accessible via short ID with no broken assets.
|
||||
|
||||
---
|
||||
|
||||
## Scenario 6 — Migration Script Is Idempotent
|
||||
|
||||
1. Run the migration script once — note how many images were migrated.
|
||||
2. Run the migration script a second time.
|
||||
3. Observe: script reports 0 images migrated (all already have short IDs), exits cleanly.
|
||||
|
||||
**Pass criteria**: Second run produces no DB changes, no storage operations, no errors.
|
||||
|
||||
---
|
||||
|
||||
## Scenario 7 — Copy URL Button Copies Short Page URL
|
||||
|
||||
1. Open any image detail page at `/i/{short_id}`.
|
||||
2. Click "Copy URL".
|
||||
3. Paste into a text editor.
|
||||
4. Observe: pasted value is the CDN file URL (e.g. `https://cdn.reactbin.juggalol.com/xK7mN2pQ`), not a UUID-based URL.
|
||||
|
||||
**Pass criteria**: Copied URL contains the short_id, not a UUID.
|
||||
56
specs/017-short-id-migration/research.md
Normal file
56
specs/017-short-id-migration/research.md
Normal file
@@ -0,0 +1,56 @@
|
||||
# Research: Short Image IDs
|
||||
|
||||
## Short ID Generation
|
||||
|
||||
**Decision**: Use `secrets.choice` over `string.ascii_letters + string.digits` (base62, 62 characters), 8 characters long.
|
||||
|
||||
**Rationale**: `secrets.choice` is cryptographically random, eliminating any bias from modular reduction that affects simpler approaches. Base62 (a–z, A–Z, 0–9) is URL-safe without percent-encoding. 8 characters gives 62⁸ ≈ 218 trillion combinations — negligible collision probability even at millions of images.
|
||||
|
||||
**Alternatives considered**:
|
||||
- `secrets.token_urlsafe(6)` — includes `-` and `_`, not pure alphanumeric
|
||||
- UUID truncation (first 8 chars of hex) — only 16 chars of alphabet (hex), dramatically fewer combinations than base62
|
||||
- nanoid (npm) — JavaScript library, requires a separate dependency for Python
|
||||
|
||||
**Collision retry**: On insert, if a `UniqueConstraint` violation is raised on `short_id`, generate a new one and retry (up to a configurable limit, e.g., 10 attempts). At 10,000 images the per-attempt collision probability is ~4.6 × 10⁻¹¹; retries are a pure safety measure.
|
||||
|
||||
---
|
||||
|
||||
## Alembic Two-Phase Migration Strategy
|
||||
|
||||
**Decision**: Two separate Alembic migrations (003 + 004), with the Python migration script run between them.
|
||||
|
||||
**Rationale**: The `short_id` column must start nullable so existing rows can be inserted without a value. The migration script fills all existing rows. Once confirmed, a second migration adds the NOT NULL constraint. Running both as one migration would require a complex inline Python script in Alembic (fragile, untestable). Two migrations with a script in between is the standard approach for backfill + constraint change.
|
||||
|
||||
**Migration 003**: `ADD COLUMN short_id VARCHAR(8) NULL UNIQUE` + GiST/B-tree index.
|
||||
**Script**: Fill all rows, idempotent (skip rows where `short_id IS NOT NULL`).
|
||||
**Migration 004**: `ALTER COLUMN short_id SET NOT NULL`.
|
||||
|
||||
---
|
||||
|
||||
## Storage Object Copy Strategy
|
||||
|
||||
**Decision**: Copy-then-verify-then-delete (not atomic rename). Using the MinIO/S3 `copy_object` API followed by a `delete_object` call.
|
||||
|
||||
**Rationale**: S3-compatible object stores do not support atomic renames. The safe approach is: copy to new key, verify new object exists (head_object), update DB, delete old key. If interrupted after copy but before delete, the old object remains — wasted storage but no data loss. The migration is idempotent: if `short_id` is already set on a row, the script skips it.
|
||||
|
||||
**Alternatives considered**:
|
||||
- `mc mv` (MinIO client CLI) — simpler but harder to script transactionally with DB updates
|
||||
- Direct Python with `aiobotocore` — chosen; same library already used by the storage backend
|
||||
|
||||
---
|
||||
|
||||
## API Route Parameter Change
|
||||
|
||||
**Decision**: Change all image route parameters from `image_id: uuid.UUID` to `short_id: str` with manual length/charset validation.
|
||||
|
||||
**Rationale**: FastAPI's `uuid.UUID` type annotation rejects non-UUID strings at the path-parsing stage, so the existing routes cannot accept short IDs without a type change. Switching to `str` with a custom validator (8 alphanumeric chars) is minimal and clear.
|
||||
|
||||
**Impact**: All routes under `/api/v1/images/{id}` change to accept an 8-char string. The `id` field in API responses is retained as the UUID; `short_id` is added as a new field. The UI switches to using `short_id` for all navigation and API calls.
|
||||
|
||||
---
|
||||
|
||||
## Response Schema: Additive Change
|
||||
|
||||
**Decision**: Add `short_id` as a new field to the image response dict. The existing `id` (UUID) field is retained.
|
||||
|
||||
**Rationale**: Adding a field is non-breaking per §3.1. Removing `id` would be a breaking change. Retaining both allows any internal tooling or API consumers that already use `id` to continue working. The UI transitions to using `short_id` for routing and API calls, but the UUID remains queryable if needed.
|
||||
104
specs/017-short-id-migration/spec.md
Normal file
104
specs/017-short-id-migration/spec.md
Normal file
@@ -0,0 +1,104 @@
|
||||
# Feature Specification: Short Image IDs
|
||||
|
||||
**Feature Branch**: `017-short-id-migration`
|
||||
**Created**: 2026-05-09
|
||||
**Status**: Draft
|
||||
**Input**: User description: "Replace UUID-based image identifiers with 8-character base62 short IDs. Short IDs become the canonical identifier in URLs (/i/:short_id replacing /images/:uuid), MinIO storage keys, and all API responses. Existing hash-based deduplication is preserved. Migration includes backfilling short IDs for existing images, renaming storage objects, and regenerating file URLs. Frontend routes update to use short IDs throughout."
|
||||
|
||||
## User Scenarios & Testing *(mandatory)*
|
||||
|
||||
### User Story 1 — Clean, Shareable Image Links (Priority: P1)
|
||||
|
||||
A user wants to share an image with someone. They copy the page URL or use the "Copy URL" button and get a short, clean link they can paste anywhere. The link is brief enough to share in a message without looking like machine-generated noise.
|
||||
|
||||
**Why this priority**: This is the primary user-facing value of the change. Every image in the library benefits immediately. Short links are more trustworthy, easier to share, and less likely to break in messaging apps that truncate long URLs.
|
||||
|
||||
**Independent Test**: Open any image detail page. Confirm the URL in the browser address bar is short (e.g. `/i/AbCdEfGh`). Copy the URL and paste it into a new tab — confirm the correct image loads. Share the link with someone who is not logged in and confirm they can view the image.
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** a user is on the image detail page, **When** they look at the browser address bar, **Then** the URL contains a short 8-character identifier rather than a long UUID.
|
||||
2. **Given** a short image URL, **When** an unauthenticated user opens it, **Then** the image loads correctly without requiring login.
|
||||
3. **Given** a short image URL, **When** it is pasted into a messaging app or email, **Then** it is compact enough to read at a glance and does not get truncated.
|
||||
|
||||
---
|
||||
|
||||
### User Story 2 — New Uploads Assigned Short IDs (Priority: P2)
|
||||
|
||||
When a new image is uploaded, the system assigns it a short ID immediately. The image is accessible via its short URL straight away. If the same file has already been uploaded before, the existing record is returned rather than creating a duplicate — the deduplication behaviour is unchanged.
|
||||
|
||||
**Why this priority**: This ensures the new convention is in place going forward. Without this, the migration work in US3 would need to be re-run for any new uploads.
|
||||
|
||||
**Independent Test**: Upload a new image. Confirm the detail page URL contains an 8-character short ID. Upload the exact same file again — confirm no new record is created and the existing short URL is returned.
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** a user uploads an image, **When** the upload completes, **Then** the image is accessible at a short URL (`/i/{short_id}`).
|
||||
2. **Given** a user uploads a file that is identical to a previously uploaded image, **When** the upload completes, **Then** the system returns the existing image's short URL rather than creating a duplicate entry.
|
||||
3. **Given** a newly uploaded image, **When** the "Copy URL" button is used, **Then** the copied link is the short image page URL.
|
||||
|
||||
---
|
||||
|
||||
### User Story 3 — All Existing Images Migrated to Short IDs (Priority: P3)
|
||||
|
||||
All images that existed before this change are assigned short IDs and remain fully accessible. Their stored files are renamed to match the new convention. After migration, all image links throughout the application use short IDs — no UUID-based links remain active.
|
||||
|
||||
**Why this priority**: Without migration, legacy images would either be inaccessible or require maintaining two parallel URL schemes. Clean cutover is preferable. This is lower priority than P1/P2 because it is an administrative operation rather than a user-facing feature, but it must complete before the feature can be considered fully shipped.
|
||||
|
||||
**Independent Test**: After running the migration, browse the library and open several images — confirm all detail pages use short URLs. Confirm no broken images or missing thumbnails.
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** images that existed before the migration, **When** the migration completes, **Then** all are accessible via short URLs.
|
||||
2. **Given** the migration has run, **When** a user browses the library and opens any image, **Then** the detail page URL is a short ID URL.
|
||||
3. **Given** the migration has run, **When** any image or thumbnail is displayed, **Then** it loads correctly with no broken images.
|
||||
4. **Given** the migration is running, **When** it encounters an error on one image, **Then** it reports the failure clearly and continues processing remaining images rather than aborting entirely.
|
||||
|
||||
---
|
||||
|
||||
### Edge Cases
|
||||
|
||||
- What happens if a short ID collision occurs during generation? The system must retry with a new ID rather than failing or overwriting an existing image.
|
||||
- What happens if a record lacks a short ID but the file content is unchanged? The migration assigns a new short ID without re-uploading the file.
|
||||
- What happens if the migration is interrupted partway through? Already-migrated images remain accessible; un-migrated images are identifiable so the migration can be re-run safely.
|
||||
- What happens if a thumbnail does not exist for an image (e.g., GIFs where generation failed)? The migration skips the thumbnail rename for that record and continues.
|
||||
- What happens if a user has bookmarked a UUID-based URL before the migration? Those URLs become invalid; this is acceptable for a personal tool with no external consumers.
|
||||
|
||||
## Requirements *(mandatory)*
|
||||
|
||||
### Functional Requirements
|
||||
|
||||
- **FR-001**: The system MUST assign every image a unique 8-character short ID composed of alphanumeric characters (a–z, A–Z, 0–9).
|
||||
- **FR-002**: Every image detail page MUST be accessible at the path `/i/{short_id}`.
|
||||
- **FR-003**: The UUID-based image detail route (`/images/{uuid}`) MUST be retired; short ID routes are the sole canonical paths.
|
||||
- **FR-004**: Image storage objects (original and thumbnail) MUST use the short ID as their storage key, following flat naming: `{short_id}` for the original and `{short_id}-thumb` for the thumbnail.
|
||||
- **FR-005**: The publicly accessible image file URL and thumbnail URL MUST reflect the new storage key names.
|
||||
- **FR-006**: On upload, the system MUST check whether an identical file (by hash) already exists and return the existing record rather than creating a duplicate, regardless of short IDs.
|
||||
- **FR-007**: The system MUST generate a new short ID on upload, retrying automatically if a collision with an existing ID is detected.
|
||||
- **FR-008**: A migration process MUST assign short IDs to all existing images that do not have one, rename their storage objects to match the new keys, and update all stored URLs.
|
||||
- **FR-009**: The migration MUST be re-runnable safely — images already migrated MUST be skipped rather than processed again.
|
||||
- **FR-010**: All application links that reference images (library grid, detail page, API responses) MUST use short IDs after the migration.
|
||||
|
||||
### Key Entities
|
||||
|
||||
- **Image**: Each image has a unique short ID (8 alphanumeric characters) that serves as its canonical identifier in URLs, storage, and API responses. The image retains its content hash for deduplication. The short ID is independent of the hash.
|
||||
- **Storage Object**: Each image has two storage objects — an original and a thumbnail — named using the short ID (`{short_id}` and `{short_id}-thumb`). Flat naming, no folder structure.
|
||||
|
||||
## Success Criteria *(mandatory)*
|
||||
|
||||
### Measurable Outcomes
|
||||
|
||||
- **SC-001**: All image detail page URLs use an 8-character alphanumeric identifier at `/i/{short_id}`.
|
||||
- **SC-002**: 100% of existing images are accessible via short URL after migration completes, with no broken images or missing thumbnails.
|
||||
- **SC-003**: Uploading the same file twice produces one record — deduplication rate remains 100% for identical files.
|
||||
- **SC-004**: The migration completes without data loss — no image file or thumbnail is deleted before its renamed copy is confirmed present in storage.
|
||||
- **SC-005**: The migration is idempotent — running it a second time produces no changes and no errors.
|
||||
|
||||
## Assumptions
|
||||
|
||||
- UUID-based image URLs do not need to remain accessible after migration; this is a personal tool with no external consumers relying on the old URL structure.
|
||||
- The migration will be run manually by the operator as a one-time administrative step; it does not need to be triggered from the UI.
|
||||
- Storage object renaming is implemented as copy-then-delete to avoid data loss if the process is interrupted mid-run.
|
||||
- The short ID character set is base62 (a–z, A–Z, 0–9); no special characters, ensuring URL-safe identifiers without percent-encoding.
|
||||
- The `hash` column is retained and continues to be used for deduplication; it is not removed as part of this change.
|
||||
- Thumbnails may not exist for all images (e.g., some GIFs); the migration handles missing thumbnails gracefully by skipping the thumbnail rename for those records.
|
||||
162
specs/017-short-id-migration/tasks.md
Normal file
162
specs/017-short-id-migration/tasks.md
Normal file
@@ -0,0 +1,162 @@
|
||||
# Tasks: Short Image IDs
|
||||
|
||||
**Input**: Design documents from `specs/017-short-id-migration/`
|
||||
**Prerequisites**: plan.md ✅, spec.md ✅, research.md ✅, data-model.md ✅, contracts/image-api.md ✅, quickstart.md ✅
|
||||
|
||||
**Tests**: Tests accompany each implementation task per §5.1. All API changes are in `api/`, all UI changes are in `ui/src/app/`.
|
||||
|
||||
**Organization**: The foundational phase (Phase 1) must complete before any user story work begins — it adds the `short_id` column, model field, utility function, and repository method that all three user stories depend on. US1 and US2 can then proceed; US3 (migration script) follows last because it operates on the fully wired system.
|
||||
|
||||
## Format: `[ID] [P?] [Story] Description`
|
||||
|
||||
- **[P]**: Can run in parallel (different files, no dependencies)
|
||||
- **[Story]**: Which user story this task belongs to
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Foundational — Short ID Infrastructure (Blocks All User Stories)
|
||||
|
||||
**Goal**: Add the `short_id` column to the database, the model, a generation utility, and a repository lookup. Every user story depends on these.
|
||||
|
||||
**Independent Test**: After this phase, `generate_short_id()` can be called from a Python shell and returns an 8-character alphanumeric string. `alembic upgrade head` applies migration 003 cleanly. A manually inserted image with a `short_id` can be fetched by `image_repo.get_by_short_id()` in a test.
|
||||
|
||||
- [X] T001 Write failing unit tests for `generate_short_id()` in `api/tests/unit/test_hashing.py`: (1) returns exactly 8 characters; (2) contains only `[a-zA-Z0-9]` characters; (3) two consecutive calls return different values (collision test); (4) function exists and is importable from `app.utils`. Run `make test-unit` and confirm new tests FAIL.
|
||||
|
||||
- [X] T002 Add `generate_short_id()` to `api/app/utils.py`: import `secrets` and `string`; define `BASE62 = string.ascii_letters + string.digits`; implement `def generate_short_id(length: int = 8) -> str: return ''.join(secrets.choice(BASE62) for _ in range(length))`. Run `make test-unit` and confirm T001 tests pass.
|
||||
|
||||
- [X] T003 Create Alembic migration `api/alembic/versions/003_add_short_id.py`: `op.add_column('images', sa.Column('short_id', sa.String(8), nullable=True))`; `op.create_index('ix_images_short_id', 'images', ['short_id'], unique=True)`. Downgrade removes index then column. Run `alembic upgrade head` in the api container and confirm migration applies cleanly.
|
||||
|
||||
- [X] T004 [P] Add `short_id` field to `Image` model in `api/app/models.py`: `short_id: Mapped[str | None] = mapped_column(String(8), unique=True, nullable=True, index=True)`. No change to column sizes for `storage_key` (keep String(64)) or `thumbnail_key` (keep String(70)) — values will simply be shorter after migration.
|
||||
|
||||
- [X] T005 Update `api/app/repositories/image_repo.py`: (a) add `async def get_by_short_id(self, short_id: str) -> Image | None` — SELECT with `Image.short_id == short_id` and `selectinload(Image.image_tags).selectinload(ImageTag.tag)`; (b) add `short_id: str` parameter to `create()` and persist it on the `Image` instance. Write a unit test in `api/tests/unit/` mocking the session to confirm `get_by_short_id` constructs the correct WHERE clause.
|
||||
|
||||
- [X] T005a Update `api/tests/integration/conftest.py`: wherever test fixtures call `image_repo.create()` or insert image rows directly, add `short_id=generate_short_id()` (import `generate_short_id` from `app.utils`). This ensures all integration test fixture images have a `short_id` value so that tests referencing `image.short_id` in URLs and assertions work correctly. Run `make test-integration` and confirm existing tests still pass (no new failures introduced).
|
||||
|
||||
**Checkpoint**: Short ID infrastructure complete. The `short_id` column exists in DB, `generate_short_id()` works, and the repo can look up images by short_id.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: User Story 1 — Clean, Shareable Image Links (Priority: P1) 🎯 MVP
|
||||
|
||||
**Goal**: All API routes accept `short_id` (8-char string) instead of UUID. `short_id` appears in every API response. The frontend navigates to `/i/:short_id` and the library uses `short_id` for navigation.
|
||||
|
||||
**Independent Test**: Upload a new image (US2 must be done for end-to-end, but US1 can be tested using a fixture image with a known `short_id` inserted directly). Call `GET /api/v1/images/{short_id}` and confirm it returns the correct image with `short_id` in the response. Navigate to `/i/{short_id}` in the browser and confirm the detail page loads.
|
||||
|
||||
- [X] T006 Write failing unit tests in `api/tests/unit/test_url_construction.py`: update `_make_image()` mock to include `short_id = 'AbCd1234'`; add assertions that `_image_to_dict` result includes `"short_id": "AbCd1234"`. Write failing unit tests in `api/tests/unit/test_short_id.py`: (1) `_validate_short_id('AbCd1234')` passes; (2) `_validate_short_id('toolong!!')` raises 422; (3) `_validate_short_id('short')` raises 422; (4) `_validate_short_id('has space!')` raises 422. Run `make test-unit` and confirm new tests FAIL.
|
||||
|
||||
- [X] T007 Update `api/app/routers/images.py` — `_image_to_dict`: add `"short_id": image.short_id` to the returned dict (between `"id"` and `"hash"`). Add `_validate_short_id(short_id: str) -> None` helper: compile `re.compile(r'^[a-zA-Z0-9]{8}$')` at module level; raise `HTTPException(422, detail={"detail": "Invalid image ID", "code": "invalid_short_id"})` if no match. Run `make test-unit` and confirm T006 tests pass.
|
||||
|
||||
- [X] T008 Update all image route handlers in `api/app/routers/images.py` — change every `image_id: uuid.UUID` path parameter to `short_id: str`; call `_validate_short_id(short_id)` at the start of each handler; replace all `image_repo.get_by_id(image_id)` calls with `image_repo.get_by_short_id(short_id)`. Affected routes: `GET /images/{short_id}`, `GET /images/{short_id}/file`, `GET /images/{short_id}/thumbnail`, `PATCH /images/{short_id}/tags`, `DELETE /images/{short_id}`. Remove `import uuid` if no longer used.
|
||||
|
||||
- [X] T009 [P] Write failing Angular tests: (a) in `ui/src/app/services/image.service.ts` — update `MOCK_IMAGE` in `detail.component.spec.ts` and any other spec files to include `short_id: 'AbCd1234'`; (b) in `ui/src/app/library/library.component.spec.ts` — add test asserting that clicking an image card calls `router.navigate` with `['/i', img.short_id]` rather than `['/images', img.id]`. Run `ng test --watch=false` and confirm new tests FAIL.
|
||||
|
||||
- [X] T010 Update `ui/src/app/app.routes.ts`: change `path: 'images/:id'` to `path: 'i/:id'`. The `DetailComponent` reads `this.route.snapshot.paramMap.get('id')` — no change needed there since the param name `:id` is unchanged.
|
||||
|
||||
- [X] T011 Add `short_id: string` to the `ImageRecord` interface in `ui/src/app/services/image.service.ts`. No changes to method signatures — `get(id)`, `updateTags(id, ...)`, and `delete(id)` already accept `string`; callers will now pass `short_id` values instead of UUIDs.
|
||||
|
||||
- [X] T011a Update `ui/src/app/detail/detail.component.ts`: change `this.imageService.updateTags(this.image.id, updated)` (×2, lines ~214 and ~224) and `this.imageService.delete(this.image.id)` (line ~235) to use `this.image.short_id` instead of `this.image.id`. After T008 the API accepts only 8-char short_ids; passing a UUID will trigger a 422. Add assertions to `ui/src/app/detail/detail.component.spec.ts` confirming that `updateTags` and `delete` are called with the `short_id` value (`'AbCd1234'`) not the UUID. Run `ng test --watch=false --include="src/app/detail/detail.component.spec.ts"` and confirm new assertions pass.
|
||||
|
||||
- [X] T012 Update `ui/src/app/library/library.component.ts`: change `router.navigate(['/images', img.id])` (×2: click handler and keydown handler) to `router.navigate(['/i', img.short_id])`. Run `ng test --watch=false` and confirm T009 Angular tests pass.
|
||||
|
||||
**Checkpoint**: US1 complete. API returns `short_id` on every image response. Routes accept short IDs. Library navigates to `/i/{short_id}`.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: User Story 2 — New Uploads Assigned Short IDs (Priority: P2)
|
||||
|
||||
**Goal**: When a new image is uploaded, a short ID is generated, used as the storage key (replacing the hash), and returned in the response. Deduplication by content hash still works.
|
||||
|
||||
**Independent Test**: Upload a new image. Confirm the response includes a `short_id` field with exactly 8 alphanumeric characters. Confirm `storage_key` equals `short_id` and `thumbnail_key` equals `{short_id}-thumb`. Upload the same image again — confirm `duplicate: true` and the same `short_id` is returned.
|
||||
|
||||
- [X] T013 Write failing integration tests in `api/tests/integration/test_upload.py`: (1) upload a new image → response includes `short_id` field matching `[a-zA-Z0-9]{8}`; (2) `storage_key` in response equals `short_id`; (3) `thumbnail_key` in response equals `{short_id}-thumb` (or is null for images without thumbnails); (4) upload same file twice → second response has `duplicate: true` and identical `short_id`. Run `make test-integration` and confirm new tests FAIL.
|
||||
|
||||
- [X] T014 Update the upload handler in `api/app/routers/images.py`: after the hash duplicate check, add collision-retry loop (up to 10 attempts): `short_id = generate_short_id()`; call `await storage.put(short_id, data, mime_type)` instead of `await storage.put(hash_hex, ...)`; call `await storage.put(f"{short_id}-thumb", ...)` instead of `f"{hash_hex}-thumb"`; pass `storage_key=short_id`, `thumbnail_key=f"{short_id}-thumb"` (or None), and `short_id=short_id` to `image_repo.create()`. Catch `IntegrityError` on `create()`, rollback, retry with new short_id. Import `generate_short_id` from `app.utils` and `IntegrityError` from `sqlalchemy.exc`. Run `make test-integration` and confirm T013 tests pass.
|
||||
|
||||
- [X] T015 Update `ui/src/app/upload/upload.component.ts`: change `this.router.navigate(['/images', res.id])` to `this.router.navigate(['/i', res.short_id])`. Add a test to the upload component spec (or update the existing navigation test) asserting the route uses `short_id`.
|
||||
|
||||
**Checkpoint**: US2 complete. All new uploads produce a short ID and are immediately accessible at `/i/{short_id}`.
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: User Story 3 — All Existing Images Migrated to Short IDs (Priority: P3)
|
||||
|
||||
**Goal**: A runnable script backfills `short_id` for all pre-existing images, copies their storage objects to the new key pattern, and updates DB records. A final Alembic migration adds the NOT NULL constraint. After this phase the system has no UUID-keyed storage objects.
|
||||
|
||||
**Independent Test**: Run the migration script — confirm it prints a count of migrated images and exits cleanly. Run it a second time — confirm it reports 0 migrated (idempotent). Browse the library and open pre-migration images — confirm all load with short ID URLs and no broken images.
|
||||
|
||||
- [X] T016 Write unit tests in `api/tests/unit/test_migration.py` covering the migration script logic: (1) an image with `short_id IS NULL` is processed (short_id generated, storage copy called, DB update called, old keys deleted); (2) an image with `short_id` already set is skipped; (3) if a storage copy fails, the error is logged and the script continues to the next image (no abort); (4) the summary at the end reports correct migrated and skipped counts. Mock the storage client and DB session. Run `make test-unit` and confirm new tests FAIL (script not yet created).
|
||||
|
||||
- [X] T017 Create `api/scripts/__init__.py` (empty) and `api/scripts/migrate_to_short_ids.py`: async main function that (a) reads DB URL and storage config from env vars via `app.config.get_settings()`; (b) creates an async DB session and `S3StorageBackend` instance; (c) queries all images where `short_id IS NULL`; (d) for each: generate short_id (retry on `UniqueViolation`), copy storage object using `data = await storage.get(old_key); await storage.put(new_key, data, image.mime_type)` (the `StorageBackend` interface provides only `get`/`put`/`delete` — there is no `copy` method), verify the copy succeeded by calling `await storage.get(new_key)` and catching any exception, update the DB row (`short_id`, `storage_key`, `thumbnail_key`), then delete old keys with `await storage.delete(old_key)`; (e) skips images where `thumbnail_key IS NULL` for the thumbnail copy step; (f) wraps each image in a try/except so a single failure logs an error and continues to the next image; (g) prints `Migrated: N, Skipped: M, Failed: K` on completion. Entry point: `if __name__ == '__main__': asyncio.run(main())`. Run `make test-unit` and confirm T016 tests pass.
|
||||
|
||||
- [X] T018 Create Alembic migration `api/alembic/versions/004_short_id_not_null.py`: `op.alter_column('images', 'short_id', nullable=False)`. **Run this migration only after the migration script completes with 0 remaining NULL rows.** Downgrade sets nullable=True. Document this ordering requirement in the migration file's docstring.
|
||||
|
||||
**Checkpoint**: US3 complete. All existing images have short IDs, storage objects use new key pattern, `short_id` column is NOT NULL.
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Polish & Cross-Cutting Concerns
|
||||
|
||||
- [X] T019 Update `api/tests/integration/test_search.py`, `test_delete.py`, `test_serving.py`, `test_tags.py`, and `test_public_access.py`: wherever tests construct a URL with `f"/api/v1/images/{image.id}"` or `f"/api/v1/images/{uuid}"`, replace with `f"/api/v1/images/{image.short_id}"`. Ensure test fixtures (conftest.py) populate `short_id` on images created for testing. Run `make test-integration` and confirm all integration tests pass.
|
||||
|
||||
- [X] T020 Update `ui/src/app/detail/detail.component.spec.ts`: add `short_id: 'AbCd1234'` to `MOCK_IMAGE` and `MOCK_IMAGE_ABS` constants. Update any test assertions that check navigation targets to use `short_id`. Run `ng test --watch=false --include="src/app/detail/detail.component.spec.ts"` and confirm all tests pass.
|
||||
|
||||
- [X] T021 Run `ng lint` across all modified UI files and `ruff check api/app/ api/tests/ api/scripts/` across all modified API files; fix any issues. Confirm `ruff format --check api/` passes.
|
||||
|
||||
- [X] T022 Run `ng build --configuration production` and confirm build succeeds with no TypeScript errors. Run `make test-unit && make test-integration` and confirm all tests pass.
|
||||
|
||||
- [ ] T023 Manually verify all seven quickstart.md scenarios in the browser: (1) new upload navigates to `/i/{short_id}`; (2) deduplication returns same short_id; (3) library cards navigate to `/i/`; (4) tag and delete work via short_id; (5) pre-migration images accessible (after running script); (6) migration is idempotent; (7) "Copy URL" copies the CDN URL with short_id.
|
||||
|
||||
---
|
||||
|
||||
## Dependencies & Execution Order
|
||||
|
||||
- T001 before T002 (write failing tests before implementation)
|
||||
- T002 before T003/T004/T005 (utility must exist before migration and model reference it)
|
||||
- T003 before T004 (DB column must exist before model references it)
|
||||
- T004 before T005 (model must have `short_id` field before repo uses it)
|
||||
- T005 before T005a (conftest fixtures need the updated `create()` signature)
|
||||
- T005a before T006 (integration test fixtures must have `short_id` before any integration tests run)
|
||||
- T006 before T007 (write failing tests before implementation)
|
||||
- T007 before T008 (helper and dict update before route param changes)
|
||||
- T008 before T009/T010/T011 (API must accept short_id before frontend uses it)
|
||||
- T009 before T010/T011/T011a/T012 (write failing tests before implementation)
|
||||
- T010, T011, T011a, T012 can run in parallel (different files)
|
||||
- T011 before T011a (interface must have `short_id` field before detail component uses it)
|
||||
- T013 before T014 (write failing tests before upload changes)
|
||||
- T014 before T015 (upload must produce short_id before frontend navigation uses it)
|
||||
- T016 before T017 (write failing tests before script)
|
||||
- T017 before T018 (script must run successfully before NOT NULL migration)
|
||||
- T019–T023 after T018 (polish after all implementation complete)
|
||||
|
||||
### Execution Order Summary
|
||||
|
||||
```
|
||||
Step 1: T001 → T002 (generate_short_id: tests → implementation)
|
||||
Step 2: T003, T004 (parallel) (Alembic 003 + model update)
|
||||
Step 3: T005 (repo: get_by_short_id + create update)
|
||||
Step 3a: T005a (conftest: fixture images get short_id)
|
||||
Step 4: T006 → T007 → T008 (API routes: tests → dict → route params)
|
||||
Step 5: T009 (Angular failing tests)
|
||||
Step 6: T010, T011, T011a, T012 (parallel) (route, interface, detail caller fix, library navigation)
|
||||
Step 7: T013 → T014 → T015 (upload: tests → handler → upload navigation)
|
||||
Step 8: T016 → T017 → T018 (migration: tests → script → NOT NULL migration)
|
||||
Step 9: T019, T020 (parallel) (test updates)
|
||||
Step 10: T021, T022, T023 (lint, build, manual verification)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Strategy
|
||||
|
||||
### MVP (US1 + US2 — full feature with new uploads)
|
||||
|
||||
1. T001–T005a — foundational infrastructure + conftest fixtures
|
||||
2. T006–T012 (including T011a) — API routes accept short_id + frontend uses `/i/`
|
||||
3. T013–T015 — new uploads generate short_id
|
||||
4. **STOP and VALIDATE**: upload a new image, confirm `/i/{short_id}` URL, confirm browsing works
|
||||
5. T016–T018 — migrate existing images
|
||||
6. T019–T023 — polish
|
||||
|
||||
### Note on Priority vs Implementation Order
|
||||
|
||||
US1 (P1) and US2 (P2) are implemented together before US3 (P3). The foundational phase is the true prerequisite for all three. US1 and US2 are tightly coupled in practice (you need uploads to produce short IDs before routing can be tested end-to-end), so they are sequenced rather than strictly priority-ordered.
|
||||
Reference in New Issue
Block a user