notifications/docs/implementation.md

# Notifications Service — Implementation Guide

Technical walkthrough of how the service is actually built, for anyone
maintaining or extending it. For what the service *does* and how to
integrate with it, see [service-overview.md](service-overview.md).

## Code layout

```
apps/
  core/               root URL ("/") — a login-gated placeholder home view, nothing else
  users/               local User model, mirrored by uuid from the accounts service
  gooyal_oauth2/        OAuth2 resource-server wiring: token validator, permission class,
                         get_application() helper
  push_notifications/    push domain: PushUser, PushMessage, BulkPushMessage
  emails/                email domain: Email, EmailQueue
utils/
  clients/gotify.py     the only code that talks to the Gotify HTTP API
  models.py              BaseModel (uuid pk, created_at/updated_at)
gotify_rest_api_client/  generated OpenAPI client for Gotify (do not hand-edit)
main/
  settings.py, celery.py, urls.py, wsgi.py/asgi.py
```

## Push notification flow

### Single push

```
POST /push/application/<user_uuid>/application/
  ↓ ApplicationPushUserViewSet.perform_create()   (apps/push_notifications/views.py)
  1. resolve target User by user_uuid
  2. resolve/create their PushUser (Gotify identity) — PushUser.objects.submit(user)
  3. resolve the calling Application from the OAuth2 token
  4. serializer.save(push_user=..., application=...)   → PushMessage row
  5. instance.send_push()   → send_push_notification.delay(uuid)   (Celery)
       ↓ apps/push_notifications/tasks.py
       gotify.send_notif(token=push_user.application_token,
                          title, message, priority,
                          extras=push_message.get_gotify_extras())
       push_message.state = DONE
```

`PushUser.objects.submit(user)` provisions three things in Gotify on first use, all via `utils/clients/gotify.py`, and persists the resulting tokens on `PushUser`:
1. `submit_user` — a Gotify *user* named after the Gooyal `user_uuid` (admin-token auth)
2. `submit_client` — a Gotify *client*, whose token becomes `PushUser.client_token` (basic-auth as the just-created user)
3. `submit_application` — a Gotify *application* under that client, whose token becomes `PushUser.application_token` — **this is the token push messages are actually sent with**

### Bulk push

An admin action (`BulkPushMessageAdmin`, `apps/push_notifications/admin.py`) triggers `BulkPushMessage.bulk_push_task()` → Celery `bulk_push` task → `BulkPushMessage.push_to_all(extract_data())`.

`extract_data()` reads a legacy `.xls` file (via `xlrd` — only `.xls`, not `.xlsx`, since `xlrd` dropped xlsx support at 2.0) with columns:

| Col | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
|---|---|---|---|---|---|---|---|---|
| Field | `user_uuid` | `title` | `message` | `priority` | `extras` (JSON string) | `click_url` | `code` | `click_object_id` |

`push_to_all()` looks up each row's `PushUser` by `user_id`; missing users are counted as failures rather than raising, so one bad row doesn't abort the batch. Each successful row creates a `PushMessage` and calls `send_push()` — it goes through the exact same Celery task and Gotify call as a single push, so there's no separate bulk-specific delivery code path to keep in sync.

## The click action mechanism

Added in migration `0004_pushmessage_click_url`, extended in `0005` with
`code`/`click_object_id` and the `NotificationLink` model. The design
constraint throughout: **most notifications aren't clickable**, so this had
to be fully optional at every layer, and had to work identically for both
the single and bulk paths without a second implementation.

Two ways to set a destination on a `PushMessage`:
1. **`click_url`** — a raw URL the caller already knows. Always wins if set.
2. **`code` + `click_object_id`** — the preferred path. `code` is a documented,
   per-notification-type string (e.g. `billboard.approved`, `escrow.timeout` —
   see [notification-click-actions.md](notification-click-actions.md) for the
   full catalog); `click_object_id` is the id of the thing the notification is
   about (an ad's or escrow deal's uuid, typically). Resolved at send time
   against `NotificationLink`, an admin-managed table
   (`code`, `url_template`, `is_active`) — `url_template` supports a
   `{object_id}` placeholder. This is what moved the actual destination
   strings out of Python and into the Django admin, so a routing-scheme
   change is a config edit, not a deploy across every producer service.

`apps/push_notifications/models.py`:

```python
GOTIFY_CLICK_EXTRA_KEY = "client::notification"   # Gotify's own reserved namespace

click_url = models.URLField(max_length=1000, null=True, blank=True)
code = models.SlugField(max_length=100, null=True, blank=True, db_index=True)
click_object_id = models.CharField(max_length=255, null=True, blank=True)

def resolve_click_url(self):
    if self.click_url:
        return self.click_url
    if not self.code:
        return None
    link = NotificationLink.objects.filter(code=self.code, is_active=True).first()
    if not link:
        return None
    return link.resolve(self.click_object_id)

def get_gotify_extras(self):
    extras = dict(self.extras or {})
    click_url = self.resolve_click_url()
    if click_url:
        notification_extra = dict(extras.get(self.GOTIFY_CLICK_EXTRA_KEY) or {})
        notification_extra["click"] = {"url": click_url}
        extras[self.GOTIFY_CLICK_EXTRA_KEY] = notification_extra
    return extras
```

Design decisions worth knowing if you touch this:
- **An unresolvable code is not an error** — no matching `NotificationLink`, or one marked `is_active=False`, just means no click action, same as a message with nothing set at all. A producer can start sending a new code before anyone's configured it in the admin; nothing breaks, the notification just isn't clickable yet.
- **The merge is additive**: any other `extras` keys a producer already sends (e.g. chat's `conversation_uuid`/`post_id`) pass through untouched — `get_gotify_extras()` only ever adds the `client::notification` key, never removes others.
- **`get_gotify_extras()` is the single place resolution happens** — `tasks.py` calls it instead of reading `push_message.extras`/`click_url` directly, so the single-push path, the bulk-push path, and both `click_url` and `code` all go through one function.
- **The admin table is deliberately not seeded with rows by any migration.** Populating it is an ops/product decision (which is the entire point of moving it out of code) — see [frontend-notification-click-actions.md](frontend-notification-click-actions.md) for the current code catalog they need to fill in.

## Email flow

Two paths, both in `apps/emails/`:

- **Immediate**: `Email.send()` → `send_email` Celery task → `Email._send_email()` → `django.core.mail.send_mail()`.
- **Queued digest**: `Email.objects.create(..., queue=some_queue)` — `EmailQueue` rows are checked every 10 seconds by `send_email_for_queued_events` (registered via `celery_app.conf.beat_schedule` directly in `apps/emails/tasks.py`, not through the `CELERY_BEAT_SCHEDULE` Django setting other Gooyal services use — see the overview doc's known-gaps section). A queue only actually sends once it's been at least 10 minutes since its `last_sent`, batching every `Email` created since then into one message.

**`_send_email()` currently hard-codes the recipient** to a literal test address instead of `self.user`'s email — this means the immediate-send path doesn't actually reach the intended recipient today. Flagging this here since it's the kind of thing that's easy to assume "must already work" when reading the flow.

## Auth internals

This service is an OAuth2 **resource server** (`django-oauth-toolkit`), not an OAuth2 provider — it doesn't issue tokens, it validates ones issued by the Gooyal accounts service via introspection (`OAUTH2_PROVIDER['RESOURCE_SERVER_INTROSPECTION_URL']`, using this service's own `CLIENT_ID`/`CLIENT_SECRET` as the introspection credentials).

`apps.gooyal_oauth2.rest_framework.IsAuthenticatedOrTokenMatchesOASRequirements` is the permission class every producer-facing endpoint uses. It accepts either:
- a plain authenticated (non-OAuth2) request, **or**
- an OAuth2 token whose scopes satisfy the view's `required_alternate_scopes`

`apps.gooyal_oauth2.utils.get_application(request)` pulls the calling `Application` off `request.auth.application` — this is how a `PushMessage`/`Email` row knows which producer service created it, without trusting a client-supplied field.

## Testing status

`apps/push_notifications/tests.py` and `apps/emails/tests.py` are both empty stubs — `python manage.py test` reports 0 tests. Everything described in this doc and in [notification-click-actions.md](notification-click-actions.md) was verified manually (unit-level checks via `manage.py shell`, and live round-trips against the local Gotify 2.6.3 container), not via an automated suite. If you're adding tests, `push_notifications` is the higher-value target — it's the domain three other services actively depend on.

## Notable migrations

- `0004_pushmessage_click_url` — adds `click_url`.
- `0005_notificationlink_pushmessage_click_object_id_and_more` — adds `NotificationLink`, `PushMessage.code`, `PushMessage.click_object_id`. See "The click action mechanism" above.

## Known technical debt (implementation-level)

- `apps/push_notifications/admin.py` has a half-built `PushMessageAdmin` custom form class (`PushMessageForm`, a `custom-action2` URL) that is **never registered** — `admin.site.register(PushMessage)` uses the plain default `ModelAdmin` instead. Dead code, safe to ignore or remove.
- Duplicate `app_name = 'push_notifications'` across `application_urls.py`/`user_urls.py` (see overview doc).
- Bulk import is locked to legacy `.xls` by the `xlrd` dependency; there's no `.xlsx` path.