wallet/docs/company_wallet_history_backfill.md

# Company-wallet history backfill

**Status:** ready to run · dry-run first · not yet executed against staging/prod
**Command:** `python manage.py backfill_company_wallets`
**Scope:** wallet service DB only — `wallet_transaction`, `wallet_account`

---

## 1. Background — why this is needed

A wallet call has **two** wallet references:

| Reference | Owner | Where it appears |
|---|---|---|
| **user-side** — `payer_wallet` / `payee_wallet` in the request body | the end user | resolved to `Account(owner=user, wallet=…)` |
| **company-side** — the first path segment of `/api/application/<wallet>/deposit\|withdraw/` | the calling application | resolved to `Account(owner=application, wallet=…)` |

For a **deposit** call the company account is the transaction's **payer**; for a
**withdraw** call it is the **payee**.

Historically several services had **no dedicated company account**, so they passed
a *user* wallet (the shared rial or reward wallet) as their own company-side
wallet. Their money therefore moved in and out of
`Account(owner=<their app>, wallet=<rial|reward>)` — an account that structurally
looks like a user balance and does not reconcile against anything.

Each of those services has since been given a real dedicated company wallet and
its code updated. This backfill rewrites the **historical** `Transaction` rows so
the company side points at the same dedicated wallet the current code uses, and
corrects the two affected `Account.balance` running totals.

Nothing outside the wallet service stores a company wallet UUID on its own rows —
the other services only ever read `settings.WALLET_*` at call time — so this repo
is the only place with data to migrate.

---

## 2. Data model recap

```
Transaction
  ├─ payer_account ──FK(PROTECT)──▶ Account(owner_uuid, owner_type, wallet ─FK▶ Wallet, balance)
  ├─ payee_account ──FK(PROTECT)──▶ Account(…)
  ├─ application ───FK──▶ gooyal_oauth2.Application
  ├─ amount, state, reference
  └─ details (JSON: description, reference_id, payer_name, payee_name, …)
```

- Wallet identity is the `Wallet` row (its `uuid`); it is never denormalised onto
  `Transaction`. The only lever is the `payer_account` / `payee_account` FK.
- `Account.balance` is a **denormalised running total**, mutated incrementally by
  `Transaction.withdraw_from_payer_balance()` (at `submit()`, `CREATED→PENDING`)
  and `deposit_to_payee_balance()` (at `verify()`, `PENDING→SUCCESS`), and undone
  by `cancel()` / `rollback()`. Repointing an FK does **not** touch it — the
  backfill fixes it explicitly.
- `StateChoices`: `CREATED 1 · DELAYED 2 · PENDING 3 · INCOMPLETE 4 · SUCCESS 5 ·
  FAILED 6 · EXPECTED_FAILURE 7 · ROLLED_BACK 8 · CANCELED 9`.
- `TypeChoices`: `USER 1 · APPLICATION 2`.

---

## 3. What moves

User-side wallets are **unchanged** — only their env-var names drifted:

```
WALLET_RIAL   = WALLET_RIAL_DEPOSIT               = af7d967f-30c0-409b-9066-2549f2da5e5e
WALLET_REWARD = WALLET_USER_BILLBOARD_VISIT_INCOME = e7c9d4d1-4d1f-43b2-96f7-4d4a168f480d
```

| Service | Flow(s) | Txn side | OLD company wallet | NEW company wallet |
|---|---|---|---|---|
| **ipg** | gateway top-up (`PaymentRequest.deposit_submit/verify`) | payer | `af7d967f…` rial | `WALLET_IPG_CREDIT` `5c693c93-6b13-476e-a720-e38f3798acae` |
| **settlement** | rial + reward payout legs | payee | `af7d967f…` / `e7c9d4d1…` | `WALLET_SETTLEMENT_TRANSIT` `939d9d70-3bda-4413-9f9e-756ef4e1525a` |
| **settlement** | commission legs¹ | payee | `af7d967f…` / `e7c9d4d1…` | `WALLET_SETTLEMENT_COMMISSION_INCOME` `ee8b050a-0ab7-48c3-a13c-01733de9bb1d` |
| **advertising** | billboard-visit reward payout, ad-balance refund, content/tip deposit to creator | payer | `e7c9d4d1…` reward | `WALLET_ADVERTISING_TRANSIT` `052d38f0-d4de-40ff-85f6-9ee6e880b7e4` |
| **advertising** | content/tip withdraw from visitor | payee | `e7c9d4d1…` reward | `WALLET_ADVERTISING_TRANSIT` `052d38f0…` |
| **promotions** | promotion payout (`Promotion.promote`) | payer | `e7c9d4d1…` reward (old `WALLET_REWARD`) | `WALLET_PROMOTIONS_CREDIT`² `f1f14c34-7e28-4d28-97c6-2bb8b2189ff3` |

¹ Payout and commission legs land on the same old wallet and are told apart by
`details.description` ∈ {`"settlement commission transaction"`, `"کارمزد تسویه حساب"`}.
² The promotions code renamed `WALLET_PROMOTIONS_TRANSIT` → `WALLET_PROMOTIONS_CREDIT`;
some env files still use the old name. Same UUID.

### Deliberately **not** touched

| | Reason |
|---|---|
| advertising `AdPayment.submit()` charge | always withdrew into `WALLET_ADVERTISING_TRANSIT`, never a user wallet |
| advertising escrow (`EscrowWalletPayment`) | `WALLET_RIAL_ESCROW_PAYMENTS → WALLET_ADVERTISING_ESCROW` was a pure rename (same UUID); the commission split to `WALLET_ADVERTISING_ESCROW_INCOME` postdates any prod data (feature unreleased) |
| every user-side `payer` / `payee` | unchanged |
| `campaign`, `crm_backend`, `ipg` commission | `campaign`/`crm_backend` do balance reads only or have no dedicated account to move to; ipg has only the one flow |

---

## 4. Balance correction

For each `(service, leg)` the command:

1. resolves `old_acct = Account(app, OLD_wallet)` and get-or-creates `new_acct = Account(app, NEW_wallet)`;
2. selects `rows = Transaction.filter(application=app, <side>_account=old_acct[, description filter])`;
3. computes `Σ = sum(amount)` over `rows` **restricted to the states whose balance
   effect is currently applied**:
   - payer / deposit leg → `{PENDING, SUCCESS, DELAYED, INCOMPLETE}`
   - payee / withdraw leg → `{SUCCESS}`
4. repoints **all** matched rows (any state) `rows.update(<side>_account=new_acct)`;
5. shifts the running totals:
   - payer leg: `old_acct.balance += Σ` , `new_acct.balance -= Σ`
     *(the old account had been debited by Σ for these payouts — it gets it back;
     the new account now carries the outflow)*
   - payee leg: `old_acct.balance -= Σ` , `new_acct.balance += Σ`
     *(the old account had been credited by Σ — it loses it; the new account gains it)*

Net change across each pair is **zero**, so system-wide balance is conserved.

Everything for one `--execute` run happens inside a single `transaction.atomic()`
with `select_for_update()` on every `Account` touched.

### Consequences to accept before running

- **The new dedicated accounts go strongly negative** — they are created today but
  now carry months of historical outflow. Each service's application UUID must be
  listed in `settings.ALLOWED_NEGATIVE_BALANCE_APPLICATIONS`; the balance itself
  reading negative is expected and correct.
- **This rewrites historical financial records.** Any reconciliation or report
  already produced from the old ledger will not reproduce afterwards.
- The old rial/reward application accounts are left at their residual (≈ 0 if every
  historical call is accounted for).

### Rejected alternative

Leave every `Transaction` untouched and post one compensating transfer per leg
(old company account → new, for the live Σ). Keeps the ledger append-only, but
per-row history still shows the old wallet — which defeats the purpose.

---

## 5. Running it

```bash
# dry run — every service, no writes
python manage.py backfill_company_wallets

# dry run — one service
python manage.py backfill_company_wallets --service settlement

# apply
python manage.py backfill_company_wallets --execute

# force an Application uuid if name resolution is ambiguous
python manage.py backfill_company_wallets --app-promotions <uuid> --execute
```

**Application resolution** (printed in every run — verify it):

1. an existing `owner_type=APPLICATION` account already on the service's new
   dedicated wallet → its `owner_uuid`;
2. otherwise `Application.name` (`ipg` / `settlement` / `ad app` / `promotion…`);
3. ambiguous or missing → the command aborts and lists all known apps; pass
   `--app-<service> <uuid>`.

**Idempotent** — the row filter is on the *old* account, so a second run moves nothing.

### Dry-run checklist

- [ ] the resolved `Application` per service is correct
- [ ] settlement: commission-leg row count ≪ payout-leg row count (the
      `details.description` split is working); investigate any settlement rows the
      dry run leaves unclassified
- [ ] the printed balance deltas are plausible against current account balances
- [ ] each service's app UUID is in `ALLOWED_NEGATIVE_BALANCE_APPLICATIONS`

---

## 6. How the mapping was derived

Cross-repo git archaeology (commit refs current as of 2026-09-02):

| Service | Commit(s) that introduced the dedicated wallet |
|---|---|
| ipg | `0083f1b` rename `WALLET_INCOME_FROM_IPG → WALLET_IPG_CREDIT`; `e4e13b2` "deposit user balance from IPG's own wallet, not the user rial wallet" |
| settlement | `cc2ffb9` "add settlement transit/commission wallets…"; `2037cd5` rename `WALLET_SETTLEMENT_TRANSIT → WALLET_SETTLEMENT_CREDIT` (code); env still `_TRANSIT` |
| advertising | `a8fb2a5` "pay billboard rewards and refunds out of WALLET_ADVERTISING_TRANSIT"; `cee2b1c` content/tip routing; `9df00d0` drop `WALLET_CONTENT_PAYMENT_TRANSIT` |
| promotions | `f59b881` add `WALLET_PROMOTIONS_TRANSIT` + per-recipient routing; `8725cdf` rename `→ WALLET_PROMOTIONS_CREDIT`; `ea070ad` `PromotionTransaction` ledger |