diff --git a/docs/superpowers/plans/2026-03-26-erp-api-migration.md b/docs/superpowers/plans/2026-03-26-erp-api-migration.md new file mode 100644 index 0000000..ab26cf9 --- /dev/null +++ b/docs/superpowers/plans/2026-03-26-erp-api-migration.md @@ -0,0 +1,647 @@ +# ERP API Migration — Replace NocoDB Museum Data with Hono ERP API + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Replace NocoDB as the museum sales data source with the Hono ERP API, keeping NocoDB only for PilgrimStats. Add "channel" as a new filterable dimension (replacing "district"). + +**Architecture:** The Hono ERP API returns transaction-level sales data (each sale with nested `Products[]`). We authenticate via JWT (POST `/auth/login`), then fetch by date range (GET `/api/getbydate`). Client-side code aggregates transactions into daily/museum/channel records that match the existing `MuseumRecord` shape. NocoDB remains solely for PilgrimStats. + +**Tech Stack:** React 19, TypeScript (strict), Vite, Chart.js, Hono ERP REST API (Azure-hosted) + +**Security note:** ERP credentials are stored as `VITE_*` env vars which get bundled into the client-side JS (same pattern as the existing NocoDB token). The ERP account (`sales_user`) is a read-only reporting account. A server-side proxy can be added later if needed. + +**Compilation note:** Tasks 3–9 form an atomic migration — the codebase will not compile between them. They must be executed as a single batch on one branch. Individual commits are for traceability, not for producing intermediate working states. + +--- + +## File Structure + +| Action | File | Responsibility | +|--------|------|----------------| +| Create | `src/utils/fetchHelpers.ts` | Shared `fetchWithTimeout` + `fetchWithRetry` (extracted from dataService) | +| Create | `src/config/museumMapping.ts` | Product description → museum mapping + channel label mapping | +| Create | `src/services/erpService.ts` | ERP API auth, fetching, transaction → MuseumRecord aggregation | +| Modify | `src/types/index.ts` | Add `channel` to MuseumRecord, add ERP API types, remove NocoDB museum types, remove `DistrictMuseumMap` | +| Modify | `src/services/dataService.ts` | Replace NocoDB fetch with ERP fetch, replace district→channel in grouping/filter functions, remove `revenue_incl_tax` fallbacks | +| Modify | `src/components/Dashboard.tsx` | Replace district filter/chart with channel, update `filterKeys` array, update all district references | +| Modify | `src/components/Comparison.tsx` | Replace district filter with channel | +| Modify | `src/components/Slides.tsx` | Full refactor: replace `DistrictMuseumMap` prop threading (10+ call sites), `SlideConfig.district`→`channel`, update `SlideEditor`/`SlidePreview`/`PreviewMode` interfaces, update `generateSlideHTML`/`generateChartScripts` | +| Modify | `src/locales/en.json` | Replace district→channel keys, add `charts.channel`, update error messages | +| Modify | `src/locales/ar.json` | Arabic translations for all channel-related keys | +| Modify | `src/App.tsx` | Update env var check for ERP config | +| Modify | `.env.local` | Add ERP API credentials | +| Modify | `.env.example` | Update to reflect ERP as primary museum data source | + +--- + +### Task 1: Environment Configuration + +**Files:** +- Modify: `.env.local` +- Modify: `.env.example` + +- [ ] **Step 1: Add ERP env vars to `.env.local`** + +```env +# Hono ERP API (museum sales data) +VITE_ERP_API_URL= +VITE_ERP_API_CODE= +VITE_ERP_USERNAME= +VITE_ERP_PASSWORD= +``` + +The actual values are in the Postman collection at `~/Downloads/hono-erp Copy.postman_collection.json`. Read that file for the credentials. Keep existing NocoDB vars (needed for PilgrimStats). + +- [ ] **Step 2: Update `.env.example`** + +Update to document both data sources: + +```env +# Hono ERP API (museum sales data — primary source) +VITE_ERP_API_URL=https://hono-erp.azurewebsites.net +VITE_ERP_API_CODE=your-api-function-key +VITE_ERP_USERNAME=your-username +VITE_ERP_PASSWORD=your-password + +# NocoDB (PilgrimStats only) +VITE_NOCODB_URL=http://localhost:8090 +VITE_NOCODB_TOKEN=your-token +VITE_NOCODB_BASE_ID=your-base-id +``` + +- [ ] **Step 3: Commit** (`.env.local` is gitignored — only commit `.env.example`) + +```bash +git add .env.example +git commit -m "feat: update env example for ERP API as primary museum data source" +``` + +--- + +### Task 2: Extract Fetch Helpers + +**Files:** +- Create: `src/utils/fetchHelpers.ts` +- Modify: `src/services/dataService.ts` (update imports) + +- [ ] **Step 1: Extract `fetchWithTimeout` and `fetchWithRetry`** + +Move these two functions from `dataService.ts` into `src/utils/fetchHelpers.ts`. Export them. Also move the constants `FETCH_TIMEOUT_MS` and `MAX_RETRIES`. + +- [ ] **Step 2: Update dataService.ts imports** + +Replace the function definitions with: +```typescript +import { fetchWithTimeout, fetchWithRetry } from '../utils/fetchHelpers'; +``` + +- [ ] **Step 3: Verify build still works** + +```bash +npm run build +``` + +- [ ] **Step 4: Commit** + +```bash +git add src/utils/fetchHelpers.ts src/services/dataService.ts +git commit -m "refactor: extract fetch helpers to shared util" +``` + +--- + +### Task 3: Museum Mapping Configuration + +**Files:** +- Create: `src/config/museumMapping.ts` + +Definitive mapping of all 47 known product descriptions to museum names, plus channel label mappings. + +- [ ] **Step 1: Create museum mapping file** + +The mapping uses keyword matching with a **priority order** — this matters for combo tickets. Check keywords in this order (first match wins): + +1. **Revelation Exhibition** — keywords: `"Revelation"`, `"الوحي"` (catches combo tickets mentioning both الوحي and القرآن الكريم) +2. **Creation Story Museum** — keywords: `"Creation Story"`, `"قصة الخلق"` +3. **Holy Quraan Museum** — keywords: `"Holy Quraan"`, `"القرآن الكريم"` +4. **Trail To Hira Cave** — keywords: `"Trail To Hira"`, `"غار حراء"` +5. **Makkah Greets Us** — keywords: `"Makkah Greets"` +6. **VIP Experience** — keywords: `"VIP Experience"` + +If no match: return `"Other"`. + +Channel label mapping: +```typescript +const CHANNEL_LABELS: Record = { + 'B2C': 'HiHala Website/App', + 'B2B': 'B2B', + 'POS': 'POS', + 'Safiyyah POS': 'Safiyyah POS', + 'Standalone': 'Standalone', + 'Mobile': 'Mobile', + 'Viva': 'Viva', + 'IT': 'IT' +}; +``` + +Exports: +- `getMuseumFromProduct(productDescription: string): string` +- `getChannelLabel(operatingAreaName: string): string` +- `MUSEUM_NAMES: string[]` +- `CHANNEL_LABELS: Record` + +- [ ] **Step 2: Commit** + +```bash +git add src/config/museumMapping.ts +git commit -m "feat: add product-to-museum and channel mapping config" +``` + +--- + +### Task 4: TypeScript Types Update + +**Files:** +- Modify: `src/types/index.ts` + +- [ ] **Step 1: Update MuseumRecord** + +Replace `district` with `channel`. Remove `museum_code` and `revenue_incl_tax` (legacy). + +```typescript +export interface MuseumRecord { + date: string; + museum_name: string; + channel: string; // was: district + visits: number; // = sum of PeopleCount per product line + tickets: number; // = sum of UnitQuantity per product line + revenue_gross: number; // = sum of TotalPrice (includes VAT) + revenue_net: number; // = revenue_gross - sum of TaxAmount + year: string; + quarter: string; +} +``` + +- [ ] **Step 2: Add ERP API types** + +```typescript +export interface ERPProduct { + ProductDescription: string; + SiteDescription: string | null; + UnitQuantity: number; + PeopleCount: number; + TaxAmount: number; + TotalPrice: number; +} + +export interface ERPPayment { + PaymentMethodDescription: string; +} + +export interface ERPSaleRecord { + SaleId: number; + TransactionDate: string; + CustIdentification: string; + OperatingAreaName: string; + Payments: ERPPayment[]; + Products: ERPProduct[]; +} + +export interface ERPLoginResponse { + token: string; +} +``` + +- [ ] **Step 3: Update Filters interface** + +```typescript +export interface Filters { + year: string; + channel: string; // was: district + museum: string; + quarter: string; +} + +export interface DateRangeFilters { + channel: string; // was: district + museum: string; +} +``` + +- [ ] **Step 4: Remove obsolete types** + +Remove: `NocoDBDistrict`, `NocoDBMuseum`, `NocoDBDailyStat`, `DistrictMuseumMap`. + +Update `SlideConfig.district` → `SlideConfig.channel`. + +- [ ] **Step 5: Commit** + +```bash +git add src/types/index.ts +git commit -m "feat: update types for ERP API — channel replaces district" +``` + +--- + +### Task 5: ERP Service + +**Files:** +- Create: `src/services/erpService.ts` + +- [ ] **Step 1: Implement auth + fetch** + +```typescript +import { fetchWithRetry } from '../utils/fetchHelpers'; +import { getMuseumFromProduct, getChannelLabel } from '../config/museumMapping'; +import type { ERPSaleRecord, ERPLoginResponse, MuseumRecord } from '../types'; + +const ERP_API_URL = import.meta.env.VITE_ERP_API_URL || ''; +const ERP_API_CODE = import.meta.env.VITE_ERP_API_CODE || ''; +const ERP_USERNAME = import.meta.env.VITE_ERP_USERNAME || ''; +const ERP_PASSWORD = import.meta.env.VITE_ERP_PASSWORD || ''; + +let cachedToken: string | null = null; + +async function login(): Promise { /* POST /auth/login, cache token */ } +async function fetchSalesByDateRange(startDate: string, endDate: string): Promise { /* GET /api/getbydate with Bearer token + code param */ } +``` + +Auth: token cached in module-level variable, re-login on 401. + +Fetch strategy: generate month boundaries from 2024-01 to current month, fetch all in parallel with `Promise.all`. + +- [ ] **Step 2: Implement aggregation function** + +```typescript +export function aggregateTransactions(sales: ERPSaleRecord[]): MuseumRecord[] +``` + +For each sale: +1. Extract date from `TransactionDate` (split on space, take first part → `"2025-01-01"`) +2. Get channel from `OperatingAreaName` via `getChannelLabel()` +3. For each product in `Products[]`: + - Get museum from `getMuseumFromProduct(product.ProductDescription)` + - Accumulate into key `${date}|${museum}|${channel}`: + - `visits += product.PeopleCount` + - `tickets += product.UnitQuantity` + - `revenue_gross += product.TotalPrice` + - `revenue_net += (product.TotalPrice - product.TaxAmount)` +4. Convert accumulated map to `MuseumRecord[]` with computed `year` and `quarter` + +Negative quantities (refunds) sum correctly by default — no special handling needed. + +- [ ] **Step 3: Export main entry point** + +```typescript +export async function fetchFromERP(): Promise +``` + +This orchestrates: login → fetch all months → aggregate → return. + +- [ ] **Step 4: Commit** + +```bash +git add src/services/erpService.ts +git commit -m "feat: add ERP service with auth, fetch, and aggregation" +``` + +--- + +### Task 6: Refactor dataService.ts + +**Files:** +- Modify: `src/services/dataService.ts` + +- [ ] **Step 1: Replace NocoDB museum fetch with ERP** + +- Remove: `discoverTableIds()`, `fetchNocoDBTable()`, `fetchFromNocoDB()`, `MuseumMapEntry`, NocoDB museum env var reads +- Keep: NocoDB config for PilgrimStats path only (`fetchPilgrimStats()` unchanged) +- Import `fetchFromERP` from erpService +- Update `fetchData()` and `refreshData()` to call `fetchFromERP()` instead of `fetchFromNocoDB()` +- Update config check: `if (!ERP_API_URL || !ERP_API_CODE || !ERP_USERNAME || !ERP_PASSWORD)` → throw DataError('config') + +- [ ] **Step 2: Replace all district→channel in data functions** + +- `filterData()`: `filters.district`→`filters.channel`, `row.district`→`row.channel` +- `filterDataByDateRange()`: same +- `groupByDistrict()`→ rename to `groupByChannel()`, change `row.district`→`row.channel` +- `getUniqueDistricts()`→ rename to `getUniqueChannels()`, change `r.district`→`r.channel` +- `getDistrictMuseumMap()`→ **delete** +- `getMuseumsForDistrict()`→ replace with `getUniqueMuseums(data: MuseumRecord[]): string[]` returning all unique museum names sorted +- Remove all `revenue_incl_tax` fallback references (e.g. `|| row.revenue_incl_tax || 0` in groupBy functions and calculateMetrics) + +- [ ] **Step 3: Update type imports** + +Remove unused NocoDB types from the import block. Add imports for new ERP-related types if needed. + +- [ ] **Step 4: Commit** + +```bash +git add src/services/dataService.ts +git commit -m "refactor: replace NocoDB museum fetch with ERP API, district→channel" +``` + +--- + +### Task 7: Update Dashboard Component + +**Files:** +- Modify: `src/components/Dashboard.tsx` + +- [ ] **Step 1: Update imports** + +- `getUniqueDistricts` → `getUniqueChannels` +- `getDistrictMuseumMap` → remove +- `getMuseumsForDistrict` → `getUniqueMuseums` +- `groupByDistrict` → `groupByChannel` + +- [ ] **Step 2: Update filter state and controls** + +- `defaultFilters`: `district: 'all'` → `channel: 'all'` +- `filterKeys` array (line 32): `'district'` → `'channel'` — this controls URL param serialization +- All `filters.district` → `filters.channel` +- District `` with `t('filters.channel')` label +- Museum `` elements) +- Museum filter: use `getUniqueMuseums(data)` (no longer cascaded) + +- [ ] **Step 2: Commit** + +```bash +git add src/components/Comparison.tsx +git commit -m "feat: update Comparison — channel replaces district" +``` + +--- + +### Task 9: Update Slides Component (FULL REFACTOR) + +**Files:** +- Modify: `src/components/Slides.tsx` + +This is a significant change — Slides.tsx has 30+ district references across 10+ call sites including function signatures, prop interfaces, and HTML export generation. + +- [ ] **Step 1: Update imports** + +- Replace `getUniqueDistricts` → `getUniqueChannels` +- Replace `getDistrictMuseumMap` → remove +- Replace `getMuseumsForDistrict` → `getUniqueMuseums` +- Remove import of `DistrictMuseumMap` type + +- [ ] **Step 2: Update component interfaces** + +All three interfaces pass `districts: string[]` and `districtMuseumMap: DistrictMuseumMap`: + +```typescript +// SlideEditorProps (line 25): districts→channels, remove districtMuseumMap +interface SlideEditorProps { + slide: SlideConfig; + onUpdate: (updates: Partial) => void; + channels: string[]; + museums: string[]; // flat list, independent of channel + data: MuseumRecord[]; + chartTypes: ChartTypeOption[]; + metrics: MetricOption[]; +} + +// SlidePreviewProps (line 35): same pattern +interface SlidePreviewProps { + slide: SlideConfig; + data: MuseumRecord[]; + channels: string[]; + museums: string[]; + metrics: MetricOption[]; +} + +// PreviewModeProps (line 43): same pattern +interface PreviewModeProps { + slides: SlideConfig[]; + data: MuseumRecord[]; + channels: string[]; + museums: string[]; + currentSlide: number; + setCurrentSlide: React.Dispatch>; + onExit: () => void; + metrics: MetricOption[]; +} +``` + +- [ ] **Step 3: Update Slides() main function** + +- `districts` memo → `channels` using `getUniqueChannels(data)` +- `districtMuseumMap` memo → `museums` using `getUniqueMuseums(data)` +- `defaultSlide.district: 'all'` → `channel: 'all'` +- Update all prop passing: `districts={districts} districtMuseumMap={districtMuseumMap}` → `channels={channels} museums={museums}` + +- [ ] **Step 4: Update SlideEditor function** + +- `getMuseumsForDistrict(districtMuseumMap, slide.district)` → just use `museums` prop directly +- Filter label: `t('filters.district')` → `t('filters.channel')` +- `