chore: add ERP API migration plan (pre-migration snapshot)

Preserves current NocoDB-based state before switching museum
sales data source to the Hono ERP API.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
fahed
2026-03-26 13:43:20 +03:00
parent 802ff28754
commit 9c1552e439

View File

@@ -0,0 +1,647 @@
# ERP API Migration — Replace NocoDB Museum Data with Hono ERP API
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace NocoDB as the museum sales data source with the Hono ERP API, keeping NocoDB only for PilgrimStats. Add "channel" as a new filterable dimension (replacing "district").
**Architecture:** The Hono ERP API returns transaction-level sales data (each sale with nested `Products[]`). We authenticate via JWT (POST `/auth/login`), then fetch by date range (GET `/api/getbydate`). Client-side code aggregates transactions into daily/museum/channel records that match the existing `MuseumRecord` shape. NocoDB remains solely for PilgrimStats.
**Tech Stack:** React 19, TypeScript (strict), Vite, Chart.js, Hono ERP REST API (Azure-hosted)
**Security note:** ERP credentials are stored as `VITE_*` env vars which get bundled into the client-side JS (same pattern as the existing NocoDB token). The ERP account (`sales_user`) is a read-only reporting account. A server-side proxy can be added later if needed.
**Compilation note:** Tasks 39 form an atomic migration — the codebase will not compile between them. They must be executed as a single batch on one branch. Individual commits are for traceability, not for producing intermediate working states.
---
## File Structure
| Action | File | Responsibility |
|--------|------|----------------|
| Create | `src/utils/fetchHelpers.ts` | Shared `fetchWithTimeout` + `fetchWithRetry` (extracted from dataService) |
| Create | `src/config/museumMapping.ts` | Product description → museum mapping + channel label mapping |
| Create | `src/services/erpService.ts` | ERP API auth, fetching, transaction → MuseumRecord aggregation |
| Modify | `src/types/index.ts` | Add `channel` to MuseumRecord, add ERP API types, remove NocoDB museum types, remove `DistrictMuseumMap` |
| Modify | `src/services/dataService.ts` | Replace NocoDB fetch with ERP fetch, replace district→channel in grouping/filter functions, remove `revenue_incl_tax` fallbacks |
| Modify | `src/components/Dashboard.tsx` | Replace district filter/chart with channel, update `filterKeys` array, update all district references |
| Modify | `src/components/Comparison.tsx` | Replace district filter with channel |
| Modify | `src/components/Slides.tsx` | Full refactor: replace `DistrictMuseumMap` prop threading (10+ call sites), `SlideConfig.district``channel`, update `SlideEditor`/`SlidePreview`/`PreviewMode` interfaces, update `generateSlideHTML`/`generateChartScripts` |
| Modify | `src/locales/en.json` | Replace district→channel keys, add `charts.channel`, update error messages |
| Modify | `src/locales/ar.json` | Arabic translations for all channel-related keys |
| Modify | `src/App.tsx` | Update env var check for ERP config |
| Modify | `.env.local` | Add ERP API credentials |
| Modify | `.env.example` | Update to reflect ERP as primary museum data source |
---
### Task 1: Environment Configuration
**Files:**
- Modify: `.env.local`
- Modify: `.env.example`
- [ ] **Step 1: Add ERP env vars to `.env.local`**
```env
# Hono ERP API (museum sales data)
VITE_ERP_API_URL=<see .env.local on machine>
VITE_ERP_API_CODE=<see .env.local on machine>
VITE_ERP_USERNAME=<see .env.local on machine>
VITE_ERP_PASSWORD=<see .env.local on machine>
```
The actual values are in the Postman collection at `~/Downloads/hono-erp Copy.postman_collection.json`. Read that file for the credentials. Keep existing NocoDB vars (needed for PilgrimStats).
- [ ] **Step 2: Update `.env.example`**
Update to document both data sources:
```env
# Hono ERP API (museum sales data — primary source)
VITE_ERP_API_URL=https://hono-erp.azurewebsites.net
VITE_ERP_API_CODE=your-api-function-key
VITE_ERP_USERNAME=your-username
VITE_ERP_PASSWORD=your-password
# NocoDB (PilgrimStats only)
VITE_NOCODB_URL=http://localhost:8090
VITE_NOCODB_TOKEN=your-token
VITE_NOCODB_BASE_ID=your-base-id
```
- [ ] **Step 3: Commit** (`.env.local` is gitignored — only commit `.env.example`)
```bash
git add .env.example
git commit -m "feat: update env example for ERP API as primary museum data source"
```
---
### Task 2: Extract Fetch Helpers
**Files:**
- Create: `src/utils/fetchHelpers.ts`
- Modify: `src/services/dataService.ts` (update imports)
- [ ] **Step 1: Extract `fetchWithTimeout` and `fetchWithRetry`**
Move these two functions from `dataService.ts` into `src/utils/fetchHelpers.ts`. Export them. Also move the constants `FETCH_TIMEOUT_MS` and `MAX_RETRIES`.
- [ ] **Step 2: Update dataService.ts imports**
Replace the function definitions with:
```typescript
import { fetchWithTimeout, fetchWithRetry } from '../utils/fetchHelpers';
```
- [ ] **Step 3: Verify build still works**
```bash
npm run build
```
- [ ] **Step 4: Commit**
```bash
git add src/utils/fetchHelpers.ts src/services/dataService.ts
git commit -m "refactor: extract fetch helpers to shared util"
```
---
### Task 3: Museum Mapping Configuration
**Files:**
- Create: `src/config/museumMapping.ts`
Definitive mapping of all 47 known product descriptions to museum names, plus channel label mappings.
- [ ] **Step 1: Create museum mapping file**
The mapping uses keyword matching with a **priority order** — this matters for combo tickets. Check keywords in this order (first match wins):
1. **Revelation Exhibition** — keywords: `"Revelation"`, `"الوحي"` (catches combo tickets mentioning both الوحي and القرآن الكريم)
2. **Creation Story Museum** — keywords: `"Creation Story"`, `"قصة الخلق"`
3. **Holy Quraan Museum** — keywords: `"Holy Quraan"`, `"القرآن الكريم"`
4. **Trail To Hira Cave** — keywords: `"Trail To Hira"`, `"غار حراء"`
5. **Makkah Greets Us** — keywords: `"Makkah Greets"`
6. **VIP Experience** — keywords: `"VIP Experience"`
If no match: return `"Other"`.
Channel label mapping:
```typescript
const CHANNEL_LABELS: Record<string, string> = {
'B2C': 'HiHala Website/App',
'B2B': 'B2B',
'POS': 'POS',
'Safiyyah POS': 'Safiyyah POS',
'Standalone': 'Standalone',
'Mobile': 'Mobile',
'Viva': 'Viva',
'IT': 'IT'
};
```
Exports:
- `getMuseumFromProduct(productDescription: string): string`
- `getChannelLabel(operatingAreaName: string): string`
- `MUSEUM_NAMES: string[]`
- `CHANNEL_LABELS: Record<string, string>`
- [ ] **Step 2: Commit**
```bash
git add src/config/museumMapping.ts
git commit -m "feat: add product-to-museum and channel mapping config"
```
---
### Task 4: TypeScript Types Update
**Files:**
- Modify: `src/types/index.ts`
- [ ] **Step 1: Update MuseumRecord**
Replace `district` with `channel`. Remove `museum_code` and `revenue_incl_tax` (legacy).
```typescript
export interface MuseumRecord {
date: string;
museum_name: string;
channel: string; // was: district
visits: number; // = sum of PeopleCount per product line
tickets: number; // = sum of UnitQuantity per product line
revenue_gross: number; // = sum of TotalPrice (includes VAT)
revenue_net: number; // = revenue_gross - sum of TaxAmount
year: string;
quarter: string;
}
```
- [ ] **Step 2: Add ERP API types**
```typescript
export interface ERPProduct {
ProductDescription: string;
SiteDescription: string | null;
UnitQuantity: number;
PeopleCount: number;
TaxAmount: number;
TotalPrice: number;
}
export interface ERPPayment {
PaymentMethodDescription: string;
}
export interface ERPSaleRecord {
SaleId: number;
TransactionDate: string;
CustIdentification: string;
OperatingAreaName: string;
Payments: ERPPayment[];
Products: ERPProduct[];
}
export interface ERPLoginResponse {
token: string;
}
```
- [ ] **Step 3: Update Filters interface**
```typescript
export interface Filters {
year: string;
channel: string; // was: district
museum: string;
quarter: string;
}
export interface DateRangeFilters {
channel: string; // was: district
museum: string;
}
```
- [ ] **Step 4: Remove obsolete types**
Remove: `NocoDBDistrict`, `NocoDBMuseum`, `NocoDBDailyStat`, `DistrictMuseumMap`.
Update `SlideConfig.district``SlideConfig.channel`.
- [ ] **Step 5: Commit**
```bash
git add src/types/index.ts
git commit -m "feat: update types for ERP API — channel replaces district"
```
---
### Task 5: ERP Service
**Files:**
- Create: `src/services/erpService.ts`
- [ ] **Step 1: Implement auth + fetch**
```typescript
import { fetchWithRetry } from '../utils/fetchHelpers';
import { getMuseumFromProduct, getChannelLabel } from '../config/museumMapping';
import type { ERPSaleRecord, ERPLoginResponse, MuseumRecord } from '../types';
const ERP_API_URL = import.meta.env.VITE_ERP_API_URL || '';
const ERP_API_CODE = import.meta.env.VITE_ERP_API_CODE || '';
const ERP_USERNAME = import.meta.env.VITE_ERP_USERNAME || '';
const ERP_PASSWORD = import.meta.env.VITE_ERP_PASSWORD || '';
let cachedToken: string | null = null;
async function login(): Promise<string> { /* POST /auth/login, cache token */ }
async function fetchSalesByDateRange(startDate: string, endDate: string): Promise<ERPSaleRecord[]> { /* GET /api/getbydate with Bearer token + code param */ }
```
Auth: token cached in module-level variable, re-login on 401.
Fetch strategy: generate month boundaries from 2024-01 to current month, fetch all in parallel with `Promise.all`.
- [ ] **Step 2: Implement aggregation function**
```typescript
export function aggregateTransactions(sales: ERPSaleRecord[]): MuseumRecord[]
```
For each sale:
1. Extract date from `TransactionDate` (split on space, take first part → `"2025-01-01"`)
2. Get channel from `OperatingAreaName` via `getChannelLabel()`
3. For each product in `Products[]`:
- Get museum from `getMuseumFromProduct(product.ProductDescription)`
- Accumulate into key `${date}|${museum}|${channel}`:
- `visits += product.PeopleCount`
- `tickets += product.UnitQuantity`
- `revenue_gross += product.TotalPrice`
- `revenue_net += (product.TotalPrice - product.TaxAmount)`
4. Convert accumulated map to `MuseumRecord[]` with computed `year` and `quarter`
Negative quantities (refunds) sum correctly by default — no special handling needed.
- [ ] **Step 3: Export main entry point**
```typescript
export async function fetchFromERP(): Promise<MuseumRecord[]>
```
This orchestrates: login → fetch all months → aggregate → return.
- [ ] **Step 4: Commit**
```bash
git add src/services/erpService.ts
git commit -m "feat: add ERP service with auth, fetch, and aggregation"
```
---
### Task 6: Refactor dataService.ts
**Files:**
- Modify: `src/services/dataService.ts`
- [ ] **Step 1: Replace NocoDB museum fetch with ERP**
- Remove: `discoverTableIds()`, `fetchNocoDBTable()`, `fetchFromNocoDB()`, `MuseumMapEntry`, NocoDB museum env var reads
- Keep: NocoDB config for PilgrimStats path only (`fetchPilgrimStats()` unchanged)
- Import `fetchFromERP` from erpService
- Update `fetchData()` and `refreshData()` to call `fetchFromERP()` instead of `fetchFromNocoDB()`
- Update config check: `if (!ERP_API_URL || !ERP_API_CODE || !ERP_USERNAME || !ERP_PASSWORD)` → throw DataError('config')
- [ ] **Step 2: Replace all district→channel in data functions**
- `filterData()`: `filters.district``filters.channel`, `row.district``row.channel`
- `filterDataByDateRange()`: same
- `groupByDistrict()`→ rename to `groupByChannel()`, change `row.district``row.channel`
- `getUniqueDistricts()`→ rename to `getUniqueChannels()`, change `r.district``r.channel`
- `getDistrictMuseumMap()`**delete**
- `getMuseumsForDistrict()`→ replace with `getUniqueMuseums(data: MuseumRecord[]): string[]` returning all unique museum names sorted
- Remove all `revenue_incl_tax` fallback references (e.g. `|| row.revenue_incl_tax || 0` in groupBy functions and calculateMetrics)
- [ ] **Step 3: Update type imports**
Remove unused NocoDB types from the import block. Add imports for new ERP-related types if needed.
- [ ] **Step 4: Commit**
```bash
git add src/services/dataService.ts
git commit -m "refactor: replace NocoDB museum fetch with ERP API, district→channel"
```
---
### Task 7: Update Dashboard Component
**Files:**
- Modify: `src/components/Dashboard.tsx`
- [ ] **Step 1: Update imports**
- `getUniqueDistricts``getUniqueChannels`
- `getDistrictMuseumMap` → remove
- `getMuseumsForDistrict``getUniqueMuseums`
- `groupByDistrict``groupByChannel`
- [ ] **Step 2: Update filter state and controls**
- `defaultFilters`: `district: 'all'``channel: 'all'`
- `filterKeys` array (line 32): `'district'``'channel'` — this controls URL param serialization
- All `filters.district``filters.channel`
- District `<select>` → Channel `<select>` with `t('filters.channel')` label
- Museum `<select>`: no longer cascaded from district/channel — just show all `getUniqueMuseums(data)`
- `availableMuseums` memo: `getMuseumsForDistrict(districtMuseumMap, filters.district)``getUniqueMuseums(data)`
- Remove `districtMuseumMap` memo entirely
- [ ] **Step 3: Update charts**
- `districtData``channelData` using `groupByChannel(filteredData, includeVAT)`
- "District Performance" → "Channel Performance" using `t('dashboard.channelPerformance')`
- Chart carousel label `t('charts.district')``t('charts.channel')` (line 88)
- Capture rate chart: `filters.district``filters.channel`, `r.district``r.channel`
- [ ] **Step 4: Update quarterly table**
Replace `filters.district``filters.channel` and `r.district``r.channel`
- [ ] **Step 5: Commit**
```bash
git add src/components/Dashboard.tsx
git commit -m "feat: update Dashboard — channel replaces district, new channel chart"
```
---
### Task 8: Update Comparison Component
**Files:**
- Modify: `src/components/Comparison.tsx`
- [ ] **Step 1: Update imports and filter references**
- Replace `getUniqueDistricts``getUniqueChannels`
- Remove `getDistrictMuseumMap`, `getMuseumsForDistrict`
- Add `getUniqueMuseums`
- Replace all `district` filter references with `channel` (includes URL params, `DateRangeFilters` usage, `<select>` elements)
- Museum filter: use `getUniqueMuseums(data)` (no longer cascaded)
- [ ] **Step 2: Commit**
```bash
git add src/components/Comparison.tsx
git commit -m "feat: update Comparison — channel replaces district"
```
---
### Task 9: Update Slides Component (FULL REFACTOR)
**Files:**
- Modify: `src/components/Slides.tsx`
This is a significant change — Slides.tsx has 30+ district references across 10+ call sites including function signatures, prop interfaces, and HTML export generation.
- [ ] **Step 1: Update imports**
- Replace `getUniqueDistricts``getUniqueChannels`
- Replace `getDistrictMuseumMap` → remove
- Replace `getMuseumsForDistrict``getUniqueMuseums`
- Remove import of `DistrictMuseumMap` type
- [ ] **Step 2: Update component interfaces**
All three interfaces pass `districts: string[]` and `districtMuseumMap: DistrictMuseumMap`:
```typescript
// SlideEditorProps (line 25): districts→channels, remove districtMuseumMap
interface SlideEditorProps {
slide: SlideConfig;
onUpdate: (updates: Partial<SlideConfig>) => void;
channels: string[];
museums: string[]; // flat list, independent of channel
data: MuseumRecord[];
chartTypes: ChartTypeOption[];
metrics: MetricOption[];
}
// SlidePreviewProps (line 35): same pattern
interface SlidePreviewProps {
slide: SlideConfig;
data: MuseumRecord[];
channels: string[];
museums: string[];
metrics: MetricOption[];
}
// PreviewModeProps (line 43): same pattern
interface PreviewModeProps {
slides: SlideConfig[];
data: MuseumRecord[];
channels: string[];
museums: string[];
currentSlide: number;
setCurrentSlide: React.Dispatch<React.SetStateAction<number>>;
onExit: () => void;
metrics: MetricOption[];
}
```
- [ ] **Step 3: Update Slides() main function**
- `districts` memo → `channels` using `getUniqueChannels(data)`
- `districtMuseumMap` memo → `museums` using `getUniqueMuseums(data)`
- `defaultSlide.district: 'all'``channel: 'all'`
- Update all prop passing: `districts={districts} districtMuseumMap={districtMuseumMap}``channels={channels} museums={museums}`
- [ ] **Step 4: Update SlideEditor function**
- `getMuseumsForDistrict(districtMuseumMap, slide.district)` → just use `museums` prop directly
- Filter label: `t('filters.district')``t('filters.channel')`
- `<select>` for district → channel: `slide.district``slide.channel`, `onUpdate({ district: ... })``onUpdate({ channel: ... })`
- Museum select: no longer cascaded, just show all `museums`
- [ ] **Step 5: Update SlidePreview function**
- `district: slide.district``channel: slide.channel` in `filterDataByDateRange` calls
- [ ] **Step 6: Update generateSlideHTML and generateChartScripts**
- Function signatures: remove `districts: string[]` and `districtMuseumMap: DistrictMuseumMap` params, add `channels: string[]` and `museums: string[]`
- Internal references: `slide.district``slide.channel`
- `filterDataByDateRange` calls: `district:``channel:`
- [ ] **Step 7: Commit**
```bash
git add src/components/Slides.tsx
git commit -m "feat: update Slides — full district→channel refactor across all interfaces"
```
---
### Task 10: Update Locale Files
**Files:**
- Modify: `src/locales/en.json`
- Modify: `src/locales/ar.json`
- [ ] **Step 1: Update English translations**
Replace/add:
```json
{
"filters": {
"channel": "Channel",
"allChannels": "All Channels"
},
"charts": {
"channel": "Channel"
},
"dashboard": {
"subtitle": "Museum analytics from Hono ERP",
"channelPerformance": "Channel Performance"
},
"errors": {
"config": "The dashboard is not configured. Please set up the ERP API connection."
}
}
```
Remove: `filters.district`, `filters.allDistricts`, `charts.district`, `dashboard.districtPerformance`.
- [ ] **Step 2: Update Arabic translations**
```json
{
"filters": {
"channel": "القناة",
"allChannels": "جميع القنوات"
},
"charts": {
"channel": "القناة"
},
"dashboard": {
"subtitle": "تحليلات المتاحف من نظام Hono ERP",
"channelPerformance": "أداء القنوات"
},
"errors": {
"config": "لم يتم تهيئة لوحة المعلومات. يرجى إعداد اتصال ERP API."
}
}
```
Remove: `filters.district`, `filters.allDistricts`, `charts.district`, `dashboard.districtPerformance`.
- [ ] **Step 3: Commit**
```bash
git add src/locales/en.json src/locales/ar.json
git commit -m "feat: update locale files — channel replaces district, ERP error messages"
```
---
### Task 11: Build Verification & Smoke Test
- [ ] **Step 1: Run `npm run build` to verify TypeScript compiles**
All `district` references should be gone. Any remaining will cause TS errors. Also search for `revenue_incl_tax` and `museum_code` — these should be fully removed.
- [ ] **Step 2: Run the dev server and verify**
```bash
npm run dev
```
Check:
1. Dashboard loads with data from ERP API (not NocoDB)
2. Channel filter shows: HiHala Website/App, B2B, POS, Safiyyah POS, Standalone, Mobile, Viva, IT
3. Museum filter shows: Revelation Exhibition, Creation Story Museum, Holy Quraan Museum, Trail To Hira Cave, Makkah Greets Us, VIP Experience
4. Museum and channel filters work independently (not cascaded)
5. All charts render: revenue trend, visitors by museum, revenue by museum, quarterly YoY, **channel performance** (was district), capture rate
6. Comparison page works with channel filter
7. Slides page works — create/preview/export with channel filter
8. PilgrimStats loads from NocoDB (capture rate chart shows pilgrim data)
9. Cache works (reload → uses cached data)
10. Offline fallback works (disconnect → shows cached data with offline badge)
- [ ] **Step 3: Commit any fixes**
---
## Appendix: All 47 Known Product Descriptions
For reference when building the mapping in Task 3.
**Revelation Exhibition (12 products):**
1. `Revelation Exhibition - Child`
2. `Revelation Exhibition - Groups`
3. `Revelation Exhibition - Individuals`
4. `Revelation Exhibition - POD`
5. `Revelation Exhibition and Trail To Hiraa Cave - Individuals` _(combo → Revelation)_
6. `معرض الوحي - أطفال | Revelation Exhibition - Child`
7. `معرض الوحي - أفراد | Revelation Exhibition - Individuals`
8. `معرض الوحي - المجموعات | Revelation Exhibition - Group`
9. `معرض الوحي - ذوي الإعاقة | Revelation Exhibition - POD`
10. `معرض الوحي - مجموعات| Revelation Exhibition - Groups`
11. `تذكرة دخول أفراد - معرض الوحي | متحف القرآن الكريم` _(combo → Revelation, because الوحي matched first)_
12. `تذكرة دخول مجموعات - معرض الوحي | متحف القرآن الكريم` _(combo → Revelation)_
**Creation Story Museum (21 products):**
1. `Creation Story - Groups`
2. `Creation Story - Individuals`
3. `Creation Story - Groups` _(extra space variant)_
4. `Creation Story - Indviduals - Open Date` _(typo "Indviduals" is in the source data)_
5. `Creation Story Group`
6. `Creation Story Individual`
7. `Creation Story School`
8. `متحف قصة الخلق - أفراد | Creation Story Museum - Individuals`
9. `متحف قصة الخلق - مجموعات| Creation Story Museum - Group`
10. `متحف قصة الخلق - مدرسة | Creation Story Museum - School`
11. `متحف قصة الخلق - أفراد - خصم بولمان زمزم`
12. `متحف قصة الخلق - مجموعات - خصم بولمان زمزم`
13. `تذكرة دخول متحف قصة الخلق (جامعة) | Creation Story Museum`
14. `تذكرة دخول متحف قصة الخلق مخفضة | Creation Story Museum`
15. `تذكرة دخول متحف قصة الخلق مخفضة 10 | Creation Story Museum`
16. `تذكرة دخول متحف قصة الخلق مخفضة 11.5 | Creation Story Museum`
17. `تذكرة دخول متحف قصة الخلق مخفضة 15 | Creation Story Museum`
18. `تذكرة دخول متحف قصة الخلق مخفضة 19 | Creation Story Museum`
19. `تذكرة مجانية دخول متحف قصة الخلق (ترويجية) | Creation Sto`
20. `تذكرة مجانية دخول متحف قصة الخلق (ذوي الهمم) | Creation Sto`
21. ` تذكرة مجانية دخول متحف قصة الخلق (أطفال) | Creation Story ` _(leading space)_
**Holy Quraan Museum (8 products):**
1. `Holy Quraan Museum - Child`
2. `Holy Quraan Museum - Child | متحف القرآن الكريم - أطفال`
3. `Holy Quraan Museum - Groups`
4. `Holy Quraan Museum - Groups | متحف القرآن الكريم - المجموعات`
5. `Holy Quraan Museum - Individu | متحف القرآن الكريم - أفراد`
6. `Holy Quraan Museum - Individuals`
7. `Holy Quraan Museum - POD`
8. `Holy Quraan Museum - POD | متحف القرآن الكريم - ذوي الإعاقة`
**Trail To Hira Cave (3 products):**
1. `Trail To Hira Cave - Car | غار حراء - الصعود بالسيارة`
2. `Trail To Hira Cave - Walking`
3. `Trail To Hira Cave - Walking | غار حراء - الصعود على الأقدام`
**Makkah Greets Us (1 product):**
1. `Makkah Greets us - Entry Ticket`
**VIP Experience (1 product):**
1. `VIP Experience`
**Total: 12 + 21 + 8 + 3 + 1 + 1 = 46 products**