# eSIM Provisioning System - Project Setup Instructions

## Project Overview
Build a closed-loop eSIM provisioning system for emergency/NTN 5G deployments. System runs on a UE device (Android or Raspberry Pi) that connects to a 5G gNB and provides WiFi AP with automatic eSIM provisioning.

## Repository Structure
Create the following directory structure:

```
esim-provisioning/
├── README.md
├── requirements.txt
├── config.yaml.example
├── docker-compose.yml (optional)
├── server/
│   ├── __init__.py
│   ├── app.py                      # Main Flask application
│   ├── profile_generator.py        # eSIM profile generation using pySim
│   ├── database.py                 # SQLite database interface
│   ├── core_adapters/
│   │   ├── __init__.py
│   │   ├── base.py                 # Base adapter class
│   │   ├── open5gs.py              # Open5GS adapter
│   │   └── free5gc.py              # free5gc adapter
│   ├── models.py                   # Database models
│   └── utils.py                    # Helper functions
├── portal/
│   ├── index.html                  # Captive portal landing page
│   ├── css/
│   │   └── style.css
│   ├── js/
│   │   └── script.js
│   └── assets/
│       └── (images if needed)
├── scripts/
│   ├── init_db.py                  # Initialize database
│   ├── sync_subscribers.py         # Sync to core network
│   ├── export_subscribers.py       # Export subscriber data
│   └── setup_wifi_ap.sh            # WiFi AP setup (for Pi)
├── systemd/
│   ├── esim-provisioning.service
│   └── captive-portal.service
├── configs/
│   ├── nginx.conf                  # nginx captive portal config
│   ├── dnsmasq.conf               # DNS/DHCP config
│   └── hostapd.conf               # WiFi AP config (for Pi)
├── android/
│   ├── README.md                   # Android setup instructions
│   └── termux_setup.sh            # Termux installation script
└── tests/
    ├── test_profile_generator.py
    ├── test_database.py
    └── test_core_adapters.py
```

## Step 1: Initialize Project

Create the repository structure and initialize Python virtual environment:

```bash
# Create project directory
mkdir esim-provisioning
cd esim-provisioning

# Initialize git
git init

# Create virtual environment
python3 -m venv venv
source venv/bin/activate

# Create all directories
mkdir -p server/core_adapters portal/css portal/js portal/assets scripts systemd configs android tests
```

## Step 2: Create Requirements File

Create `requirements.txt` with:
```
Flask==3.0.0
pySim==1.0.0
cryptography==41.0.7
pymongo==4.6.0
requests==2.31.0
pyyaml==6.0.1
```

## Step 3: Create Configuration Template

Create `config.yaml.example`:
```yaml
network:
  mcc: "302"
  mnc: "720"
  plmn: "302720"
  op_key: "63bfa50ee6523365ff14c1f45f88737d"  # Hex string

wifi:
  ssid: "Emergency-5G"
  channel: 6
  ip: "192.168.4.1"
  subnet: "192.168.4.0/24"
  dhcp_range: "192.168.4.10,192.168.4.250"

server:
  host: "0.0.0.0"
  port: 5000
  debug: false

database:
  path: "/opt/esim-provisioning/data/subscribers.db"

rate_limiting:
  max_profiles_per_mac: 1
  window_hours: 1

core:
  type: "open5gs"  # or "free5gc"
  mongodb_uri: "mongodb://localhost:27017"
  database_name: "open5gs"

profile:
  carrier_name: "Ad-Hoc Network"
  profile_name: "Emergency 5G"
  apn: "internet"
```

## Step 4: Core Server Components

### database.py
Create database interface with:
- SQLite connection management
- Schema creation (subscribers table)
- CRUD operations for subscribers
- Rate limiting checks by MAC address
- Export functionality

Schema:
```sql
CREATE TABLE subscribers (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    imsi TEXT UNIQUE NOT NULL,
    iccid TEXT UNIQUE NOT NULL,
    ki TEXT NOT NULL,
    opc TEXT NOT NULL,
    mac_address TEXT,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    synced_to_core BOOLEAN DEFAULT 0
);

CREATE INDEX idx_imsi ON subscribers(imsi);
CREATE INDEX idx_mac_address ON subscribers(mac_address);
```

### profile_generator.py
Implement eSIM profile generation:
- Generate random IMSI (15 digits: MCC + MNC + MSIN)
- Generate random Ki (16 bytes)
- Derive OPc from Ki and OP using Milenage algorithm
- Generate ICCID (19-20 digits)
- Create SGP.22 eSIM profile using pySim
- Return profile bytes and credentials

Key functions:
```python
def generate_imsi(mcc, mnc) -> str
def generate_iccid() -> str
def generate_ki() -> bytes
def derive_opc(ki: bytes, op: bytes) -> bytes
def create_esim_profile(imsi, ki, opc, iccid, profile_name, carrier_name) -> bytes
```

### core_adapters/base.py
Define abstract base class:
```python
class CoreAdapter(ABC):
    @abstractmethod
    def add_subscriber(self, imsi, ki, opc):
        pass
    
    @abstractmethod
    def remove_subscriber(self, imsi):
        pass
    
    @abstractmethod
    def list_subscribers(self):
        pass
```

### core_adapters/open5gs.py
Implement Open5GS adapter using MongoDB:
- Connect to MongoDB
- Insert subscriber with proper schema
- Configure slice, APN, AMBR settings
- Handle errors gracefully

### app.py
Main Flask application with endpoints:

```python
GET  /                    # Captive portal redirect
GET  /activate            # Serve captive portal HTML
POST /api/provision       # Generate and provision eSIM
GET  /api/profile/:id     # Download eSIM profile (LPA)
GET  /api/export          # Export subscribers JSON
GET  /api/status          # System status
```

Implement:
- Rate limiting middleware (check MAC address)
- Profile generation workflow
- LPA protocol handling
- Error handling and logging

## Step 5: Captive Portal

### portal/index.html
Simple single-page interface:
- "Activate eSIM" button
- Instructions for installation
- Progress indicator
- Success/error messages
- Mobile-friendly responsive design

### portal/js/script.js
Client-side logic:
- POST to `/api/provision` with MAC address
- Poll for profile generation status
- Trigger eSIM download via LPA URL scheme
- Handle errors

## Step 6: System Scripts

### scripts/init_db.py
Initialize SQLite database with schema

### scripts/sync_subscribers.py
Sync subscribers to core:
```python
# Read from SQLite
# For each unsynced subscriber:
#   - Call core adapter
#   - Mark as synced
#   - Log result
```

### scripts/export_subscribers.py
Export to JSON format:
```json
{
  "exported_at": "2025-02-11T10:30:00Z",
  "count": 123,
  "subscribers": [
    {
      "imsi": "302720123456789",
      "ki": "00112233445566778899aabbccddeeff",
      "opc": "63bfa50ee6523365ff14c1f45f88737d",
      "iccid": "8910302720123456789",
      "created_at": "2025-02-11T09:15:00Z"
    }
  ]
}
```

### scripts/setup_wifi_ap.sh
Bash script for Raspberry Pi WiFi AP setup:
- Install hostapd, dnsmasq, nginx
- Configure network interfaces
- Set up iptables for captive portal
- Enable IP forwarding
- Configure services

## Step 7: Configuration Files

### configs/nginx.conf
Configure captive portal:
- Listen on port 80
- Catch-all server block
- Redirect to /activate
- Proxy /api/* to Flask backend

### configs/dnsmasq.conf
DNS/DHCP configuration:
- Interface binding
- DHCP range
- DNS hijacking (return WiFi AP IP for all domains)
- Lease time

### configs/hostapd.conf
WiFi AP configuration:
- SSID
- Channel
- Interface
- Open network (no password)

## Step 8: Systemd Services

### systemd/esim-provisioning.service
Flask server service:
- Run as dedicated user
- Auto-restart on failure
- Log to journal
- Load config from /etc/esim-provisioning/config.yaml

### systemd/captive-portal.service
Nginx + dnsmasq orchestration

## Step 9: Testing

Create unit tests for:
- Profile generation (valid IMSI, Ki, OPc formats)
- Database operations (CRUD, rate limiting)
- Core adapters (mock MongoDB)
- API endpoints (Flask test client)

## Step 10: Documentation

### README.md
Include:
- Project overview
- System requirements
- Installation instructions (Pi and Android)
- Configuration guide
- Usage instructions
- Troubleshooting
- Architecture diagram
- License

## Implementation Priority

1. **First:** Database layer + models
2. **Second:** Profile generator (without pySim, use mock for testing)
3. **Third:** Flask API with all endpoints
4. **Fourth:** Core adapters (Open5GS first)
5. **Fifth:** Captive portal frontend
6. **Sixth:** Integration with pySim for real profile generation
7. **Seventh:** System scripts and configs
8. **Eighth:** Testing and documentation

## Key Implementation Notes

- Use environment variables or config.yaml for all configuration (never hardcode)
- Log everything (use Python logging module)
- Handle all errors gracefully (no stack traces to user)
- Validate all inputs (IMSI format, MAC address format)
- Use secrets module for cryptographic randomness
- Make core adapters truly modular (easy to add new cores)
- Keep captive portal extremely simple (mobile-first)
- Rate limiting must be robust (prevent abuse)
- Database operations should be atomic
- Profile generation should be fast (<1 second)

## Testing Checklist

- [ ] Can generate valid IMSI
- [ ] Can generate valid Ki/OPc
- [ ] Can create eSIM profile
- [ ] Can store subscriber in database
- [ ] Rate limiting works (same MAC can't request multiple times)
- [ ] Can export subscribers to JSON
- [ ] Open5GS adapter can add subscriber
- [ ] API endpoints return correct responses
- [ ] Captive portal loads on mobile
- [ ] Profile downloads correctly
- [ ] End-to-end: WiFi → provision → 5G connection

## Deployment Targets

**Raspberry Pi:**
- Raspberry Pi 4 (4GB+ RAM)
- Quectel RM500Q 5G modem
- Raspberry Pi OS Lite 64-bit
- Python 3.11+

**Android:**
- Android 10+ with root/Termux
- 5G modem built-in
- Termux + proot environment
- Python 3.11+

## Security Considerations (Future)

Currently implementing with no security for simplicity, but design should allow easy addition of:
- WiFi password authentication
- HTTPS for API endpoints
- Activation code requirements
- Subscriber audit logging
- Profile encryption at rest

## Success Criteria

System is complete when:
1. User connects to WiFi
2. Captive portal loads automatically
3. Click "Activate eSIM" button
4. eSIM profile downloads to device
5. Device installs profile
6. Device connects to 5G using new eSIM
7. Open5GS authenticates successfully

Total user time: <60 seconds

---

## Starting Point

Begin by creating the project structure, then implement in this order:

1. `database.py` - Get data persistence working
2. `profile_generator.py` - Mock implementation first
3. `app.py` - Flask API with all endpoints
4. `portal/index.html` - Simple UI
5. `core_adapters/open5gs.py` - Core integration
6. Integration testing
7. Real pySim integration
8. Deployment scripts

Focus on getting a working end-to-end flow before optimizing or adding features.