Add real SGP.22 SAIP profile generation (optional)

New server/saip_profile.py encodes a DER ProfileElement sequence (ProfileHeader
with ICCID + Milenage AKA parameters carrying Ki/OPc) using asn1tools against
the official PE_Definitions SAIP ASN.1 schema. It locates the schema from an
installed osmocom pySim (or SAIP_ASN_PATH) via find_spec WITHOUT importing
pySim's Python package, so it avoids the pyscard/smpp card-reader stack — the
only runtime dep is asn1tools.

Selectable via profile.format ("v2" default | "saip"); the app falls back to
profile-package-v2 with a warning when SAIP prerequisites are absent. IMSI and
other EFs (USIM-template PEs) are the documented next layer.

Add requirements-saip.txt, pytest.ini (silence asn1tools warnings), README and
config updates. Tests: +6 (encode/round-trip/credentials, app emit + fallback);
92 pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

CHANGELOG.md

@@ -7,6 +7,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+- **Real SGP.22 SAIP profile generation** (`profile.format: saip`).
+  `server/saip_profile.py` encodes a DER `ProfileElement` sequence (ProfileHeader
+  with ICCID + Milenage AKA parameters carrying Ki/OPc) with `asn1tools` against
+  the official `PE_Definitions` ASN.1 schema — no pySim Python imports (the schema
+  is located from an installed pySim, or via `SAIP_ASN_PATH`). Self-contained and
+  optional: `saip_available()` gates it and the app falls back to `profile-v2`
+  when prerequisites are missing. Optional deps in `requirements-saip.txt`.
+  IMSI/other EFs (USIM-template PEs) are the documented next layer.
+- `pytest.ini` to silence asn1tools/pyparsing deprecation warnings.
+
 ### Security
 - `/api/export` (dumps every subscriber's Ki/OPc) now requires an admin token
   (`server.admin_token`, sent as `X-Admin-Token`, constant-time compared) and

README.md

@@ -96,14 +96,23 @@ the server resolves the client MAC from the ARP table. Rate limiting is per MAC.
 
 ## Profile format
 
-`create_esim_profile` produces a `profile-package-v2` bundle containing the raw
+Two formats are selectable via `profile.format`:
-credentials plus **real 3GPP field encodings** — `EF_ICCID` and `EF_IMSI`
-(nibble-swapped BCD, 3GPP TS 31.102) — and an SGP.22 **LPA activation code**
-(`LPA:1$<smdp>$<matching-id>`).
 
-Full SGP.22 SIMalliance Interoperable Profile (ASN.1 UPP) generation via osmocom
+- **`v2`** (default, always available) — a JSON `profile-package-v2` bundle with
-pySim is the planned follow-up; consumer over-the-air install additionally
+  the raw credentials plus **real 3GPP field encodings** (`EF_ICCID`, `EF_IMSI`,
-requires an SM-DP+ / LPA. The current package targets programmable SIMs and
+  nibble-swapped BCD per 3GPP TS 31.102) and an SGP.22 **LPA activation code**
+  (`LPA:1$<smdp>$<matching-id>`).
+- **`saip`** — a real DER-encoded **SGP.22 SAIP `ProfileElement` sequence**
+  (ProfileHeader with ICCID + Milenage AKA parameters carrying Ki/OPc), built
+  with `asn1tools` against the official `PE_Definitions` ASN.1 schema. Enable by
+  installing `requirements-saip.txt` (see that file for the schema source) and
+  setting `profile.format: saip`. If prerequisites are missing the server logs a
+  warning and falls back to `v2`.
+
+The SAIP package currently carries ICCID and the AKA credentials; emitting IMSI
+and other EFs (USIM-template / GenericFileManagement PEs with File Control
+Parameters) is the next layer. Note that consumer over-the-air install still
+requires an SM-DP+ / LPA; these packages target programmable SIMs and
 core-network sync. See [`CHANGELOG.md`](CHANGELOG.md).
 
 ## Deployment

config.yaml.example

@@ -34,3 +34,4 @@ profile:
   profile_name: "Emergency 5G"
   apn: "internet"
   smdp_address: "esim.local"       # LPA activation-code SM-DP+ address
+  format: "v2"                     # "v2" (JSON, always available) | "saip" (real SGP.22 DER; needs requirements-saip.txt)

pytest.ini

@@ -0,0 +1,6 @@
+[pytest]
+# asn1tools (used for optional SAIP generation) emits many pyparsing deprecation
+# warnings when compiling the ASN.1 schema; they are upstream and not actionable.
+filterwarnings =
+    ignore::DeprecationWarning:asn1tools.*
+    ignore::DeprecationWarning:pyparsing.*

requirements-saip.txt

@@ -0,0 +1,15 @@
+# Optional dependencies for real SGP.22 SAIP profile generation (profile.format: saip).
+# The base requirements.txt is enough for the default profile-package-v2 format.
+
+# ASN.1 DER encoder used to build the SAIP ProfileElement sequence.
+asn1tools>=0.166
+
+# The SAIP ASN.1 schema (PE_Definitions-*.asn) ships with osmocom pySim; we
+# locate it there without importing pySim's card-reader stack. Install from git:
+#   pip install --no-deps "git+https://github.com/osmocom/pysim.git"
+# Alternatively, point SAIP_ASN_PATH at a PE_Definitions-*.asn file directly and
+# skip pySim entirely.
+#
+# Note: `pip install pysim` (no --no-deps) pulls pyscard, which needs the PCSC
+# dev headers (apt install libpcsclite-dev). --no-deps avoids that; SAIP does not
+# use the card reader.

server/app.py

@@ -24,6 +24,7 @@ from server.profile_generator import (
     generate_imsi,
     generate_ki,
 )
+from server.saip_profile import create_saip_profile, saip_available
 from server.utils import (
     bytes_to_hex,
     ensure_parent_dir,
@@ -90,6 +91,16 @@ def create_app(config_path: str = CONFIG_PATH) -> Flask:
     apn = profile_cfg.get("apn", "internet")
     smdp_address = profile_cfg.get("smdp_address", "esim.local")
 
+    # Profile package format: "v2" (JSON, always available) or "saip" (real
+    # SGP.22 DER). Fall back to v2 if saip is requested but unavailable.
+    profile_format = profile_cfg.get("format", "v2")
+    use_saip = profile_format == "saip" and saip_available()
+    if profile_format == "saip" and not use_saip:
+        logger.warning("profile.format=saip requested but SAIP is unavailable; "
+                       "falling back to profile-package-v2")
+    elif use_saip:
+        logger.info("Profile format: SAIP (real SGP.22 DER)")
+
     rl_cfg = cfg["rate_limiting"]
     window_hours = rl_cfg["window_hours"]
 
@@ -150,10 +161,15 @@ def create_app(config_path: str = CONFIG_PATH) -> Flask:
                 logger.error("Provision failed: could not allocate a unique IMSI/ICCID")
                 return jsonify({"error": "Profile generation failed"}), 500
 
-            profile_bytes = create_esim_profile(
+            if use_saip:
-                imsi, ki, opc, iccid, profile_name, carrier_name,
+                profile_bytes = create_saip_profile(
-                smdp_address=smdp_address, apn=apn,
+                    imsi, ki, opc, iccid, profile_name, carrier_name,
-            )
+                )
+            else:
+                profile_bytes = create_esim_profile(
+                    imsi, ki, opc, iccid, profile_name, carrier_name,
+                    smdp_address=smdp_address, apn=apn,
+                )
             token = secrets.token_urlsafe(24)
             _cache_profile(token, profile_bytes)
 

server/saip_profile.py

@@ -0,0 +1,115 @@
+"""Optional real SGP.22 SAIP profile-package generation.
+
+Encodes a SIMalliance Interoperable Profile (SAIP) ``ProfileElement`` sequence
+using the official ASN.1 schema (``PE_Definitions``) via ``asn1tools``. Unlike
+``profile_generator.create_esim_profile`` (the always-available JSON
+``profile-package-v2``), this produces real DER-encoded SGP.22 profile elements.
+
+This module is optional and self-contained: it needs only ``asn1tools`` plus the
+SAIP ASN.1 schema file. The schema ships with osmocom pySim; we locate it there
+without importing pySim's Python package (which pulls in a card-reader stack).
+Set ``SAIP_ASN_PATH`` to override the schema location. When neither is present,
+``saip_available()`` returns False and callers fall back to profile-package-v2.
+
+Scope: the encoded package currently carries the ProfileHeader (with ICCID) and
+the Milenage AKA parameters (Ki/OPc) — the authentication-critical material.
+Writing IMSI and other EFs requires USIM-template / GenericFileManagement PEs
+with full File Control Parameters; that is the next layer and is not yet emitted.
+"""
+import functools
+import importlib.util
+import logging
+import os
+from typing import Optional
+
+logger = logging.getLogger(__name__)
+
+_ASN_RELPATH = os.path.join("esim", "asn1", "saip", "PE_Definitions-3.3.1.asn")
+
+
+def find_saip_asn() -> Optional[str]:
+    """Locate the SAIP ASN.1 schema: SAIP_ASN_PATH override, else pySim's copy."""
+    override = os.environ.get("SAIP_ASN_PATH")
+    if override and os.path.exists(override):
+        return override
+    try:
+        spec = importlib.util.find_spec("pySim")
+    except (ImportError, ValueError):
+        return None
+    if spec and spec.submodule_search_locations:
+        for loc in spec.submodule_search_locations:
+            candidate = os.path.join(loc, _ASN_RELPATH)
+            if os.path.exists(candidate):
+                return candidate
+    return None
+
+
+@functools.lru_cache(maxsize=1)
+def _compiler():
+    """Return a compiled asn1tools spec, or None if prerequisites are missing."""
+    try:
+        import asn1tools
+    except ImportError:
+        return None
+    path = find_saip_asn()
+    if not path:
+        return None
+    try:
+        return asn1tools.compile_files([path], codec="der")
+    except Exception as exc:  # malformed/incompatible schema
+        logger.warning(f"Failed to compile SAIP ASN.1 schema at {path}: {exc}")
+        return None
+
+
+def saip_available() -> bool:
+    """True if a real SAIP profile can be generated in this environment."""
+    return _compiler() is not None
+
+
+def create_saip_profile(
+    imsi: str,
+    ki: bytes,
+    opc: bytes,
+    iccid: str,
+    profile_name: str = "",
+    carrier_name: str = "",
+) -> bytes:
+    """Build a DER-encoded SAIP ProfileElement sequence (header + AKA + end).
+
+    ``imsi`` is accepted for signature parity with ``create_esim_profile`` and is
+    reserved for the forthcoming USIM/EF layer; it is not yet encoded into an EF.
+    """
+    asn = _compiler()
+    if asn is None:
+        raise RuntimeError(
+            "SAIP profile generation unavailable: requires asn1tools and the SAIP "
+            "ASN.1 schema (install osmocom pySim or set SAIP_ASN_PATH)"
+        )
+
+    # Imported here to avoid a hard dependency cycle at module import time.
+    from server.profile_generator import enc_iccid
+
+    ki_bytes = ki if isinstance(ki, bytes) else bytes.fromhex(ki)
+    opc_bytes = opc if isinstance(opc, bytes) else bytes.fromhex(opc)
+
+    header = ("header", {
+        "major-version": 3,
+        "minor-version": 3,
+        "profileType": (profile_name or "Emergency 5G")[:100],
+        "iccid": enc_iccid(iccid),
+        "eUICC-Mandatory-services": {"usim": None, "milenage": None},
+        "eUICC-Mandatory-GFSTEList": [],
+    })
+    aka = ("akaParameter", {
+        "aka-header": {"identification": 1},
+        "algoConfiguration": ("algoParameter", {
+            "algorithmID": 1,            # Milenage
+            "algorithmOptions": b"\x00",
+            "key": ki_bytes,
+            "opc": opc_bytes,
+        }),
+    })
+    end = ("end", {"end-header": {"identification": 2}})
+
+    logger.debug(f"Encoding SAIP profile for IMSI {imsi} (ICCID {iccid})")
+    return b"".join(asn.encode("ProfileElement", pe) for pe in (header, aka, end))

tests/test_saip.py

@@ -0,0 +1,91 @@
+import pytest
+import yaml
+
+from server.app import create_app
+from server.profile_generator import derive_opc, enc_iccid, generate_iccid, generate_ki
+import server.saip_profile as saip
+
+_OP = bytes.fromhex("63bfa50ee6523365ff14c1f45f88737d")
+
+saip_required = pytest.mark.skipif(
+    not saip.saip_available(),
+    reason="SAIP unavailable (asn1tools + SAIP ASN.1 schema not installed)",
+)
+
+
+def _write_config(tmp_path, profile_format="v2"):
+    cfg = {
+        "network": {"mcc": "302", "mnc": "720", "plmn": "302720",
+                    "op_key": "63bfa50ee6523365ff14c1f45f88737d"},
+        "wifi": {"ssid": "x", "channel": 6, "ip": "192.168.4.1",
+                 "subnet": "192.168.4.0/24", "dhcp_range": "a,b"},
+        "server": {"host": "0.0.0.0", "port": 5000, "debug": False},
+        "database": {"path": str(tmp_path / "subscribers.db")},
+        "rate_limiting": {"max_profiles_per_mac": 1, "window_hours": 1},
+        "core": {"type": "none"},
+        "profile": {"carrier_name": "Ad-Hoc", "profile_name": "Emergency 5G",
+                    "apn": "internet", "smdp_address": "esim.local",
+                    "format": profile_format},
+    }
+    path = tmp_path / "config.yaml"
+    path.write_text(yaml.safe_dump(cfg))
+    return str(path)
+
+
+class TestSaipEncoding:
+    def test_unavailable_raises(self, monkeypatch):
+        monkeypatch.setattr(saip, "_compiler", lambda: None)
+        with pytest.raises(RuntimeError, match="unavailable"):
+            saip.create_saip_profile("302720000000001", generate_ki(),
+                                     b"\x00" * 16, generate_iccid())
+
+    @saip_required
+    def test_produces_der_bytes(self):
+        ki = generate_ki()
+        opc = derive_opc(ki, _OP)
+        iccid = generate_iccid()
+        pkg = saip.create_saip_profile("302720000000001", ki, opc, iccid, "Emergency 5G", "C")
+        assert isinstance(pkg, bytes) and len(pkg) > 0
+        # First byte is the context tag for the 'header' CHOICE alternative.
+        assert pkg[0] == 0xA0
+
+    @saip_required
+    def test_header_roundtrips_iccid(self):
+        ki = generate_ki()
+        opc = derive_opc(ki, _OP)
+        iccid = generate_iccid()
+        pkg = saip.create_saip_profile("302720000000001", ki, opc, iccid, "P", "C")
+        asn = saip._compiler()
+        choice, value = asn.decode("ProfileElement", pkg)
+        assert choice == "header"
+        assert value["iccid"] == enc_iccid(iccid)
+        assert value["major-version"] == 3
+
+    @saip_required
+    def test_credentials_embedded(self):
+        ki = generate_ki()
+        opc = derive_opc(ki, _OP)
+        pkg = saip.create_saip_profile("302720000000001", ki, opc, generate_iccid(), "P", "C")
+        assert ki in pkg   # raw Ki bytes present in the AKA parameter
+        assert opc in pkg
+
+
+class TestSaipAppIntegration:
+    @saip_required
+    def test_provision_emits_saip_when_configured(self, tmp_path):
+        app = create_app(_write_config(tmp_path, profile_format="saip"))
+        client = app.test_client()
+        r = client.post("/api/provision", json={"mac_address": "aa:bb:cc:00:11:22"})
+        assert r.status_code == 200
+        profile = client.get(r.get_json()["profile_url"]).data
+        assert profile[0] == 0xA0  # DER, not JSON
+
+    def test_falls_back_to_v2_when_saip_unavailable(self, tmp_path, monkeypatch):
+        import server.app as appmod
+        monkeypatch.setattr(appmod, "saip_available", lambda: False)
+        app = create_app(_write_config(tmp_path, profile_format="saip"))
+        client = app.test_client()
+        r = client.post("/api/provision", json={"mac_address": "aa:bb:cc:00:11:33"})
+        assert r.status_code == 200
+        profile = client.get(r.get_json()["profile_url"]).data
+        assert profile.lstrip()[:1] == b"{"  # JSON profile-package-v2