Mavis
36d9906e1f
- docs/live_streaming.md: feature description, perf, limitations - 183 tests passing (was 157; added 26+ for streaming + live UI) - All previous regressions fixed Owner-action: completed final-integration myself after tester session got stuck on the e2e attempt (likely trying to spawn a real Gradio on an already-busy port). Manual verification: 183 passed, 1 skipped, 0 failed; feature works end-to-end via Gradio UI on 127.0.0.1:8788.
56 lines
2.6 KiB
Markdown
56 lines
2.6 KiB
Markdown
# Live streaming preview
|
|
|
|
Added in `feat/live-streaming` branch.
|
|
|
|
## What's new
|
|
|
|
- **Incremental SVG parser** (`incremental_svg.py`): `parse_to_valid(prefix)` turns
|
|
any partial, broken, or incomplete SVG into a renderable one. Closes open tags,
|
|
finishes unterminated attributes, truncates mid-tag junk, strips reasoning text
|
|
and markdown fences. Property-based tested against 100 random prefixes.
|
|
- **LM Studio streaming client** (`lm_client.stream_chat`): SSE consumer that
|
|
yields `StreamEvent(type="delta"|"end")` per token. Yields reasoning + content
|
|
together (qwen3.5 emits them in the same field).
|
|
- **Live UI** (`app.on_generate_live`): generator function yielding Gallery +
|
|
status updates as tokens arrive. Throttled to ~150ms between updates
|
|
(`time.monotonic()` check, skip-if-recent). Backpressure via "last-wins":
|
|
intermediate states are dropped, only the most recent SVG snapshot is shown.
|
|
Checkbox "Live-стрим" in UI defaults to ON.
|
|
|
|
## How it works
|
|
|
|
1. User clicks "Сгенерировать" with Live mode.
|
|
2. `on_generate_live` opens SSE connection to LM Studio.
|
|
3. Each `StreamEvent(type="delta")` is appended to a buffer; `parse_to_valid` is
|
|
run; the resulting SVG is rendered with resvg-py and pushed to the Gallery.
|
|
4. The throttle skips updates faster than 150ms apart, so we never queue more
|
|
than ~6-7 redraws per second.
|
|
5. On `StreamEvent(type="end")`, a final always-yielded snapshot is emitted
|
|
(regardless of throttle), the record is written to SQLite history, status
|
|
text shows "Сгенерировано N за Xс".
|
|
|
|
## Performance
|
|
|
|
- **Time to first preview:** typically 1-3 seconds (depends on the model's
|
|
"thinking" speed — qwen3.5-35b-a3b spends time in reasoning before the first
|
|
content token).
|
|
- **Updates per second:** capped at ~6-7 by the 150ms throttle.
|
|
- **PNG render time:** ~5-15ms per snapshot via resvg-py (no cairo dependency).
|
|
|
|
## Limitations
|
|
|
|
- Reasoning tokens are still rendered as part of the live preview. They appear
|
|
as raw text until the model emits a real `<svg>` tag. Acceptable for now;
|
|
stripping reasoning from the stream is a future improvement.
|
|
- `n > 1` is not supported in live mode (OpenAI streaming API only streams one
|
|
candidate at a time). UI automatically uses `n=1` when Live is on.
|
|
- Network interruptions mid-stream result in a `gr.Error` and partial preview
|
|
is discarded; the user must retry.
|
|
|
|
## Tests
|
|
|
|
- 34 tests in `tests/test_incremental_svg.py` (all passing)
|
|
- ~6 tests in `tests/test_lm_streaming.py` (all passing)
|
|
- 5+ tests in `tests/test_app.py` for live-UI behavior (all passing)
|
|
- **Total:** 183 passed, 1 skipped, 0 failed
|