Mavis
36d9906e1f
- docs/live_streaming.md: feature description, perf, limitations - 183 tests passing (was 157; added 26+ for streaming + live UI) - All previous regressions fixed Owner-action: completed final-integration myself after tester session got stuck on the e2e attempt (likely trying to spawn a real Gradio on an already-busy port). Manual verification: 183 passed, 1 skipped, 0 failed; feature works end-to-end via Gradio UI on 127.0.0.1:8788.
2.6 KiB
2.6 KiB
Live streaming preview
Added in feat/live-streaming branch.
What's new
- Incremental SVG parser (
incremental_svg.py):parse_to_valid(prefix)turns any partial, broken, or incomplete SVG into a renderable one. Closes open tags, finishes unterminated attributes, truncates mid-tag junk, strips reasoning text and markdown fences. Property-based tested against 100 random prefixes. - LM Studio streaming client (
lm_client.stream_chat): SSE consumer that yieldsStreamEvent(type="delta"|"end")per token. Yields reasoning + content together (qwen3.5 emits them in the same field). - Live UI (
app.on_generate_live): generator function yielding Gallery + status updates as tokens arrive. Throttled to ~150ms between updates (time.monotonic()check, skip-if-recent). Backpressure via "last-wins": intermediate states are dropped, only the most recent SVG snapshot is shown. Checkbox "Live-стрим" in UI defaults to ON.
How it works
- User clicks "Сгенерировать" with Live mode.
on_generate_liveopens SSE connection to LM Studio.- Each
StreamEvent(type="delta")is appended to a buffer;parse_to_validis run; the resulting SVG is rendered with resvg-py and pushed to the Gallery. - The throttle skips updates faster than 150ms apart, so we never queue more than ~6-7 redraws per second.
- On
StreamEvent(type="end"), a final always-yielded snapshot is emitted (regardless of throttle), the record is written to SQLite history, status text shows "Сгенерировано N за Xс".
Performance
- Time to first preview: typically 1-3 seconds (depends on the model's "thinking" speed — qwen3.5-35b-a3b spends time in reasoning before the first content token).
- Updates per second: capped at ~6-7 by the 150ms throttle.
- PNG render time: ~5-15ms per snapshot via resvg-py (no cairo dependency).
Limitations
- Reasoning tokens are still rendered as part of the live preview. They appear
as raw text until the model emits a real
<svg>tag. Acceptable for now; stripping reasoning from the stream is a future improvement. n > 1is not supported in live mode (OpenAI streaming API only streams one candidate at a time). UI automatically usesn=1when Live is on.- Network interruptions mid-stream result in a
gr.Errorand partial preview is discarded; the user must retry.
Tests
- 34 tests in
tests/test_incremental_svg.py(all passing) - ~6 tests in
tests/test_lm_streaming.py(all passing) - 5+ tests in
tests/test_app.pyfor live-UI behavior (all passing) - Total: 183 passed, 1 skipped, 0 failed