VibecoderMcSwaggins commited on
Commit
3d9fb4c
Β·
1 Parent(s): 4927db5

docs: add P1 bug report for streaming spam + API key persistence

Browse files
docs/bugs/P1_MAGENTIC_STREAMING_AND_KEY_PERSISTENCE.md ADDED
@@ -0,0 +1,164 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Bug Report: Magentic Mode Streaming Spam + API Key Persistence
2
+
3
+ ## Status
4
+ - **Date:** 2025-11-29
5
+ - **Reporter:** CLI User
6
+ - **Priority:** P1 (UX Degradation)
7
+ - **Component:** `src/app.py`, `src/orchestrator_magentic.py`
8
+
9
+ ---
10
+
11
+ ## Bug 1: Token-by-Token Streaming Spam
12
+
13
+ ### Symptoms
14
+ When running Magentic (Advanced) mode, the UI shows hundreds of individual lines like:
15
+ ```
16
+ πŸ“‘ STREAMING: Below
17
+ πŸ“‘ STREAMING: is
18
+ πŸ“‘ STREAMING: a
19
+ πŸ“‘ STREAMING: curated
20
+ πŸ“‘ STREAMING: list
21
+ ...
22
+ ```
23
+
24
+ Each token is displayed as a separate streaming event, creating visual spam and making it impossible to read the output until completion.
25
+
26
+ ### Root Cause
27
+ **File:** `src/orchestrator_magentic.py:247-254`
28
+
29
+ ```python
30
+ elif isinstance(event, MagenticAgentDeltaEvent):
31
+ if event.text:
32
+ return AgentEvent(
33
+ type="streaming",
34
+ message=event.text, # Single token!
35
+ data={"agent_id": event.agent_id},
36
+ iteration=iteration,
37
+ )
38
+ ```
39
+
40
+ Every LLM token emits a `MagenticAgentDeltaEvent`, which creates an `AgentEvent(type="streaming")`.
41
+
42
+ **File:** `src/app.py:170-180`
43
+
44
+ ```python
45
+ async for event in orchestrator.run(message):
46
+ event_md = event.to_markdown()
47
+ response_parts.append(event_md) # Appends EVERY token
48
+
49
+ if event.type == "complete":
50
+ yield event.message
51
+ else:
52
+ yield "\n\n".join(response_parts) # Yields ALL accumulated tokens
53
+ ```
54
+
55
+ For N tokens, this yields N times, each time showing all previous tokens. This is O(NΒ²) string operations and creates massive visual spam.
56
+
57
+ ### Proposed Fix Options
58
+
59
+ **Option A: Buffer streaming tokens (recommended)**
60
+ ```python
61
+ # In app.py - accumulate streaming tokens, yield periodically
62
+ streaming_buffer = ""
63
+ last_yield_time = time.time()
64
+
65
+ async for event in orchestrator.run(message):
66
+ if event.type == "streaming":
67
+ streaming_buffer += event.message
68
+ # Only yield every 500ms or on newline
69
+ if time.time() - last_yield_time > 0.5 or "\n" in event.message:
70
+ yield f"πŸ“‘ {streaming_buffer}"
71
+ last_yield_time = time.time()
72
+ elif event.type == "complete":
73
+ yield event.message
74
+ else:
75
+ # Non-streaming events
76
+ response_parts.append(event.to_markdown())
77
+ yield "\n\n".join(response_parts)
78
+ ```
79
+
80
+ **Option B: Don't yield streaming events at all**
81
+ ```python
82
+ # In app.py - only yield meaningful events
83
+ async for event in orchestrator.run(message):
84
+ if event.type == "streaming":
85
+ continue # Skip token-by-token spam
86
+ # ... rest of logic
87
+ ```
88
+
89
+ **Option C: Fix at orchestrator level**
90
+ Don't emit `AgentEvent` for every delta - buffer in `_process_event`.
91
+
92
+ ---
93
+
94
+ ## Bug 2: API Key Does Not Persist in Textbox
95
+
96
+ ### Symptoms
97
+ 1. User opens the "Mode & API Key" accordion
98
+ 2. User pastes their API key into the password textbox
99
+ 3. User clicks an example OR clicks elsewhere
100
+ 4. The API key textbox is now empty - value lost
101
+
102
+ ### Root Cause
103
+ **File:** `src/app.py:223-237`
104
+
105
+ ```python
106
+ additional_inputs_accordion=additional_inputs_accordion,
107
+ additional_inputs=[
108
+ gr.Radio(...),
109
+ gr.Textbox(
110
+ label="πŸ”‘ API Key (Optional)",
111
+ type="password",
112
+ # No `value` parameter - defaults to empty
113
+ # No state persistence mechanism
114
+ ),
115
+ ],
116
+ ```
117
+
118
+ Gradio's `ChatInterface` with `additional_inputs` has known issues:
119
+ 1. Clicking examples resets additional inputs to defaults
120
+ 2. The accordion state and input values may not persist correctly
121
+ 3. No explicit state management for the API key
122
+
123
+ ### Proposed Fix Options
124
+
125
+ **Option A: Use `gr.State` for persistence**
126
+ ```python
127
+ api_key_state = gr.State("")
128
+
129
+ def research_agent(message, history, mode, api_key, api_key_state):
130
+ # Use api_key_state if api_key is empty
131
+ effective_key = api_key or api_key_state
132
+ ...
133
+ return response, effective_key # Return to update state
134
+ ```
135
+
136
+ **Option B: Use browser localStorage via JavaScript**
137
+ ```python
138
+ demo.load(js="""
139
+ () => {
140
+ const saved = localStorage.getItem('deepboner_api_key');
141
+ if (saved) document.querySelector('input[type=password]').value = saved;
142
+ }
143
+ """)
144
+ ```
145
+
146
+ **Option C: Environment variable only (remove BYOK textbox)**
147
+ Remove the API key input entirely. Require users to set `OPENAI_API_KEY` in HuggingFace Secrets. This is more secure but less user-friendly.
148
+
149
+ **Option D: Use Gradio LoginButton or HuggingFace OAuth**
150
+ Leverage HF's built-in auth and secrets management.
151
+
152
+ ---
153
+
154
+ ## Recommended Priority
155
+
156
+ 1. **Bug 1 (Streaming Spam)**: Fix first - it makes Advanced mode unusable for reading output
157
+ 2. **Bug 2 (Key Persistence)**: Fix second - annoying but users can re-paste
158
+
159
+ ## Testing Plan
160
+
161
+ 1. Run Advanced mode query, verify no token-by-token spam
162
+ 2. Paste API key, click example, verify key persists
163
+ 3. Refresh page, verify key persists (if using localStorage)
164
+ 4. Run `make check` - all tests pass