add FAILED-APPROACHES.md — log of everything that didn't work and why

FAILED-APPROACHES.md

@@ -0,0 +1,182 @@
+# PluginBridge — Failed Approaches & Architecture Decisions Log
+> Everything we tried that didn't work, why it failed, and what we did instead.
+> Use this to avoid repeating mistakes or to revisit with better solutions later.
+
+---
+
+## 1. Out-of-Process GUI via Floating Window (Sprint 2, v0.2)
+
+**What we tried:**
+Host the plugin in a separate Helper process. Show the plugin's GUI as a floating window from the Helper process, overlaying Ableton.
+
+**What worked:**
+- Audio routing via shared memory ✅
+- Crash safety (Helper crashes, Ableton survives) ✅
+- GUI showed correctly when Ableton was in WINDOWED mode ✅
+
+**What broke:**
+When Ableton was in macOS FULLSCREEN, the plugin GUI opened in a separate Space (different virtual desktop), not over Ableton.
+
+**Fixes attempted (all failed in fullscreen):**
+
+| Attempt | Code | Result |
+|---|---|---|
+| `NSApplicationActivationPolicyAccessory` | Set in `main.cpp initialise()` | No Dock icon ✅ but broke fullscreen behavior ❌ |
+| `setLevel: 3` (NSFloatingWindowLevel) | In `showAsFloatingPanel()` | Not high enough — fullscreen apps are above level 3 ❌ |
+| `setLevel: 25` (NSStatusWindowLevel) | In `showAsFloatingPanel()` | Still opens in separate Space ❌ |
+| `setCollectionBehavior: canJoinAllSpaces + fullScreenAuxiliary` (257) | Before `setVisible(true)` | Still separate Space ❌ |
+| `setCollectionBehavior` + `stationary` + `ignoresCycle` | Full flags (1<<0 \| 1<<4 \| 1<<6 \| 1<<8) | Still separate Space ❌ |
+| `orderFrontRegardless` | After setting level | Window appears but wrong Space ❌ |
+| `setHidesOnDeactivate: 0` | Combined with above | No effect on Space problem ❌ |
+| `setAlwaysOnTop(true)` (JUCE method) | Reverted to original approach | Was original working code but didn't fix fullscreen either ❌ |
+
+**Root cause:**
+macOS fullscreen creates a dedicated compositor Space. Apple prevents other processes from injecting windows into a fullscreen app's Space. No combination of window level, collection behavior, or activation policy reliably works for cross-process window display in fullscreen. This is a macOS platform limitation, not a code bug.
+
+**What we did instead:**
+Switched to in-process hosting (v0.3). Plugin GUI is embedded inside PluginBridge's own editor. Ableton manages the window. No separate process window needed.
+
+---
+
+## 2. X Button (Close) Not Working on Helper Window (Sprint 2, v0.2)
+
+**What happened:**
+The Helper's floating window had minimize and close (X) buttons visible, but clicking X did nothing.
+
+**Root cause:**
+When the Helper ran as `NSApplicationActivationPolicyAccessory`, macOS doesn't deliver close-button clicks to accessory-policy processes the same way. The native title bar's close button was visible but non-functional.
+
+**Fix by Claude Code (commit 0632681):**
+Removed `makeAccessoryProcess()`, added `showAsFloatingPanel()` with objc_msgSend level tricks. This fixed the X button but broke the fullscreen behavior (see #1 above).
+
+**What we did instead:**
+In v0.3 (in-process hosting), there's no separate window at all. The plugin editor is a child component inside PluginBridge's editor. No close button needed — the DAW manages the window lifecycle.
+
+---
+
+## 3. Plugin Crashes Crashing Ableton (v0.3 initial)
+
+**What happened:**
+After moving to in-process hosting, loading Kickstart 2 crashed Ableton (because the plugin crash happens inside Ableton's process).
+
+**Root cause:**
+`loadPlugin()` called `createPluginInstance()` directly without first testing if the plugin was safe. Some plugins (Kickstart 2, Ozone 12, Gullfoss) call `abort()` or show `NSAlert` during loading, which kills the host process.
+
+**Fix attempted that was missing:**
+The design called for Helper-based test loading BEFORE in-process load, but it was left as a `// TODO` stub.
+
+**What we did:**
+Implemented `testPluginSafety()` — spawns `PluginBridgeHelper --test "/path"` before loading. If Helper survives (exit 0) → safe to load in-process. If Helper crashes → add to blocklist → show "Not compatible" → Ableton stays alive.
+
+---
+
+## 4. In-Process Hosting = No Crash Safety for Audio Processing
+
+**Current limitation (v0.3):**
+If a plugin passes the test-load but crashes during `processBlock()` later (rare but possible), Ableton will still crash. The Helper test only validates loading, not runtime stability.
+
+**Potential future solutions:**
+- Wrap `processBlock()` in a signal handler (fragile, not recommended)
+- Use `setjmp`/`longjmp` around processBlock (undefined behavior with C++)
+- Move back to out-of-process audio routing BUT keep GUI in-process (hybrid approach)
+- Accept the risk — plugins that crash during audio processing are very rare
+
+**Status:** Accepted risk. Most crashes happen during loading, not during audio processing.
+
+---
+
+## 5. Plugin Scanning via PluginDirectoryScanner (Early research, never shipped)
+
+**What was planned:**
+Use JUCE's `PluginDirectoryScanner` to scan all installed plugins on first run.
+
+**Why it was rejected:**
+Some plugins execute code during scanning (show NSAlert dialogs, call abort()). This would crash the scanner — or worse, crash Ableton if scanning happened in-process.
+
+**What we did instead:**
+Read `.vst3` and `.component` filenames from disk (no code execution). Only actually load a plugin when the user selects it (after Helper test).
+
+---
+
+## 6. `addDefaultFormats()` — Deleted in JUCE 8
+
+**What happened:**
+Early code used `formatManager.addDefaultFormats()` which doesn't exist in JUCE 8.
+
+**Fix:**
+Explicitly register formats:
+```cpp
+formatManager.addFormat(new juce::VST3PluginFormat());
+formatManager.addFormat(new juce::AudioUnitPluginFormat());
+```
+
+---
+
+## 7. `MSG_NOSIGNAL` — Doesn't Exist on macOS
+
+**What happened:**
+Socket code used `MSG_NOSIGNAL` flag which is Linux-only.
+
+**Fix:**
+```cpp
+signal(SIGPIPE, SIG_IGN);  // Global
+int nosigpipe = 1;
+setsockopt(fd, SOL_SOCKET, SO_NOSIGPIPE, &nosigpipe, sizeof(nosigpipe));  // Per-socket
+```
+
+---
+
+## 8. `juce::Thread::setCurrentThreadPriority()` — Removed in JUCE 8
+
+**What happened:**
+Audio thread priority setting via JUCE API no longer exists.
+
+**Fix:**
+Use native pthread:
+```cpp
+struct sched_param param;
+param.sched_priority = sched_get_priority_max(SCHED_FIFO);
+pthread_setschedparam(pthread_self(), SCHED_FIFO, &param);
+```
+
+---
+
+## 9. `create_audio_track()` Serialization Error (Ableton LOM, not PluginBridge)
+
+**Context:** From the broader AI Music Production system, not PluginBridge-specific.
+
+**What happened:**
+`song.create_audio_track()` in Ableton's LOM returns a "not JSON serializable" error.
+
+**Reality:**
+The track IS created. The error is in the return value serialization, not the action.
+
+**Workaround:**
+Always check `len(song.tracks)` to confirm track was created, ignore the error message.
+
+---
+
+## Architecture Evolution Summary
+
+```
+v0.1 (Sprint 1): Plugin shell + MCP server only (no hosting)
+v0.2 (Sprint 2): Out-of-process hosting via shared memory
+                  ✅ Crash-safe audio
+                  ❌ GUI in separate window → fullscreen broken
+v0.3 (Sprint 3): In-process hosting (like SnappySnap)
+                  ✅ GUI embedded in Ableton's window
+                  ✅ Helper tests plugin safety before loading
+                  ❌ No crash safety during audio processBlock (accepted risk)
+```
+
+---
+
+## Revisit Candidates (Future)
+
+| Problem | Possible Future Solution | Difficulty |
+|---|---|---|
+| Fullscreen floating window from separate process | Wait for Apple to provide a supported API, or use private `CGSOrderWindow` API | Hard, fragile |
+| Runtime crash safety (processBlock) | Hybrid: audio in Helper, GUI in-process. Two instances synced via IPC. | Very hard |
+| Blocklist is manual (per-machine) | Share blocklist via cloud / community database | Medium |
+| Test-load adds 2-3 second delay first time | Cache results aggressively, pre-test in background on startup | Medium |
+| Some plugins crash in createEditor() not createInstance() | Test GUI creation in Helper too (spawn with --test-gui flag) | Medium |