- route.ts: replace @anthropic-ai/sdk with groq-sdk, rewrite chat loop
- voice-tool-schemas.ts: convert from Anthropic format to OpenAI/Groq function tools
- voice-history.ts: extend HistoryMessage type to include tool role, simplify cache stubs
No prompt caching (Groq does not support it), tool calling preserved.
docker --env-file не поддерживает многострочные значения и не парсит
кавычки. Сырой JSON service-account ломается на newline'ах в private_key
поле → docker пытается парсить '-----END PRIVATE KEY-----' как имя
переменной и валится с 'contains whitespaces'.
Решение: base64 GOOGLE_SA_JSON_B64 (одна строка ASCII, никаких кавычек).
Старая GOOGLE_SA_JSON оставлена как fallback. Третий fallback на файл —
для локальной разработки.
Безопасность:
- Rate-limit на /api/voice/chat (20/мин per cookie/IP, env VOICE_RATE_LIMIT).
Защищает от случайных циклов и утечки PIN.
- Усечение user prompt'а до 4000 символов в /api/voice/chat.
- Tool-loop защита от циклов: если LLM дважды просит тот же tool с теми же
args — прерываем (раньше мог уйти в бесконечный цикл при tool error'ах).
Чистка кода:
- lib/debug.ts — vlog/vwarn/verror гейтят браузерные логи за
NEXT_PUBLIC_VOICE_DEBUG=1 (или localStorage 'voice-debug=1').
Серверные console.log оставлены — полезны в Docker logs.
- lib/audio-wav.ts — вынесена дублированная floatToWav из VoiceController.
- Удалены orphan компоненты FocusCard.tsx и CountdownCard.tsx
(не подключены, отвергнуты по UX-фидбеку).
Resilience:
- WakeWordDetector: drop-on-busy в onChunk — на медленных устройствах
(Android, бюджетный CPU) backlog inference больше не копится.
- voice-history fallback на /tmp/voice-history если /data не примонтирован
(локальная разработка / нестандартная конфигурация).
Шаг 2 миграции: убираем зависимость от Python-агента для базового
голосового сценария. Тап на круглую кнопку-микрофон в правом нижнем
углу → MicVAD (Silero v5) ловит речь → автостоп по тишине → /api/voice/stt
→ /api/voice/chat → ответ через SSE и TTS как раньше.
- components/VoiceController.tsx — push-to-talk UI + MicVAD orchestration
- VoiceOverlay теперь слушает window CustomEvent('voice-local'), чтобы
орб моргал ещё до round-trip на сервер (wake/listening мгновенно).
- public/vad/ — silero v5/legacy onnx + ort wasm + audio worklet,
раздаются через baseAssetPath: '/vad/' (не зависит от внешнего CDN,
важно если планшет без интернета или с RU-блоком).
Что осталось от home-voice-assistant: только wake-word. После Шага 3
(onnxruntime-web + перенос openwakeword .onnx) Python-агент уйдёт целиком.
Шаг 1 миграции голосового стека из home-voice-assistant в сам tablet:
- /api/voice/chat — Claude Haiku 4.5 с tool-loop (max 4 раунда), prompt
caching на system + старой истории, история в /data/voice-history/.
Эмитит command/response/error в voice-bus → орб моргает как раньше.
- /api/voice/stt — Groq whisper-large-v3-turbo, multipart или raw audio.
- lib/voice-text.ts — порт clean_for_speech (без pymorphy3, время в
именительном падеже) и strip_fillers + RESET_PATTERNS.
- lib/voice-executors.ts — tool executors через loopback fetch на
существующие /api/voice/tools/* и /api/voice/timer.
- Поддержка ANTHROPIC_PROXY/GROQ_PROXY (fallback на HTTPS_PROXY).
После деплоя нужны GROQ_API_KEY и ANTHROPIC_API_KEY в tablet.env.
Шаги 2 (push-to-talk в браузере) и 3 (wake-word) — отдельно.
Focus/Countdown не лежат по high-density. На Home вернул старый
weather-hero: WeatherAnimation (фон + иконка), feelsLike/humidity/wind,
76px display-цифра (чуть крупнее прежних 64). FocusCard и CountdownCard
файлы оставляем для будущего, но на главной не подключаем. Убрал
сопутствующие state/fetch (tramNext, countdowns, nextEvent).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Голосовой ассистент теперь может создавать, изменять и удалять события
в календарях Даниила и Светы.
- POST /api/voice/tools/events — create (title, date, start_time, end_time,
all_day, owner). Маппит owner (daniil/sveta) в calendar_id и проксирует
в /api/calendar POST.
- PUT — update (event_id, owner, fields). Передаёт только изменённые
поля + нужный calendarId.
- DELETE ?event_id=X&owner=Y — удаление.
- GET — теперь возвращает id события и owner (daniil/sveta), чтобы
скрипт мог их передать в update/delete.
- range=month поддержан с year/month query params.
Все три метода под bearer auth (VOICE_API_KEY), как остальные voice tools.
Loopback к /api/calendar идёт через internalHeaders() x-voice-internal.
UI:
- Replace Notes column on Home bento with TimerHomeWidget. Shows all
active timers as stacked cards with big 30px countdowns, per-timer
+1/-1 minute buttons and cancel. Colors: indigo default, amber in
last 10s, red when expired. Empty state suggests voice command.
- Existing chip TimerWidget (bottom-right) kept for ambient view on
other tabs — redundant on Home, but harmless.
API:
- /api/voice/timer accepts cookie OR bearer (browser widget cancel
works with user's auth_token cookie; Python script uses bearer).
- New action 'adjust' — shifts endsAt by delta_seconds. Clamps so
endsAt never goes into the past.
- Cancel now supports {label} in addition to {id} (fuzzy substring
match, most-recently-started wins). Emits timer_cancel with id+label
so clients can refresh.
- findByLabel / adjustTimer helpers in lib/timers.ts.
Tool endpoints (events, notes, transport, weather) call other /api/*
routes via loopback (http://localhost:3000). Those routes are
middleware-protected — cookie-less loopbacks were getting 401, which
surfaced to the voice agent as get_today_events → tool_http_502.
Add internal header bypass: middleware lets the request through when
x-voice-internal matches VOICE_API_KEY. Only our own tool endpoints
use this header, from inside the same container, so the blast radius
is limited to loopback traffic.
- middleware.ts: check x-voice-internal before cookie
- lib/voice-tools.ts: internalHeaders() helper
- app/api/voice/tools/{weather,transport,events,notes}: use it
Adds the infrastructure for Claude tool use + visual timer.
Tablet API surface (all bearer-authed with VOICE_API_KEY, middleware bypassed):
- /api/voice/tools/weather — current + short forecast via Open-Meteo
- /api/voice/tools/transport — tram arrivals by direction / route filter
- /api/voice/tools/events — Google Calendar today/week
- /api/voice/tools/notes — notes + shopping lists
- /api/voice/timer — start (with seconds+label), cancel; GET list (cookie ok)
Active timers persisted at /data/tablet-timers.json
UI:
- VoiceOverlay stripped to minimal Siri look: no agent emoji/name, just the
pulsing orb (3-layer radial gradient, independent breath animations),
subtle status label on wake only, transcription/response text centered.
Agents distinguished by orb color (Cosmo indigo/violet, Люся pink).
- TimerWidget: bottom-right chip stack with countdown, progress bar, turns
amber in last 10s. On expiry, fires fullscreen alarm overlay with beep
(WebAudio osc) + Остановить button.
Other:
- lib/timers.ts — persistent timer store in /data
- lib/voice-tools.ts — shared bearer-auth helper
- middleware — bypass list now covers /api/voice/tools/* and /api/voice/timer
ElevenLabs Cloudflare returns 302 to a region-restricted help page
when requested from a Russian IP. Tablet host (.60) is in RU, so the
Stage 2 call was failing with 502 upstream.
Fix: use https-proxy-agent when ELEVENLABS_PROXY (or generic HTTPS_PROXY
/ HTTP_PROXY) env var is set. Tinyproxy on .103 (non-RU egress host)
acts as the tunnel.
- package.json: add https-proxy-agent ^7.0.6
- app/api/voice/tts: switch from global fetch to node:https with
explicit Agent (either direct or HttpsProxyAgent). Still streams
MP3 back via Readable.toWeb so Next.js Response pipes it to the
browser as audio arrives.
Operational: set ELEVENLABS_PROXY=http://192.168.31.103:8888 in
tablet.env after bringing tinyproxy up on .103.
Stage 2 of voice integration — centralizes TTS on the tablet so the
Python satellite no longer needs ElevenLabs credentials or mpv.
- app/api/voice/tts — POST {text, agent}, proxies to ElevenLabs
streaming endpoint with flash_v2_5 default, returns audio/mpeg.
Per-agent voice id via COSMO_TTS_VOICE / LUSYA_TTS_VOICE env.
- VoiceOverlay — on response/error events fetches TTS and plays via
HTMLAudioElement; on wake event stops playback (barge-in). Dismiss
timer extended by text length so long responses do not cut off.
- Autoplay caveat: browser may block first playback until user taps
anywhere on the page (FKB: enable Force Autoplay to bypass).
Adds the tablet side of voice assistant integration. External Python
script (openWakeWord + Groq STT + OpenClaw) will POST state transitions
to /api/voice/event with a bearer token, and the tablet shows a
fullscreen overlay with Siri-style animated blob + current agent +
recognized text / response text.
- lib/voice-bus.ts — in-process EventEmitter singleton, preserved
across hot reloads via globalThis
- app/api/voice/event — POST, bearer-auth via VOICE_API_KEY env,
validates event kind, broadcasts on voiceBus
- app/api/voice/stream — GET, SSE endpoint, per-connection listener
with 15s keep-alive ping and abort-signal cleanup
- components/VoiceOverlay — full-screen overlay, 3-layer pulsing
Siri blob, per-agent palette (cosmo indigo/violet, lusya pink/rose),
auto-dismiss timeouts (wake=20s safety, response=6s, error=4s),
auto-reconnect on SSE drop
- middleware bypasses /api/voice/event so the script does not need
a user auth cookie
- VoiceOverlay mounted in HomePageInner outside tab routing so it
appears on every view
Google Calendar API rejects all-day events where start.date == end.date
(end is exclusive). POST/PUT were sending the same date for both,
producing Invalid start time when toggling Весь день on edit.
- Added nextDayISO helper (UTC-safe +1d arithmetic)
- all-day: start = { date }, end = { date: nextDay, dateTime: null }
- timed: also explicitly nulls start.date/end.date so patching a
timed-only event over a previously all-day one doesnt leave stale
date fields that also trigger Invalid start time
- AnimatePresence mode=wait keeps the DOM empty while the outgoing tab
finishes its exit transition (200ms) before mounting the incoming
tab. On touch devices this shows as a brief black frame — reported
as экран остается черным.
- Switch to mode=sync: outgoing fades out while incoming fades in,
no gap. initial=false suppresses the enter animation on first render.
- Drop the y: 12 → -8 slide (cheap jank on hydration), keep just
opacity. Duration 200ms → 150ms for snappier feel.
- WeatherDayModal now accepts the full forecast array and an onChange
callback; supports horizontal drag (framer-motion) plus prev/next
chevrons and a dot-indicator. Drag > 60px switches day; style uses
semantic tokens (shadow-xl, surface-1).
- NotesTab list items wrap each note in a motion.button with drag=x,
constrained to -80px. Below it a gradient+trash reveal layer. Drag
past 60px opens the existing confirmDelete modal.
- HomePageInner adds a night-shift overlay (fixed, mixBlendMode multiply,
rgba(255,120,40,0.12)) active 22:00-06:00, auto-checked each minute,
fades in/out over 800ms. No user toggle yet — fully automatic.
CSS grid items default min-width to min-content; the tram widget 3-col
subgrid plus its баbadges and long затем text forced its cell wider
than 1.1fr, collapsing the outer layout on tablet. Fixes:
- outer bento row → gridTemplateColumns: minmax(0, 1fr) minmax(0, 1.1fr)
- events+notes row same treatment
- TransportWidget inner subgrids: 58px → 52px badge, 1fr → minmax(0, 1fr)
- Cell: minWidth: 0, overflow: hidden, затем text trimmed with ellipsis
and short м suffix (5м instead of 5 мин)
- big number 32→28px, badge 22→20px to fit in denser columns
The big Доброе утро line on Home ate one row for marginal value.
Moved it into TopBar as a center column via 3-col grid (1fr auto 1fr),
so it appears on all tabs and leaves the Home body denser.
- Removes the date label from the greeting row (user request — тoo dense
on tablet and redundant with TopBar clock).
- Scrolling: drops overflowY: auto on the inner Events/Notes cards and
removes flex: 1 / minHeight: 0 from their grid. On iPad-class touch
nested scroll containers fought the root scroll; now only the root
scrolls, which the browser handles natively.
- TopBar: replaces hardcoded rgba(255,255,255,X) on sensor icons, chip
background, chip border and header bottom-border with semantic tokens
(--text-tertiary, --surface-2, --border-subtle, --hairline) so the
thermometer/humidity/wind icons are visible in light theme.
- introduces semantic CSS tokens (--surface-1/2/3, --border-subtle/strong,
--hairline, --shadow-sm/md/lg/xl) with distinct dark and light values;
fixes broken light theme caused by hardcoded rgba(255,255,255,X)
- drops glassmorphism on cards — solid var(--surface-1) with 1px border
and layered shadows; glass kept only for aurora page background
- introduces .card/.card-raised/.card-hero utility classes
- Home page restructured into a bento grid:
* greeting row with inline day/date
* hero weather (64px number, large icon, ощущается/влажность/ветер)
next to the tram widget (1fr 1.1fr)
* forecast as a single hairline-separated band (no per-day cards)
* events+notes in a 2-column grid; events card combines today and
tomorrow with a divider; notes card styled via surface tokens
- TransportWidget repainted to use tokens, larger numbers (32px for the
next arrival), imminent highlight uses color-mix against surface-2
Weather hint (оденьтесь потеплее / не забудьте зонт) was pushing the
home screen past one viewport on the tablet — removed the block and its
helper fn. New tram color palette per user preference.
Next.js could not resolve undici as a top-level import even though it
ships internally. Drop that path and call the ORGP endpoint via the
built-in node:https with a per-request Agent(rejectUnauthorized: false).
Adds runtime = nodejs on the route so Node APIs are guaranteed.
ORGP SPb uses a TLS chain Node rejects by default (curl works with -k
but Node fetch doesnt). Use an undici Agent with rejectUnauthorized
false for this one hop. Also drop the conflicting next.revalidate: 0
option — cache: no-store already covers it.
Adds a live transit widget on the home screen showing upcoming trams
at both directions of the stop: toward Новочеркасская (stopID 16226)
and toward пр. Большевиков (stopID 16354).
- /api/transport proxies the СПб ORGP endpoint /stop/{id}/arriving
(DataTables POST format, JSON response with route number + minutes).
No auth required, free.
- TransportWidget renders two glassmorphism cards with route badges,
minutes-to-arrival, wheelchair indicator; imminent (<=2 min) arrivals
get a colored highlight. Filters to trams 23/27/39; refreshes every 30s.
- Route colors: 23 blue, 27 amber, 39 purple.
- calendar API: today/week ranges use Moscow time (UTC+3) instead of UTC — previously today events did not appear until 03:00 MSK
- settings tab: add -webkit-overflow-scrolling: touch + touchAction pan-y for tablet scroll
- NotesTab: add date picker (pinDate) in editor header + date badge in list
- home: pinnedNotes now filters by pinDate (today or future), falls back to latest
- notes/auth: storage moved from /tmp to /data (falls back to /tmp if /data missing)
- deploy workflow: mount /opt/digital-home/smart-home-tablet-data:/data so notes survive redeploys
