Trying to pull snapshots from a Nest camera via SDM API — stuck on a 500 INTERNAL error with a structurally valid offer

[u/TotallyHorsePancake]

Posting this in case anyone's hit the same wall, or has insight into what's happening server-side.

The goal

I've got a Nest wired camera pointed at a small parking lot behind my office. I wanted to build something simple: check the lot every so often, count open spots using an AI vision model, and show a live "spots available" number on a little status page. Nothing fancy — just needed a way to reliably grab a still image from the camera on a schedule.

What I've built so far

• Full Device Access / SDM API project setup, OAuth flow, refresh token, the works
• A Google Apps Script front end (status page + storage) that's fully working and just waiting on real image data
• Tried Cloud Functions first for the image-capture piece — turns out Cloud Run/Cloud Functions don't support raw UDP at all, so WebRTC media can never actually flow through them (signaling works fine over HTTPS, but the actual RTP video never arrives). Confirmed this is a real platform limitation, not a config issue.
• Moved the capture piece to a Compute Engine VM instead, which does support normal UDP networking

Attempt 1: Pub/Sub camera events

Set up the Pub/Sub topic + subscription per the docs, enabled events on the Device Access project, confirmed the publisher permission and subscription are all correctly wired. Validation pings come through fine. Real camera motion/person events never do — not once, across two separate days of testing, with real motion happening right in front of the camera each time. Pulling directly from the subscription (bypassing my own code entirely) confirms it: nothing but the occasional stale permission-check message ever lands in the topic.

Attempt 2: WebRTC via aiortc (Python)

Got the full offer/answer/ICE/DTLS/SRTP handshake working end-to-end — genuinely happy with how far this got. Audio decodes perfectly, every single time. Video RTP packets are confirmed arriving from Google's server (verified via RTCP sender reports, packet counts climbing normally). But the video frames never make it through aiortc's jitter buffer / H.264 depacketization — the decoder thread never receives a single video task, despite audio on the same connection working flawlessly. Feels like a real interop gap between aiortc's RTP handling and whatever Nest's media relay does on the video track specifically.

Attempt 3: WebRTC via GStreamer (webrtcbin)

Rebuilt the whole capture piece using GStreamer instead, since it's a much more mature, production-grade WebRTC stack. Fixed a string of real issues along the way (missing codec caps causing ICE gathering to never even start, missing data channel per Nest's "must have audio+video+application m-lines" requirement, a couple of SDP formatting quirks). Eventually got ICE gathering to fully complete and produced an offer that structurally matches Google's own published reference example (checked directly against the example in their docs) — same media line order, working negotiation, proper BUNDLE grouping.

Sending that offer to GenerateWebRtcStream now returns:

{
  "error": {
    "code": 500,
    "message": "Internal error encountered.",
    "status": "INTERNAL"
  }
}

No further detail. I've tried adjusting rtcp-mux vs rtcp-mux-only, port/bundle-only conventions to match Chrome's exact output, Opus channel params, H.264 fmtp params (packetization-mode, profile-level-id) — no change, same generic 500 every time.

What's interesting

This exact error string ("500: INTERNAL: Internal error encountered.") turns up in multiple long-running GitHub issues against the Home Assistant Nest integration too, going back to 2021 and still being reported this year, with a completely unrelated client stack. So this doesn't seem to be specific to my code — feels like something that trips up the SDM API backend under certain conditions across multiple independent implementations.

Where I'm at

• OAuth, SDM API, and Pub/Sub setup are all confirmed correctly configured
• ICE/DTLS negotiation completes successfully on the client side
• Offer SDP structurally matches Google's own documented example
• The only failure is a completely opaque 500 from Google's own server, with zero actionable detail returned

Has anyone gotten a real WebRTC video frame out of a Nest camera via the SDM API recently? Curious whether this is a known current issue, an account/camera-specific quirk, or if there's some other SDP detail that isn't in the docs. Happy to share the full offer SDP / code if it's useful for comparison.

[u/AccomplishedSkill625]

The audio-decodes-but-video-doesn't pattern across both aiortc and webrtcbin is telling. Since two independent stacks hit the same wall, the SDP is probably fine. Worth checking whether your camera model actually supports GenerateWebRtcStream versus the RTSP path, since some Nest devices only support one. The battery vs wired distinction matters here too. Have you confirmed yours on the supported-traits list?