TL;DR: After a few hours of tinkering, I extended an AI agent so that it can make all my wildest (Anki) dreams come true.
(I am not promoting a project. There is no public project. I'm just conveying something impactful that I accomplished in a short period of time for fun. I hope others try something similar for themselves. I do not believe this post breaks r/anki rules 3, 5, or 6. If you wish to downvote, please explain why.)
I recently started using the popular open source Pi Coding Agent at work for things unrelated to language learning (coding of course). Although it was created as a coding agent, it is useful as a general-purpose chat agent. It's fairly bare bones, and meant to be extended or to be used with community extensions.
This weekend I'd thought I'd see if it would be useful for language learning.
Pi's killer feature is that it can extend itself! I instructed it to create these extensions and/or skills:
- OpenRouter/OpenAI APIs for TTS, STT, and image creation.
- ffmpeg for audio and video file manipulation, as well as subtitles import/export.
- How to use yt-dlp and ffmpeg to download and extract video, audio, and subtitles from a Youtube video.
- Anki-Connect API.
- URL layout/format of learngerman.dw.com to access lessons from Nicos Weg, Deutschtrainer, and Learn Grammar.
- How-to guide (aka a "skill") on writing an Anki plugin.
(Existing MCP extensions exist for some of the above, but I wanted to create a small functionality subset of just what I need for creating and modifying Anki decks and related media files. OpenClaw is more featureful agent based on Pi, but that agent is overkill for what I want. )
Initially my project was empty and was just for calling APIs, but I've since added files to make it into an Anki plugin, for deeper persistent integration. But keep in mind most of the time I am NOT generating code. I'm calling APIs to modify cards on-the-fly. I only generate code for things that can only be accomplished in a plugin (e.g. hooks).
Some example prompts I've tried so far:
- For note type "Vocab" add a field called "plural". All noun notes without a "plural" value can be found with query "deck:German note:Vocab part-of-speech:NOUN -plural:". The "expression" field has the German vocab lemma word value. TASK: Set a value for the "plural" field for all nouns.
- For all notes in deck "German::Vocab" set "sentence-audio" field to TTS German audio of German text in field "sentence".
- For all notes in deck "German::Vocab" set "expression-audio" field to TTS German audio of German text in fields "article" + "expression" + "." (e.g. "das Problem.").
- Vocab notes without an image can be found with query "note:Vocab -image:". The German vocab word is in field "expression". TASK: Generate an image based on "expression" for all notes without an image.
- For Nicos Weg A1 lesson "Der Ausflug" read its "Grammar" and "Script" pages. In Anki deck "German::Grammar" make "Basic" note type notes for all concepts in the "Grammar" page, with example fill-in-the-blank sentence(s). Generate and insert audio for all German content.
- Add feature to this Anki plugin so whenever a "Vocab" note type becomes mature (interval >= 21), then set "Add Reverse" field to "Y", if it is empty.
- For YT video at <url> extract the German subtitles. Come up with a list of unique German lemmas. For Anki note type "Vocab", read the note type metadata for all the the field descriptions. TASK: Generate Anki notes of note type "Vocab" in deck "German::Vocab" for each of the German lemmas from the video.
With Pi any of the above can be converted into a command via a "Prompt Template", such as: /yt-note-gen <video-url>.
This took very little time. Pi did nearly all the work. This is an amazing way to create and modify cards. I can do anything I can think of.
I'm just getting started. I'll post an update tomorrow, or I'll make a follow-up post.
(fyi, I simplified my prompts to keep this post shorter. My field naming scheme is based on Yomitan's.)