2026Live
LLM Issue Triage
- TypeScript
- GitHub Actions
- Node.js
- Gemini API
- Embeddings
- zod
- esbuild
- Vitest
Overview
A GitHub Action that triages new GitHub issues automatically with AI. It labels each issue by type and priority with the repository's own labels, finds duplicate issues with embedding similarity, asks for missing bug details and writes the result as a single comment. It runs on a free Gemini key or any OpenAI-compatible API and is published on GitHub Marketplace.
01 · Problem
Issues in open source projects arrive unsorted: no labels, unclear priority, the same bug reported twice, bug reports without a version or steps. Maintainers have to read and label every issue by hand and keep asking the same questions.
02 · Solution
I built an Action that runs on every new issue. A chat model returns type, priority, labels and missing details in one request following a JSON schema; the answer is validated with zod and filtered against the repository's real labels. Duplicates are found among the last 100 issues with embeddings and cosine similarity. Because anyone can write an issue in a public repository, the issue text is passed as data to resist prompt injection, the comment is built from a template, and mentions and links are neutralised.
03 · Outcome
Published on GitHub Marketplace and triaging its own repository's issues. In the live trial it labelled Turkish and English issues correctly, found the duplicate bug and did not follow the injection attempt. The project has 137 automated tests, and every security guard was verified by breaking it on purpose and watching a test fail.