AI app builders make it easy to ship. We wanted to know what that code looks like from a security point of view, so we pointed the PreflightPR scanner at 456 public GitHub repositories built with tools like Lovable, Bolt, v0 and GPT Engineer, and then checked the scanner's own results by hand.
This page reports aggregate numbers only. We do not name any repository or owner, we did not test any key we found, and we did not contact or open anything on these projects.
The short version
| Finding | Repos | Share |
|---|---|---|
Committed a .env file to git |
89 of 456 | 19.5% |
Committed a private secret we confirmed by hand (service_role key, server secret in .env, client secret, private key, API key) |
14 of 456 | 3.1% |
| Hard-coded a Supabase service_role key | 5 of 456 | 1.1% |
Has a Supabase policy letting anonymous users update or delete rows (using (true)) |
14 of 87 with Supabase migrations | 16.1% |
| Creates a table that never gets Row Level Security enabled (confirmed by hand) | 6 of 87 with Supabase migrations | 6.9% |
| Has at least one dependency with a published security advisory | 394 of 456 | 86.4% |
Finding 1: one in five repos commits a .env file
89 of 456 repos (19.5%) had a .env file tracked in git. The good news is that most of these are harmless: in 63 of the 89, every variable used a public prefix like VITE_ or NEXT_PUBLIC_, which in practice means a Supabase URL and the anon key. Those values ship to the browser anyway.
The rest is where it matters. By hand we confirmed 6 repos whose committed .env held real-looking server-side secrets: things like JWT signing secrets, email app passwords, image and AI provider API keys, a database password, and a Google OAuth client secret. Once a file like that is pushed, deleting it is not enough. It stays in git history and the keys need to be rotated.
If this sounds like your project, our guide on rotating leaked Supabase and OpenAI keys walks through it.
Finding 2: anon keys everywhere (fine), service_role keys in 5 repos (not fine)
118 repos (25.9%) had a Supabase JWT hard-coded in source. We decoded the role inside each token (without using it). Almost all were anon keys, which are designed to be public, so we do not count those as a problem.
But 5 repos (1.1%) hard-coded a service_role key, in frontend files, scripts, or SQL migrations. A service_role key skips Row Level Security completely, so anyone who finds it can read and write the whole database. Across all checks, we confirmed private secrets in 14 repos (3.1%). Treat that as a floor, not a ceiling (see limitations).
Finding 3: Supabase policies that let anyone write
87 repos included Supabase SQL migrations, so we could read their database rules. We looked for policies that apply to anonymous visitors and use using (true) or with check (true), which means "no condition at all."
| Policy type (anonymous or public role) | Repos | Share of 87 |
|---|---|---|
Anyone can read a table (select) |
45 | 51.7% |
| Anyone can insert rows | 33 | 37.9% |
Anyone can update or delete rows, or do everything (all) |
14 | 16.1% |
Public reads and inserts are often intentional (a product catalog, a contact form). Public update and delete rarely are. In the examples we read, these covered things like student and faculty records, abandoned carts and applications. Any visitor with the anon key from the frontend could change or wipe them.
We also confirmed 6 repos (6.9%) that create a table and never enable RLS anywhere in their migrations. One caveat: RLS can also be switched on in the Supabase dashboard, which we cannot see from the repo.
The Supabase RLS checklist shows how to write tighter policies.
Finding 4: dependencies age fast
394 of 456 repos (86.4%) had at least one package with a published advisory in their lockfiles. The median affected repo had 27 such packages. Most are build tools pulled in by Vite and friends (for example vite itself in 302 repos), which matter less in production. A few ship to users:
- 243 repos (53.3%) pinned a version of React Router's core (
@remix-run/router) with known advisories. - 50 repos (11.0%) had an
axiosversion with known advisories. - 48 repos had a Next.js version with known advisories in a lockfile.
AI builders scaffold a project once and rarely bump versions afterward, so the lockfile you got on day one is often the one you launch with.
What we left out, and why
We ran every check in the scanner, then hand-checked samples. Some checks were too noisy to publish as a rate:
- Generic secret patterns. The raw secret scan flagged 179 repos (39.3%), but most hits were anon keys, test fixtures, docs examples and log lines. In a hand-checked sample of 30 generic "API key" hits, only 4 looked like real credentials. We only report secrets we confirmed by hand.
- "Open" Next.js API routes. 38 of the 42 repos with Next.js API routes were flagged for routes with no visible auth check. Many of those are meant to be public (chat demos, webhooks from other services), and 11 of the 38 had middleware that may handle auth. We are not publishing a rate until we tighten this check.
- Stripe webhooks. Only 7 repos used Stripe at all. Of 3 webhook handlers flagged, 2 turned out to be Clerk and Polar webhooks, not Stripe. That left 1 real unverified Stripe webhook, too few to call a trend.
- Tables without RLS. The raw check flagged 14 repos. Hand review confirmed 6. The others were the parser matching words in comments, tables outside the public schema, or RLS enabled with different syntax. Every RLS number above uses the confirmed count.
These misses are already on our list of scanner improvements.
Methodology
- Date: collected and scanned on October 15, 2026.
- Finding repos: GitHub code search for markers that AI builders leave behind:
lovable-taggerinpackage.json,lovable.dev/projects,gpt-engineer,bolt.newor "Built with Bolt" in a README, andv0.devplussupabasein a README. That gave 1,788 unique non-fork repos. - Filtering: we dropped forks, templates, archived repos, and repos under 30 KB or over 80 MB, which left 1,599. We randomly sampled 460, cloned 459 (one failed), and removed 3 that had exactly the same file tree as another repo in the sample. Final N = 456.
- What's in the sample: 186 repos use Lovable's
lovable-tagger, 279 use Vite, 164 use the Supabase JS client, and 48 use Next.js. Most were last updated in 2025 or 2026. - Scanning: each repo was scanned locally and read-only with the same open-source engines PreflightPR uses (gitleaks over full git history, Semgrep rules, OSV-Scanner for dependencies, and our own Supabase SQL checks). We did not use the GitHub App and did not open issues or pull requests.
- Hand review: we checked every service_role and Stripe hit, every committed
.envwith private-looking variables, every repo flagged for missing RLS, and random samples of the other checks.
Limitations
- Public repos only. Private repos may look better or worse. People who publish their code may also differ from people who don't.
- Markers bias the sample. Builders that leave no fingerprint in the code (or people who remove it) are underrepresented, and Lovable is the easiest one to detect.
- Repo is not the live app. Some keys may already be rotated, and some RLS settings may live only in the dashboard. We did not test any key or touch any live database.
- Supabase findings cover the 87 repos with migrations, not all 456.
- Secret counts are a lower bound. We only counted secrets we confirmed by hand, so real exposure is likely somewhat higher.
Check your own app
Most of these problems take minutes to fix once you know they're there. Run a free scan on your repo to see if any of them apply to you, or start with our pre-launch security checklist.