I Built an AI Reviewer That Refuses to Tell Me If My Idea Will Work

This blog has been quiet for ten years — the last post here is a Windows 10 dark-theme registry hack. I'm putting it back to work. Going forward this is where I'll write about the AI systems and tools I'm building, with more room than a social post gives me. First one: a small protocol that talked me out of a bad idea before I wasted months on it.


The wrong question

The first startup idea I tested with my own AI review protocol didn't survive the review.

That wasn't a failure of the protocol. It was the point.

I built the protocol after realizing something uncomfortable: asking an LLM whether an idea is good is almost the wrong question. Modern language models are very good at constructing coherent arguments, but they can't determine whether a business will succeed — that depends on evidence the model simply doesn't have. Ask for a brutal review and you usually get the tone of criticism wrapped around the same underlying agreement. Better prompting doesn't fix that. Restricting what the model is allowed to judge does.

So instead of asking for opinions, I started restricting what the model was allowed to judge. I call the result Crucible.

Three roles, no overlap


Crucible splits judgment three ways, and the roles never overlap:

  • The model checks the argument — arithmetic on my own numbers, contradictions between my own claims, structural impossibility. Things checkable tonight, from what I've already written.
  • Reality tests the assumption — through one experiment I actually run, with a pass/fail threshold I set before running it.
  • I own the bet — I write down what I expect to happen, and what I'll do if I'm wrong, before I see the result.

The model never predicts outcomes. "Will this work?" belongs to reality, not to a language model — and no amount of clever prompting changes that.

The assumption ledger

Every load-bearing claim in an idea gets exactly one tag:

  • KNOW — evidence in hand. I can name it.
  • BELIEVE — reasons, but no evidence.
  • HOPE — neither. Every question I can't answer lands here.

Nothing gets promoted by argument. A better pitch doesn't move a tag — only evidence does. Alongside that, every factual claim gets tiered by where it came from: VERIFIED against a live source I checked this session, RECALL from the model's training data (confirmed before anyone acts on it), or REASONED out from something I already stated. Mixing these up — treating a recalled fact as a verified one — is exactly how founders end up building on numbers that were never checked.

One experiment, written down first

Every session ends the same way: one experiment, aimed at the riskiest assumption in the stack. I define what counts as pass or fail before I run it, cap what it costs, and set a deadline. The experiment measures what people actually do, not what they say they'd do.

Then two lines go on the record, before I see any result: what I predict will happen, and what I'll do if I'm wrong.

A ledger keeps all of this across sessions — including whether my predictions turned out to be right. That's the part I actually care about. Not a verdict on any single idea, but a calibration record of my own judgment over time.

What it actually killed

The first idea I ran through Crucible was "Git for Knowledge" — an engineering-memory platform I'd spent a week convincing myself was worth building. It didn't survive. The protocol exposed several independent problems: weak differentiation from tools that already existed, no clearly identifiable buyer, and platform risk on the data it depended on. I recorded the result and moved on.

The discipline from that session, and from the ones since, fed into a different project: Engineering Historian, a knowledge system for AI coding agents that I've been building and using in production for the past few months. Different tool, same underlying principle — provenance on every claim, and conditions for killing my own work decided in advance, not after I already like the answer.

Try it

Crucible is open source, MIT licensed, built on the Agent Skills open standard — a portable format any capable AI agent can load. It's one of a small, growing set of agent skills I'm building and publishing in the open.

Repo: github.com/AdityaVasireddy/agent-skills

If you run an idea through it and it tells you something you didn't want to hear, I'd genuinely like to know.

Location: United States

Comments

Post a Comment

Popular posts from this blog

SQL Server 2008 MERGE statement

One Statement for INSERT, UPDATE, DELETE In earlier versions of SQL Server we had to write separate statements for Insert,update and Delete based on certain conditions but now, using MERGE statement we can include the logic of such data modifications in one statement that even checks when the data is matched then just update it and when unmatched then insert it. One of the most important advantage of MERGE statement is all the data is read and processed only once. In previous versions three different statement has to be written to process three different activity (INSERT, UPDATE or DELETE), however using MERGE statement all update activity can be done in one pass of database table. This is quite an improvement in performance of database query. Syntax of MERGE statement is as following: MERGE [ TOP ( expression ) [ PERCENT ] ] [ INTO ] target_table [ WITH ( ) ] [ [ AS ] table_alias] USING ON [ WHEN MATCHED [ AND ] THEN ] [ WHEN NOT MATCHED [ BY TARGET...
Read more

Google Authentication with ASP.Net

Image
Code download available at the bottom of the post In this post i would like to share my experience with  Google Authentication with Asp.net membership (Forms Authentication) High level google authentication work flow with ASP.NET membership 1.              Your application requests access and gets an unauthorized request token from Google's authorization server. 2.              Google asks the user to grant you access to the required data. 3.              Your application gets an authorized request token from the authorization server. 4.              You exchange the authorized request token for an access token. 5.              You use the access token to requ...
Read more