Open dataset of startup opportunities — discovered from real internet complaints, built by the community, hosted on GitHub.
ProblemAtlas is a community-built dataset of real unmet needs and startup opportunities.
Nine AI agents scrape Reddit, Hacker News, and GitHub Issues, then extract pain points, cluster them into opportunity themes, and score each one. The results go into structured JSON files that live in this repo.
You run the pipeline locally with your own API key. You open a PR. The dataset grows.
No central server. No hosting cost. No single point of failure.
You run: python pipeline/run.py --query "python debugging" --category developer-tools --contributor yourname
You get: data/developer-tools/python-debugging-2025-06.json
You PR: git add . && git commit -m "data: python debugging" && gh pr create
CI checks: schema validated automatically
Merged: GitHub Pages site updates within minutes
data/
├── developer-tools/
│ └── python-debugging-2025-06.json
├── ai/
│ └── agent-observability-2025-06.json
├── productivity/
├── fintech/
├── saas/
├── health/
├── education/
└── schema.json
Each file is one pipeline run. Each opportunity has:
- Pain points with severity scores and real quotes
- Trend signal (rising / stable / declining)
- Competitor map with weaknesses
- Market estimate (TAM, willingness to pay, segments)
- MVP suggestion (features, stack, timeline)
- Bear case (risks and assumptions)
- Final score across 6 dimensions
Browse it: your-org.github.io/problematics
Raw data: data/
Schema: data/schema.json
git clone https://github.com/your-org/problematics
cd problematics
pip install -r pipeline/requirements.txt
export ANTHROPIC_API_KEY=sk-ant-... # or OPENAI_API_KEY
python pipeline/run.py \
--query "your problem space here" \
--category developer-tools \
--contributor yourgithubnameThe pipeline runs locally (~2-3 minutes), validates the output, and tells you exactly what to commit.
export OPENAI_API_KEY=sk-...
python pipeline/run.py --query "..." --provider openai --category ai --contributor yournamemake install
make run QUERY="solo founder productivity" CATEGORY=productivity CONTRIBUTOR=yourname
make validate # validate all data files
make site # preview the site locallyReal sources: Reddit · Hacker News · GitHub Issues
│
Scout → Pain → Cluster → Trend → Competitor → Market → Builder → Bear → Judge
│
Ranked JSON opportunities
| Agent | Role |
|---|---|
| 🔭 Scout | Fetches real posts from HN, Reddit, GitHub Issues |
| ⚡ Pain | Extracts pain points with severity scores and quotes |
| 🔗 Cluster | Groups pain points into opportunity themes |
| 📈 Trend | Scores momentum — rising / stable / declining |
| 🥊 Competitor | Maps existing solutions and their weaknesses |
| 💰 Market | Estimates TAM and willingness to pay |
| 🔨 Builder | Suggests a concrete MVP with stack and timeline |
| 🐻 Bear | Devil's advocate — surfaces risks and assumptions |
| ⚖️ Judge | Final weighted score across 6 dimensions |
| Dimension | What it measures |
|---|---|
| Frequency | How many people hit this problem |
| Severity | Daily impact — time and money lost |
| Market | Revenue potential and addressable market |
| Trend | Is the problem growing? |
| Competition Gap | How open is the market? (higher = better) |
| Feasibility | Can a small team build this in weeks? |
- Free forever. No server, no database, no infrastructure bills.
- Transparent. Every data point has a contributor, a date, and a PR.
- Composable. Raw JSON — use it however you want.
- Compounds. Every merged PR makes it harder to replicate.
- Community-owned. No single company controls it.
The dataset is the moat. The agents are just the shovel.
High impact:
- Run the pipeline on an underexplored problem space
- Add a new data source to the Scout Agent (Product Hunt, App Store, G2 reviews)
- Build a domain-specific scoring agent (healthcare, fintech, education)
Medium impact:
- Improve agent prompts for a category you know well
- Add a new enrichment agent (Geography, Regulatory, Pricing)
- Improve the site UI or add new filter/sort options
See CONTRIBUTING.md for the full guide.
MIT — use it, fork it, build on it, sell products with it. The dataset is public domain. Credit appreciated but not required.