HOW OPENSOYCE SCORES PROJECTS
"OpenSoyce is not a security audit. It is a signal layer for builders."
Score distributions were recalibrated to spread out the 7–9 cluster and to actually use the 0–2 floor. Cosmetic metadata flags (license presence, GitHub description, etc.) award fewer points; real content signals (README depth, advisory cleanliness, SECURITY.md, star count past 100k) carry more. Existing embedded badges will render different numbers on next cache refresh — this is intentional, not a regression.
THE SOYCE SCORE
The Soyce Score is a weighted composite computed from thirteen GitHub repository signals: last commit date, star count, contributor count, fork count, license, open issue count, description / topics / homepage presence, 30-day commit volume, README content, SECURITY.md presence, release recency, recent-issue triage rate, and published security advisories (CVEs the maintainers disclosed in this repo's own code).
| Pillar | Weight | Key Signals & Criteria Covered |
|---|---|---|
| MAINTENANCE | 30% | Commit recency, latest tagged release within 1-2 years, issue triage rate (recent comments & closed issues). |
| COMMUNITY | 25% | Log-scaled star count, contributor count, fork milestones. |
| SECURITY | 20% | CVE advisories (weighted), license permissiveness (MIT etc), issues-per-star ratio, and SECURITY.md presence. |
| DOCUMENTATION | 15% | Repo topics & homepage, README length, installation guides, heading structure, and code example codeblocks. |
| ACTIVITY | 10% | Last 30 days commit volume (frequency & velocity). |
SCORE RANGES
| Range | Verdict | Interpretation Guidelines |
|---|---|---|
| 85+ | USE READY | Production grade. Real CVE hygiene, active maintenance, and strong documentation. Safe to adopt. |
| 70–84 | SOLID | Mature core infrastructure with minor gaps. Trustworthy base for internal customization. |
| 60–69 | STABLE | Healthy maintained library in stable mode without daily commits. Ideal for light integrations. |
| 40–59 | WATCHLIST | Works today but signals are mixed. Verify per-pillar breakdowns before adoption. |
| 25–39 | RISKY | Concerns in multiple pillars. Maintenance debt, licensing gaps, or unaddressed CVEs. |
| < 25 | STALE | Effectively abandoned or dormant. No commits, releases, or triage for a long period. |
GRAVEYARD RULES
A project doesn't die when people stop starring it. It dies when the pulse stops.
A PROJECT ENTERS THE GRAVEYARD WHEN:
- • LAST COMMIT > 18 MONTHS AGO
- • ISSUES HAVE GONE UNRESPONSIVE FOR > 1 QUARTER
- • NO RECENT TAGGED RELEASES OR NPM PUBLISHES
- • EXPLICIT DEPRECATION BY MAINTAINERS
NOTE: GRAVEYARD ≠ WORTHLESS. SOME GRAVEYARD PROJECTS HAVE EXCELLENT FORKABLE INFRASTRUCTURE.
WHAT WE DON'T CLAIM
THE SIGNAL VOCABULARY
Every label has a definition. These are frozen.
USE READY
Safe to adopt — strong across all pillars
"Production-grade signals. Real CVE hygiene, active maintenance, strong docs. Ready for serious adoption evaluation."
SOLID
Healthy and dependable — solid to build on
"Mature core infrastructure. Best for teams who want to build on top of it."
STABLE
Mature, lower-velocity, still maintained
"Healthy maintained library — may be in stable mode (releases + triage) without daily commits. Don't confuse with risky."
WATCHLIST
Real issues; verify before adoption
"Works today, but signals are mixed. Check the per-pillar breakdown before adopting."
RISKY
Multiple bands flag concerns
"Real concerns in multiple pillars. Maintenance debt, licensing gap, or unaddressed advisories."
STALE
Abandoned or dormant
"Effectively abandoned. No recent commits, no releases, no triage."
GRAVEYARD
"Shaped the ecosystem. Now a museum piece. Some have forkable infrastructure."
Editorial tiers (e.g. HIGH MOMENTUM for hand-curated rising stars) are not shown in the public score card. A real momentum heuristic is on the roadmap — until then, the algorithm only earns the bands above.
KNOWN LIMITATIONS
What this scanner does not see — named out loud.
Honest disclosure, not marketing. The AI tooling ecosystem moves faster than any static scoring model, and several failure modes are specific to it. Below is what we know we don't yet cover, and where a mitigation exists, the commit that ships it.
AI ecosystem velocity
Projects like LangChain and HuggingFace move weekly. Soyce scores reflect signals as of the most recent scan; a single bad release week can shift a score and a single good triage day can shift it back...
Single-maintainer projects with massive adoption
@huggingface/transformers.js has 18M monthly downloads and one primary maintainer. The current scoring model doesn't separately penalize bus-factor risk on these projects beyond the generic contributo...
Federation of @scope/pkg namespaces
@huggingface/* on npm is not the same as huggingface/* on GitHub. The resolver maps via the npm `repository` field but doesn't verify that the GitHub org name matches the npm scope. We cross-check pac...
Typo-squat homoglyph detection (informational only)
Each scanned package name is reduced to a Unicode-TR39 confusables skeleton (Cyrillic/Greek/fullwidth lookalikes of common ASCII letters, plus the digit/letter confusables 0→o, 1→l, etc.) and compared...
Cross-ecosystem bridge attacks (curated map v0)
When an npm package transitively installs a Python package via Python bindings (or vice-versa), a single-ecosystem scan misses the OTHER half — PyPI dependency confusion against `langchain` can't be c...
huggingface_hub and model weights (model-file scanning)
OpenSoyce scores the maintainer + code health of repos. It does NOT analyze actual model weight files (pickle opcode scanning of .pt / .bin / .pkl payloads downloaded by `from_pretrained()` — differen...
Model-weight loader posture (informational only)
AI projects load model weights via `huggingface_hub.from_pretrained()` and `torch.load()`, which historically default to pickle format. Pickle is a code-execution format — loading untrusted weights ca...
PyPI coverage edge cases
For poetry.lock without a companion pyproject.toml, we cannot reliably tell direct vs transitive dependencies. The Risk Profile surfaces this honestly via the `directUnknown` caveat in the Tree Comple...
Verdict bands cap on hidden vulns
If a repo's composite score is high (≥7.0) but the repo has 3+ open HIGH/CRITICAL advisories on its own code, the verdict band is capped at WATCHLIST. This is intentional honesty (P0-AI-1) but it mean...
Single-maintainer band-cap (AI signals v0.1)
When >85% of recent commits come from one contributor AND there are ≤2 non-bot contributors AND the last commit was >30 days ago, the verdict band caps from USE READY to SOLID. Composite score is unch...
Signed reports (Ed25519)
Reports emitted by --out, --json, and --sarif flags are cryptographically signed with OpenSoyce's Ed25519 signing key. The signature lives inside each report (top-level `signature` field for JSON; `ru...
Fork-velocity-of-namesake (migration detection)
When a queried repo's verdict is low (WATCHLIST / RISKY / STALE) AND it's dormant (>180 days since last commit) AND a top-3 fork has recent activity (≤90 days) plus at least 10% of the original's star...
Postinstall script detection (informational only)
npm preinstall / install / postinstall hooks run arbitrary code on `npm install` — the attack vector behind event-stream, ua-parser-js, colors.js, and faker.js. Inventory + vuln rows surface a ⚠ INSTA...
Dependency confusion detection (user-declared private list)
Birsan 2021 attack: an attacker publishes a package on public PyPI / npm matching your private package name, and misconfigured index priority pulls the public copy into your CI. There is no public reg...
If you find a failure mode that isn't listed here, that's a bug in our disclosure — please open an issue.
EMBEDDABLE BADGES
Any analyzed project gets a live Soyce Score badge. Color shifts with the number.