Community Detection Deep Dive

Question

What is community detection, what is it for exactly, and how does this vault already use it?

Answer

Community detection is an unsupervised graph algorithm that partitions a graph’s nodes into groups (“communities”) where nodes are more densely connected to each other than to the rest of the graph. It works purely from edge structure — no labels, no content analysis, no LLM. On a wiki, pages are nodes and [[wikilinks]] are edges, so it answers: “which pages form natural clusters just from who links to whom?”

How it works (Louvain — this vault’s implementation)

• Implemented at scripts/hybrid_query_helper.py:196 via nx.community.louvain_communities(U, resolution=1.1, seed=42). Louvain greedily moves nodes between communities to maximize modularity — how much denser intra-community edges are than random expectation. The current partition scores 0.5483, which is strong.
• Undirected projection — Louvain is canonically undirected, so the directed wikilink graph is projected with G.to_undirected().
• resolution=1.1 is not a default guess: it was chosen by a sweep over this vault’s own graph (docs/archive/optimization-roadmap.md Phase 0) as the highest-modularity value keeping the largest community under 40% of nodes; resolution=1.0 left it at 44.8% — under-resolved.
• seed=42 is fixed so the same graph always yields the same partition; an unseeded partition churns community IDs on every export and makes diffs meaningless.
• Stable IDs — each community’s ID is its lexicographically smallest member slug, because list position is not stable identity across networkx versions.
• scripts/graph-export.ps1:121-144 shells out to scripts/detect_communities_helper.py (PowerShell can’t run Louvain) and writes wiki/graph/communities.json. The MCP server exposes it as the graph_communities tool (mcp/server.py:533,766).

What it’s for, exactly

1. Surprising connections — operationalizes the vault’s core value proposition: “the LLM finds correlations between things you saved separately that you would never have drawn on your own.” Community detection makes that automatic instead of anecdotal.
2. Clustering, not retrieval — it groups related things you may never have linked directly; keyword search cannot see a cluster spanning topics.
3. Navigation at scale — past ~100 pages, “what are the themes of my vault?” stops being answerable by reading index.md; communities are a generated table of contents.
4. Hierarchical summarization — GraphRAG’s core premise is extract a graph, run community detection, then summarize each community to answer global questions.
5. Functional module discovery in code — the same math over call edges instead of wikilinks.

Ecosystem usage (evidence)

• nashsu/llm_wiki — Louvain for automatic knowledge clustering, one of its headline features.
• Graphify — Leiden (Louvain’s successor; better at resolving small communities) over code subsystems.
• Nowledge Mem — community detection in its explicit knowledge-graph layer.
• SwarmVault — Louvain with auto-tuned resolution for small/sparse graphs, with per-repo config override.
• GraphRAG — community detection as the backbone of global-query summarization.

What this vault’s current communities reveal

wiki/graph/communities.json (regenerated 2026-08-03, modularity 0.5483, 5 communities, 259 members):

Community ID	Size	What it is
adelaida-diaz-roa-second-brain-knowledge	98	Main-wiki core — llm-wiki, second-brain, knowledge-graph, memory systems
active-prompt	58	Prompt wiki — techniques, models, prompting guides
action-safety-tiering	54	Prompt wiki — code-agent memory, context windows, tokens
AGENTS	38	Prompt wiki — agent architecture, system prompts, anti-injection
accommodation	11	Main-wiki cluster — conversation design, intent classification

The algorithm independently rediscovered the two-wiki split — three prompt-wiki clusters vs two main-wiki clusters (the main-wiki core plus a small conversation-design cluster) — a grouping nobody hand-drew. Caveat: community IDs and sizes drift between runs (agent-memory-systems→adelaida-..., caveman→bifrost→action-safety-tiering) because IDs are lexicographic-minimum slugs, which change as pages are added; membership is the signal, not the name. The owner-readable interpretation of these communities is filed at wiki/queries/as-comunidades-do-grafo-explicadas.md.

Costs & caveats

• Parameters matter — resolution changes cluster size; the sweep made it data-driven but it is vault-specific.
• Modularity resolution limit — Louvain can miss small communities (why Graphify uses Leiden); resolution=1.1 is the mitigation here.
• IDs not stable — slugs shift as the graph grows.
• No interpretation = no value — the export is computed and exposed via MCP but nothing reads it for the owner. Bridge/isolated detection was roadmap; it is now implemented in scripts/detect_communities_helper.py and written to wiki/graph/insights.json (122 bridges, 0 isolated, 0 singleton communities on the current graph), and the owner-facing reading is filed as wiki/queries/as-comunidades-do-grafo-explicadas.md.

Recommended next step

The interpretation page as-comunidades-do-grafo-explicadas names the 5 communities for the owner and reads the new wiki/graph/insights.json (bridges = “surprising connections”; isolated = orphans; both zero-isolated healthy here). Open wiki/graph/index.html, which already colors nodes by community. The next step is to act on the bridges: the current graph shows only 12 bridges reaching out of the main-wiki core — deliberately cultivating pages that link the main wiki and the prompt wiki is the growth lever the algorithm is pointing at.

Related

• coolest-under-explored-feature — where community detection was identified as the vault’s under-explored insight layer
• knowledge-graph — the graph structure community detection runs on
• llm-wiki — the pattern whose wikilinks are the graph edges
• vault-roadmap — Phase 4 items 16-18 (typed relationships, graph traversal, hybrid search)
• nashsu-llm-wiki — most popular implementation using Louvain
• graphify — Leiden-based alternative (85.8k stars)
• the-llm-finds-correlations-between-things-you-saved-separately-that-you-would-never-have-drawn-on-your-own — the value proposition community detection operationalizes
• as-comunidades-do-grafo-explicadas — owner-readable plain-language reading of the current 5 communities + bridges

Sources

• wiki/graph/communities.json (generated 2026-08-03, modularity 0.5483, 5 communities, 259 members)
• wiki/graph/insights.json (generated 2026-08-03 — 122 bridges, 0 isolated, 0 singleton communities)
• scripts/hybrid_query_helper.py:196-229 — Louvain implementation, resolution/seed rationale
• scripts/graph-export.ps1:121-144 — shell-out to community detection helper
• mcp/server.py:533,766 — graph_communities MCP tool
• wiki/entities/nashsu-llm-wiki.md
• wiki/entities/graphify.md
• wiki/entities/nowledge-mem.md
• wiki/concepts/knowledge-graph.md
• wiki/queries/coolest-under-explored-feature.md
• wiki/queries/as-comunidades-do-grafo-explicadas.md
• raw/articles/knowledge-management-comparison-2026.md
• raw/external/github-com-swarmvault-e25e16e0.md
• raw/external/github-com-codebase-memory-mcp-f33fbe15.md