Slow Indexing

Symptoms

canopy index takes more than 2-3 minutes for a typical codebase.
Progress appears to stall with no output for long periods.
The embedding phase (built by default) takes considerably longer than the AST and search phases.

For reference, the Pith monorepo (622 TypeScript/Rust source files) indexes in approximately 2.6 seconds for an incremental run. A full re-index from scratch takes under 10 seconds on modern hardware.

Likely Causes

Cause	Indicator
Large repository with many files	`canopy index --list-ignored` reports > 50,000 files to index
Binary and generated files being indexed	Unusual file types in index
No `.gitignore` in the repository	No automatic exclusions
`--full` flag used unnecessarily	Wipes and rebuilds entire index
Embedding generation (on by default)	Slow or unreachable Ollama
First-time embedding model download	Downloading `nomic-embed-text` (~270 MB)

Step-by-Step Fix

Step 1: Check how many files would be indexed

canopy index . --list-ignored

This walks the repo and prints what would be indexed vs skipped (grouped by top-level directory) without indexing anything. If the would index count is unexpectedly high (tens of thousands), proceed to Step 2.

Step 2: Add exclusions to `.canopyignore`

Create or edit <repo>/.canopyignore (gitignore syntax):

dist/
build/
.next/
coverage/
node_modules/
vendor/
generated/
__pycache__/
target/
.cache/
*.min.js
*.bundle.js

Canopy already respects .gitignore automatically, plus a bundled default ignore set (node_modules/, build artifacts, and similar). Use .canopyignore for directories that exist in your repo but are not in .gitignore. One-off exclusions can also be passed on the command line with --ignore-pattern '<glob>'.

After adding exclusions, run a full re-index to apply them:

canopy index . --full

Step 3: Use incremental indexing for subsequent runs

After the initial full index, subsequent runs should be incremental (the default):

canopy index .

Only files modified since the last index are re-processed. Reserve --full for when you suspect the index is stale or corrupt.

Step 4: Address slow embedding generation

Embeddings are built by default. If the embedding phase is slow, check your Ollama server:

curl http://localhost:11434/api/tags

If Ollama is not running, start it:

ollama serve

The AST index and keyword search work without embeddings — only semantic search requires them, and embedding failure is never fatal. To skip the embedding phase and get a keyword-only index:

canopy index . --no-embeddings

Re-run canopy index . later to add the embeddings.

Step 5: Pre-provision the embedding runtime

If the first indexing run is slow due to model download, provision the embedding backend once up front:

canopy setup --only embedding

This installs Ollama and the embedding model. Subsequent indexing runs skip the download.

Step 6: Bound git history ingestion

Git commit history is ingested automatically whenever a .git/ directory is present, capped at 1000 commits by default. On repos with very deep history, tune or skip it:

# Ingest fewer commits
canopy index . --max-commits 200

# Skip git ingestion entirely
canopy index . --no-git

When to Escalate

Contact support at support@gulfshieldtech.com if:

Indexing a codebase of fewer than 10,000 source files takes more than 5 minutes with embeddings disabled.
canopy index --list-ignored shows significantly fewer files than expected (the index may be silently skipping files).

Include the output of:

canopy index . --log-level debug 2>canopy-debug.log
canopy doctor --repo .

Attach canopy-debug.log to your support request.