hay — the manual

Start here

Homebrew is the supported install path on macOS and Linux. It installs the exact archives the release workflow built, whose checksums entered the formula only after their build-provenance attestations verified:

brew install mneves75/tap/hay

Or build from source — one line, or inspect first (Rust edition 2024, MSRV 1.88):

curl -fsSL https://raw.githubusercontent.com/mneves75/hay/v0.3.0/install.sh | HAY_REF=v0.3.0 bash

# inspect-first form:
git clone --depth 1 --branch v0.3.0 https://github.com/mneves75/hay.git && cd hay && HAY_REF=v0.3.0 ./install.sh

The script gates on the toolchain version, clones, builds release, installs via cargo install, and smoke-tests the installed binary. Set CARGO_INSTALL_ROOT to choose its destination; the script passes that root to Cargo and verifies the same path. Then search exactly like you would with ripgrep:

hay validateSession src/          # ranked results, best first
hay -t ts -w fetchUser            # TypeScript only, whole words
hay -C 3 -m 20 'parseConfig'      # context, capped result count

The deal hay offers is narrow and honest: each complete search returns the exact match set from an equivalently configured ripgrep invocation, verified continuously by a differential test that normalizes traversal inputs on both sides — but ordered by likelihood of being the answer instead of by file path. If the corresponding orderings differ in which lines match, that is a bug; the project treats it as such.

The mental model

hay is built around one opinion, borrowed from how coding agents read: a line that declares a thing is worth more than a line that merely mentions it. When an agent searches validateSession, the function definition is the answer; the forty call sites are supporting material; the dead planning document that mentions the name is noise.

Mechanically, hay uses ripgrep’s matching and walking crates with a deterministic ignore policy: repository .gitignore rules apply, while global gitignore, .git/info/exclude, .ignore, and .rgignore do not. The differential harness disables those same non-.gitignore sources for ripgrep. hay then retains the 20,000 strongest-by-prescore candidates, scores them, sorts them, and prints. Scoring never changes which retained lines match — only their order.

If you want the full story — why ranking exists at all, what a bootstrap interval is, and the C bug the benchmark itself caught — read BENCHMARK_FEYNMAN.html. It assumes no information-retrieval background.

How results are ranked

Every matching line gets a score from four signals. The defaults:

Tie-breaks fall back to file path, so output is deterministic for a given tree.

Results are interleaved by file

After scoring, hay round-robins the ranked list by file: the first pass carries each file’s strongest line, the second its next-strongest, and so on. An agent opens files, so a first page of ten files is worth more than ten lines of one module — measured on the behavioural corpus, this alone lifted the answer-in-top-10 rate from 59.2% to 78.0% without changing a single score. The sequence of distinct files is unchanged, so -l output is identical.

hay --no-diversify config .   # strict score order instead

Modes that do not rank

-c, --count-matches, -v, -o and --stream have nothing to order. They run ripgrep's way — its parallel traversal, its output, its per-file -m, and no candidate cap, so a pattern matching more than 20,000 lines is answered exhaustively rather than exiting 2. Until 0.3.0 the first four exited 2 telling you to go and use rg; a search tool you can only reach for once you already know the question ranks is a decision you have to make before you can use it.

hay --stream config .      # everything rg would print, streaming, no cap
hay -c config .            # matching lines per file
hay -o config .            # just the matched substrings
hay -v config .            # the lines that did not match

Turning signals off

Each signal has an ablation switch, because a contribution you cannot switch off is a belief, not a measurement:

hay --no-definition validateSession .
hay --no-path        validateSession .
hay --no-word        validateSession .
hay --no-tf          validateSession .
hay --no-diversify   validateSession .

Four other signals were built, measured and deleted: exact-case matching, a comment penalty, binary whole-word matching, a filename match, and (in 0.3.0) a markdown-heading signal that reversed hay's documentation deficit and still did not ship, because the benchmark that liked it defines its ground truth as the thing the signal detects.

This is also how several published claims here were proven: the definition signal’s value on C was established by turning it off and watching the kernel score collapse to ripgrep’s.

Flags

The parser is lexopt — the same crate ripgrep moved to — so combined short flags (-in), --flag=value, attached values (-C3) and -- all work as you expect.

Output formats

Plain

Ripgrep-compatible path:line:text, rank-ordered, with -- marking a true discontinuity between context windows:

$ hay -C1 -F validateSession src/
src/auth.ts:2:export function validateSession(t: string) {
src/auth.ts-3-  return t
--
src/session.ts:17:// delegates to validateSession

JSON Lines (--json)

Ripgrep-shaped objects, one per line, match and context messages only:

{"type":"match","data":{"path":{"text":"src/auth.ts"},
 "lines":{"text":"export function validateSession(t: string) {\n"},
 "line_number":2,"absolute_offset":31,
 "submatches":[{"match":{"text":"validateSession"},"start":16,"end":30}]}}

• No begin/end/summary records: those describe a file-scoped stream, and rank order interleaves files by design.
• Under -l --json you get plain paths — the files-only question has no match record to carry it, and a lone begin would be neither the documented contract nor rg-shaped.
• Every line is one valid JSON object, so jq -c 'select(.type=="match")' composes cleanly (recipe below).

--explain: reading a score

Each result is prefixed with its total score and the per-signal breakdown:

$ hay --explain -F validateSession src/ | head -1
   9.00 [def +6.0 path +1.0 word +1.0 tf +1.00]  src/auth.ts:2:export function validateSession(t: string) {

Differences from ripgrep

For equivalent flags and normalized traversal inputs, match semantics are parity by construction. The differential test fails the build if hay and the corresponding ripgrep invocation disagree on which lines match.

Exit codes

The candidate cap

hay buffers up to 20,000 matching lines to score them. A pattern broader than that (say, -e e on the Linux kernel) ranks only the 20,000 strongest-by-prescore candidates — and says so on stderr while still exiting 2, because the printed list is not every match in the tree. For exhaustive counting or completeness checks on very broad patterns, use rg. That is not a limitation hay apologises for; it is the trade that makes ranking affordable, stated where you can see it.

Recipes for agent loops

Claude Code, Codex CLI, pi, Aider, Cursor — they all read search results the same way: first page, act on it. That is exactly the surface hay improves, so wiring it into an agent is one paragraph of instructions plus the right defaults.

## Code search

Use `hay` instead of `rg` for concept and symbol searches:

    hay -F -m 10 "<symbol>" src/

- always `-F` when searching identifiers (they are literals, not regexes)
- exit 0 = results ranked best-first; 1 = no match; 2 = INCOMPLETE — read stderr,
  then narrow (add -g/-t) rather than trusting the output wholesale
- need exhaustive lists or match counts? fall back to `rg`
- structured: `hay --json -F "sym" . | jq 'select(.type=="match")'

Make hay the default, not just available

The reliable way to make an agent reach for hay first is an instruction-level default: a rule that names hay as the default and rg as the explicit fallback. Agents follow tool rules far more reliably than they discover tools. Strengthen the universal snippet's first line to a default:

## Code search — defaults

`hay` is the DEFAULT search tool in this repo. Reach for it first for every
"where is X / what defines X / how does X work" search:

    hay -F -m 10 "<symbol>" .

Use `rg` only when the task needs every match, an exact count, `-v`, or
path-ordered output — and say so when you switch.

• Claude Code — put the rule in CLAUDE.md (or keep it in AGENTS.md and add a CLAUDE.md containing just @AGENTS.md — Claude Code does not read AGENTS.md directly). Pre-approve the binary so the default costs no permission prompts: /permissions → allow Bash(hay:*). Claude Code's built-in Grep tool is ripgrep internally and cannot be rewired; the rule should therefore say "prefer running hay in the shell over the Grep tool for concept searches", which Claude honors.
• Codex CLI — the same rule in the repo's AGENTS.md holds per project; for every project, put it in ~/.codex/AGENTS.md. State the rg fallback explicitly or Codex will treat hay as a synonym and lose the exhaustiveness distinction.
• pi — the repo AGENTS.md is read natively; nothing else needed.
• Aider — a CONVENTIONS.md file is inert on its own: load it with aider --read CONVENTIONS.md, or persist it in .aider.conf.yml as read: [CONVENTIONS.md].
• Cursor — rules are .mdc files under .cursor/rules/ with a rule type; make this one always-on (alwaysApply: true in the frontmatter, or the "Always" type in the UI), otherwise it may be absent exactly when the agent picks its search tool.
• Verify it took — ask the agent "where is <some symbol> defined?" and check the transcript ran hay. A default that was never observed firing is a claim, not a configuration.

Find the definition, read it

# fixed-string, top hit only — the common agent case
hay -F -m 1 "$symbol" "$root"

Structured consumption

# every match as TSV: path, line number, trimmed text
hay --json -F "$symbol" "$root" \
| jq -r 'select(.type=="match")
         | [.data.path.text, .data.line_number,
            (.data.lines.text | gsub("\\n$";"") | .[0:160])] | @tsv'

Context without reordering surprises

hay -C 3 -m 10 -F "$symbol" "$root"   # blocks stay grouped, gaps show as --

Honesty check on a big tree

hay -m 0 -F "$symbol" "$root" 2>err.log || true
grep -q "ranked the .* strongest" err.log && echo "truncated — switch to rg for exhaustiveness"

Install on an agent's machine

brew install mneves75/tap/hay   # macOS and Linux

# or from source:
curl -fsSL https://raw.githubusercontent.com/mneves75/hay/v0.3.0/install.sh | HAY_REF=v0.3.0 bash   # inspect first!
# inspect-first form:
git clone --depth 1 --branch v0.3.0 https://github.com/mneves75/hay.git && cd hay && HAY_REF=v0.3.0 ./install.sh

FAQ

Is it faster than ripgrep? Roughly comparable on typical queries; ranking adds work, measured honestly on the benchmark page. Read the timing chart there — ratios, not absolutes, survive machine noise.

Why did my test file rank low? The path prior downgrades tests deliberately: in agent transcripts, tests are rarely the answer to “where is X defined”. Overriding is a one-flag matter (--no-path), and the ablation switches exist precisely so you never have to trust the prior.

Can I use it as a drop-in in scripts? Yes, wherever scripts consume path:line:text and tolerate rank order plus the three declined flags above. Where scripts need exhaustiveness guarantees, keep rg.

Does the ranking ever change which lines match? Never. That is invariant #1, policed by a differential test against ripgrep on every change to walking, matching, or output.

Where do the numbers on the website come from? BENCHMARK.md / benchmark.html: public corpora, ground truth from a parser (ast-grep), paired bootstrap intervals cross-checked by Fisher randomization. The honest headline is also there: the task favours hay by construction, so read margins, and weigh them against the behavioural evaluation described in README.md, whose ground truth nobody designed around the tool.

The measurement kit

hay ships inside a repository that is mostly instrument: measure-mrr.ts turns local agent transcripts into a behavioural test collection (queries paired with the files the agent opened next), benchmark.ts runs the public-corpus comparison, and BENCHMARK_FEYNMAN.md explains both from zero. Developer-facing commands live in HOWTO.md; the negative result that started it all is the top of README.md. One rule for visitors: grep-hygiene.ts is the falsified metric, kept reproducible — do not extend it, do not quote it.