mighty-gerbils/gerbil
mighty-gerbils/gerbil
Watch
7
1
Fork
You've already forked gerbil
 3
Code Issues 138 Pull requests 11 Projects Releases Wiki Activity

v0.19: Add lsp, fmt, and lint tools #1514

Open
jay wants to merge 14 commits from jay/gerbil:v0.19-lsp-fmt-lint into v0.19-staging
pull from: jay/gerbil:v0.19-lsp-fmt-lint
merge into: mighty-gerbils:v0.19-staging
Conversation 4 Commits 14 Files changed 105 +27982 -78
jay commented 2026-10-19 23:37:49 +00:00
Member
Copy link

Three new tools ship with Gerbil and share one analysis engine. That engine uses Gerbil's own expander and compiler, not heuristics over the source text.

Command Tool Purpose
gerbil lsp gxlsp Language server (LSP 3.17, stdio)
gerbil fmt gxfmt Formatter
gerbil lint gxlint Linter

gxlsp

  • Features: diagnostics, definition, references, rename, hover, completion, signature help, document highlight, document and workspace symbols, formatting and range formatting, watched files.
  • Analysis: each open module is expanded as the module it is, in a worker process per package environment (gerbil.pkg). Macros and imports resolve exactly as gxc resolves them.
  • Diagnostics: once editing pauses, a buffer shows exactly what gerbil lint reports. Compiler errors arrive about 1 s after typing stops, from a low-priority worker, so typing stays fast.
  • Project index: the workspace is indexed in the background, which gives references, workspace symbols and rename across files.
  • Rename: limited to the workspace's own files. Every changed file is expanded again to verify the result; the server refuses a rename it cannot verify.
  • Configuration: initialization options analysisTimeout and indexWorkspace.

gxfmt

  • Commands: gxfmt check|fix PATH..., and stdin through - or --stdin-filename PATH.
  • What it changes: indentation and whitespace only. It keeps comments and line breaks, never collapses deliberate column alignment, and every result reads back to the same data.
  • .gxfmt: per-directory indentation rules for your own forms, e.g. (my-def defun), ((with-tx) 2). Rules are N, defun, block or call.
  • ; gxfmt: off / ; gxfmt: on: regions left byte-for-byte unchanged.
  • Library: :gerbil/tools/gxfmt/formatter exports format-source and format-source-lines. :gerbil/tools/gxfmt/rules exports load-indent-rules and default-indent-rules.

gxlint

  • Checks:
    • read-error, expansion-error: always on.
    • compile-error: the errors gxc -O would raise, such as arity or type mismatches and unknown keywords or slots.
    • unused-import, unused-definition, unused-local, set-imported, unused-suppression: on by default.
    • unused-parameter: off by default.
  • Suppression: ; gxlint: ignore ID, ignore-line, ignore-file.
  • .gxlint: per-directory enable, disable, severity, ignore GLOB, exclude GLOB, side-effect-modules, features.
  • Output: text or --format json, with --fail-level, -j and --timeout. Exit status is 0 when clean, 1 for findings, 2 for errors.

Other changes

  • src/tools/analysis/: the shared engine, with expansion, symbol facts, the project index and the worker pool.
  • etc/gerbil-mode.el: Emacs indentation now matches gxfmt. It reads .gxfmt and honors gxfmt: off regions. It also reads here-strings and #u8( correctly and indents with spaces only. A test checks that its rule table matches gxfmt's.
    • Breaking: the rules moved to a property only gerbil-mode reads, so it no longer changes scheme-mode's global indentation. As a result, (put 'x 'scheme-indent-function n) no longer affects gerbil-mode. Use .gxfmt or gerbil-put-indent instead.
    • No new Emacs dependencies.
  • src/.gxfmt: indentation rules for Gerbil's own macros.
  • Docs: src/tools/{gxlsp,gxfmt,gxlint}.md, doc/reference/dev/bach.md.

Compared with existing work

ober/gerbil-lsp This PR
Analysis Gambit read plus matching on form names (def, defstruct…); symbols found by text around the cursor Gerbil's own expander and compiler: user macros, imports and includes resolve as in gxc
Diagnostics gxc -S on the saved file The live buffer: read, expansion and compile errors plus lint findings, identical to gerbil lint
Rename Text replacement in open documents Binding-aware across the workspace, verified by expanding again
Formatting Gambit pretty-print, which drops comments gxfmt: keeps comments and layout, same output as Emacs indentation
Distribution Separate package Built with Gerbil, available as gerbil lsp, gerbil fmt and gerbil lint
  • ober/gerbil-lsp-plugin: packages ober/gerbil-lsp for Claude Code. It has no analysis of its own.
  • gerbil-mcp: an MCP server whose tools let AI agents evaluate, check and expand code. It serves agents, not editors, and doesn't replace an LSP, a formatter or a linter. It could call gerbil lint and gerbil fmt.

Notes

  • Macros run: as with gxc, analysis runs the user's macros. They run in worker processes, never in the server, and nothing is written to disk.
  • Memory: an analysis worker takes about 400–800 MB.
  • Testing: ./build.sh test tools/analysis/... tools/gxlint/... tools/gxlsp/... tools/gxfmt/... gives 25 modules and 279 cases. Also tested end to end in Emacs 30 with Eglot.

Warning

LLM-assisted with Opus 5.5 and gerbil-mcp

Three new tools ship with Gerbil and share one analysis engine. That engine uses Gerbil's own expander and compiler, not heuristics over the source text. | Command | Tool | Purpose | |---|---|---| | `gerbil lsp` | `gxlsp` | Language server (LSP 3.17, stdio) | | `gerbil fmt` | `gxfmt` | Formatter | | `gerbil lint` | `gxlint` | Linter | ## gxlsp - **Features:** diagnostics, definition, references, rename, hover, completion, signature help, document highlight, document and workspace symbols, formatting and range formatting, watched files. - **Analysis:** each open module is expanded as the module it is, in a worker process per package environment (`gerbil.pkg`). Macros and imports resolve exactly as `gxc` resolves them. - **Diagnostics:** once editing pauses, a buffer shows exactly what `gerbil lint` reports. Compiler errors arrive about 1 s after typing stops, from a low-priority worker, so typing stays fast. - **Project index:** the workspace is indexed in the background, which gives references, workspace symbols and rename across files. - **Rename:** limited to the workspace's own files. Every changed file is expanded again to verify the result; the server refuses a rename it cannot verify. - **Configuration:** initialization options `analysisTimeout` and `indexWorkspace`. ## gxfmt - **Commands:** `gxfmt check|fix PATH...`, and stdin through `-` or `--stdin-filename PATH`. - **What it changes:** indentation and whitespace only. It keeps comments and line breaks, never collapses deliberate column alignment, and every result reads back to the same data. - **`.gxfmt`:** per-directory indentation rules for your own forms, e.g. `(my-def defun)`, `((with-tx) 2)`. Rules are `N`, `defun`, `block` or `call`. - **`; gxfmt: off` / `; gxfmt: on`:** regions left byte-for-byte unchanged. - **Library:** `:gerbil/tools/gxfmt/formatter` exports `format-source` and `format-source-lines`. `:gerbil/tools/gxfmt/rules` exports `load-indent-rules` and `default-indent-rules`. ## gxlint - **Checks:** - `read-error`, `expansion-error`: always on. - `compile-error`: the errors `gxc -O` would raise, such as arity or type mismatches and unknown keywords or slots. - `unused-import`, `unused-definition`, `unused-local`, `set-imported`, `unused-suppression`: on by default. - `unused-parameter`: off by default. - **Suppression:** `; gxlint: ignore ID`, `ignore-line`, `ignore-file`. - **`.gxlint`:** per-directory `enable`, `disable`, `severity`, `ignore GLOB`, `exclude GLOB`, `side-effect-modules`, `features`. - **Output:** text or `--format json`, with `--fail-level`, `-j` and `--timeout`. Exit status is 0 when clean, 1 for findings, 2 for errors. ## Other changes - `src/tools/analysis/`: the shared engine, with expansion, symbol facts, the project index and the worker pool. - `etc/gerbil-mode.el`: Emacs indentation now matches gxfmt. It reads `.gxfmt` and honors `gxfmt: off` regions. It also reads here-strings and `#u8(` correctly and indents with spaces only. A test checks that its rule table matches gxfmt's. - **Breaking:** the rules moved to a property only gerbil-mode reads, so it no longer changes `scheme-mode`'s global indentation. As a result, `(put 'x 'scheme-indent-function n)` no longer affects gerbil-mode. Use `.gxfmt` or `gerbil-put-indent` instead. - No new Emacs dependencies. - `src/.gxfmt`: indentation rules for Gerbil's own macros. - Docs: `src/tools/{gxlsp,gxfmt,gxlint}.md`, `doc/reference/dev/bach.md`. ## Compared with existing work | | ober/gerbil-lsp | This PR | |---|---|---| | Analysis | Gambit `read` plus matching on form names (`def`, `defstruct`…); symbols found by text around the cursor | Gerbil's own expander and compiler: user macros, imports and `include`s resolve as in `gxc` | | Diagnostics | `gxc -S` on the saved file | The live buffer: read, expansion and compile errors plus lint findings, identical to `gerbil lint` | | Rename | Text replacement in open documents | Binding-aware across the workspace, verified by expanding again | | Formatting | Gambit `pretty-print`, which drops comments | gxfmt: keeps comments and layout, same output as Emacs indentation | | Distribution | Separate package | Built with Gerbil, available as `gerbil lsp`, `gerbil fmt` and `gerbil lint` | - **ober/gerbil-lsp-plugin:** packages ober/gerbil-lsp for Claude Code. It has no analysis of its own. - **gerbil-mcp:** an MCP server whose tools let AI agents evaluate, check and expand code. It serves agents, not editors, and doesn't replace an LSP, a formatter or a linter. It could call `gerbil lint` and `gerbil fmt`. ## Notes - **Macros run:** as with `gxc`, analysis runs the user's macros. They run in worker processes, never in the server, and nothing is written to disk. - **Memory:** an analysis worker takes about 400–800 MB. - **Testing:** `./build.sh test tools/analysis/... tools/gxlint/... tools/gxlsp/... tools/gxfmt/...` gives 25 modules and 279 cases. Also tested end to end in Emacs 30 with Eglot. > [!WARNING] > LLM-assisted with Opus 5.5 and `gerbil-mcp`
jay added 14 commits 2026-10-19 23:37:49 +00:00 
Serves LSP 3.17 over stdio as gxlsp and gerbil lsp: lifecycle,
incremental document sync with utf-8/16/32 positions, read-error
diagnostics and document symbols from a tolerant reader pass.
Unsaved buffers are expanded in memory by one long-lived worker
process per package environment, form by form so independent errors
are all reported. Workers are killed and respawned on deadlines,
crashes and memory growth, and exit when the server goes away.
Saving a module re-analyzes the open documents that import it.
Analysis workers index each buffer from its expansion plus a guarded
scan of the source, and ship definition sites for every referenced
binding: same file, project modules, compiled library modules mapped
back to their sources, and runtime externs. Positions the index cannot
know exactly get no answer rather than a guess. Navigation keeps
working mid-edit from the last good index mapped through edits.
Hover describes the binding go-to-definition resolves, with the
signature read from its definition form and the comment above it.
Completion offers only names in scope at the cursor and phase,
answering at once from the last good analysis while editing.
Signature help follows the argument position in unbalanced text.
A dedicated low-priority worker summarizes the workspace's modules in
the background without touching the workers that serve open documents.
References combine open documents with the project index and leave out
files whose summaries may be stale after a dependency changed.
Rename covers locals and project-level bindings across the workspace,
and refuses whenever it cannot be sure: library and generated names,
incomplete indexes, include fragments, other environments and
reserved names. Before answering, the edited files and their importers
are re-expanded in memory and the rename is refused if any reference
would change binding or any new error appears.
Keeps the author's line breaks and fixes indentation and whitespace
only, following Gerbil's Emacs indentation rules plus block rules
for with-* and call-with-* forms. Lexical and never failing; never
changes what the reader reads. gerbil fmt check|fix PATH...
Projects declare how their own macros indent in .gxfmt files, merged
from the root down. Regions between gxfmt: off and gxfmt: on comments
are left untouched. gerbil-mode now indents exactly like gxfmt, reads
.gxfmt too, and no longer changes plain scheme-mode buffers.
Formatting and range formatting answer with per-line edits computed by
the gxfmt library, reading .gxfmt rules fresh for every request.
gxfmt reads stdin from the descriptor directly, since Gambit polls a
pipe forever when it is already at end of input. Outgoing messages are
now encoded on the writer thread.
The expansion, binding index, project environment and worker modules
move out of gxlsp unchanged, so other tools can share them. The worker
run loop takes a hook for extra per-file checks and the command that
starts it.
Uses inside top-level include fragments are attributed to the fragment
file, so definition, references and rename see them; rename refuses
whenever a fragment's includers cannot all be verified. gxlsp reads
its protocol input from the descriptor so an empty input ends the
session, and the lexical-diagnostics fallback only publishes for the
version it was scheduled for.
Reports read and expansion errors, unused imports, definitions and
locals, and set! of imported bindings, analyzing each file in
isolated workers. Findings can be suppressed with gxlint: ignore
comments and configured with .gxlint files.
The project's own modules are optimized in import order inside the
worker, without writing files, so arity and type errors are reported
exactly as gxc would raise them. A worker whose optimization had to
be abandoned at the deadline retires instead of serving more requests.
Analysis workers run gxlint's checks with each document's .gxlint, so
a settled buffer shows exactly what gerbil lint reports. Compiler
errors arrive from a low-priority worker once the buffer is idle, so
typing stays fast. Adds user documentation for gxlint and gxlsp.
jay requested review from Owners 2026-10-19 23:38:02 +00:00
vyzo commented 2026-10-23 10:35:49 +00:00
Owner
Copy link

this will probably need some changes with typed-gerbil, better to merge it before.

this will probably need some changes with typed-gerbil, better to merge it before.
vyzo commented 2026-10-23 10:37:13 +00:00
Owner
Copy link

it will take me a little while to review this, it is very large.

it will take me a little while to review this, it is very large.
vyzo commented 2026-10-23 12:06:20 +00:00
Owner
Copy link

astra is reviewing.

astra is reviewing.
vyzo requested changes 2026-10-23 12:27:17 +00:00
vyzo left a comment
Copy link

Request changes on 19a0a675db1c04d82e8b7a2254758d85633ef393. The reusable analysis engine belongs in stdlib; that should be a merge requirement alongside fixing the correctness problems below.

  1. [P1] Rename can silently change program behavior—and its verifier approves it.
    index.ss:419 records an occurrence only when its source spelling matches the symbol name. That drops pipe-quoted identifiers. The fallback scan repeats the same mistake in tokens.ss:65.

    Reproduced with:

    (export f)
(def x 1)
(def (f) (let ((x 2)) |x|))

    Renaming the inner x to y produces:

    (def (f) (let ((y 2)) |x|))

    f now returns 1 instead of 2. I exercised the rename handler and verification callback: verification returned (verified) and accepted the edit. Re-expanding does not establish safety when verification uses the same incomplete occurrence index. Preserve reader-normalized identifier identity and account for every affected reference; refuse rename when coverage cannot be established.

  2. [P1] Formatting destroys an unrelated file occupying its temporary filename.
    gxfmt.ss:232 uses the predictable sibling .NAME.gxfmt-tmp, opens it with truncate: #t, then renames it over the source. I reproduced this with a pre-existing .source.ss.gxfmt-tmp: its contents were overwritten and the file disappeared.

    Concurrent invocations also collide. Use an exclusively created temporary file in the destination directory. The stdlib already provides this machinery in :std/io/tempfile.

  3. [P2] Glob exclusions silently skip unintended files.
    config.ss:154 discards the slash following ** and tries the remainder at every character position. Consequently:

    (glob-match? "**/foo.ss" "barfoo.ss")     ; => #t
(glob-match? "**/foo.ss" "dir/barfoo.ss") ; => #t

    Both should be false. This affects exclusions and diagnostic suppression. **/ must consume complete directory components, including zero components.

  4. [P2] Signature parameter offsets violate the protocol.
    signature-help.ss:76 uses the negotiated document encoding for signature-label offsets. Those offsets are specifically UTF-16. For (λ x) with UTF-8 negotiated, I reproduced [4,5] for x; the required range is [3,4].

    It also always sends offset pairs without checking labelOffsetSupport. Use UTF-16 offsets when supported and substring labels otherwise. Both requirements are described in the official signature-help specification.

  5. [P2] Deferred analysis has no global worker limit.
    scheduler.ss:791 creates a deferred worker per environment, and scheduler.ss:883 starts them outside max-live-workers. Opening documents across enough environments can therefore start arbitrarily many processes. The PR itself estimates 400–800 MB per worker. Lower process priority does not bound memory use. Deferred work needs a bounded queue and an explicit concurrency budget.

The structural changes I would require are:

  • Put the reusable engine in stdlib with a usable library API. Source positions, lexical analysis, expansion/indexing, binding facts, formatting, lint checks and rename verification should expose typed results independently of CLI or LSP transport. Keep command dispatch, protocol encoding, subprocess hosting and editor scheduling in tools. Currently even analyze-buffer is housed in the worker module and returns its wire representation.

  • Remove the hand-maintained expander fork. expander-mirror.ss:3 explicitly documents 532 lines of copied expander behavior that must track core changes. It also replaces the global reader procedure to implement source overlays. Expose buffer/source-provider and diagnostic-collection hooks in the expander. Moving this file unchanged into stdlib would preserve the maintenance problem and introduce unsafe concurrency assumptions into a library API.

  • Consolidate the scanners. Formatter tokenization, cursor context, navigation, lexical-text detection and suppression handling each implement overlapping lexical rules. This already produces inconsistent behavior: I reproduced the cursor scanner reporting code inside a here-string, while the formatter lexer explicitly handles here-strings. A shared tolerant lexer should provide token spans and lexical state.

  • Use shared traversal procedures or compiler visitor machinery. Indexing, definition collection, binding classification and parameter collection each dispatch over expanded core forms. Centralize traversal and phase handling, with operation-specific callbacks or methods. The compiler already has the defcompile-method framework. Surface syntax recognition also needs consolidation; separate lists of recognized binding forms will drift as macros evolve.

  • Generate repetitive wire codecs from declarations. payload.ss manually maintains structures, exports, serializers and matching deserializers. A small schema macro can generate the mechanical parts while retaining explicit code for compression and special fields. This is a concrete opportunity for macros to remove duplicated knowledge.

There is also straightforward stdlib duplication:

New implementation Existing facility / change
Manual LRU cache with timestamps and a full-table eviction scan :std/struct/lru
Manual path simplification, plus separate containment helpers :std/string/path — path-simplify, subpath?
Handwritten lint option parser :std/cli/getopt, already used by the formatter
Private logger :std/log
Redefined string-suffix? and substring search Existing string procedures

The formatter and linter also duplicate their stdin-reading loops, and several components independently traverse source directories. Shared procedures with explicit policy arguments would remove that repetition.

Validation: 13 test modules passed using the installed Gerbil runtime with the PR's sources loaded. The broader run encountered two source-path expectation failures and subsequently terminated with exit 137; I cannot claim a complete suite pass. The rename, temporary-file collision, glob, signature-offset and here-string findings were separately reproduced. The rename reproduction used the real handler and verifier with an in-process stand-in for worker transport. No repository source files were changed.

Request changes on `19a0a675db1c04d82e8b7a2254758d85633ef393`. The reusable analysis engine belongs in stdlib; that should be a merge requirement alongside fixing the correctness problems below. 1. **[P1] Rename can silently change program behavior—and its verifier approves it.** [index.ss:419](https://git.cons.io/mighty-gerbils/gerbil/src/commit/19a0a675db1c04d82e8b7a2254758d85633ef393/src/tools/analysis/index.ss#L419) records an occurrence only when its source spelling matches the symbol name. That drops pipe-quoted identifiers. The fallback scan repeats the same mistake in [tokens.ss:65](https://git.cons.io/mighty-gerbils/gerbil/src/commit/19a0a675db1c04d82e8b7a2254758d85633ef393/src/tools/analysis/tokens.ss#L65). Reproduced with: ```scheme (export f) (def x 1) (def (f) (let ((x 2)) |x|)) ``` Renaming the inner `x` to `y` produces: ```scheme (def (f) (let ((y 2)) |x|)) ``` `f` now returns `1` instead of `2`. I exercised the rename handler and verification callback: verification returned `(verified)` and accepted the edit. Re-expanding does not establish safety when verification uses the same incomplete occurrence index. Preserve reader-normalized identifier identity and account for every affected reference; refuse rename when coverage cannot be established. 2. **[P1] Formatting destroys an unrelated file occupying its temporary filename.** [gxfmt.ss:232](https://git.cons.io/mighty-gerbils/gerbil/src/commit/19a0a675db1c04d82e8b7a2254758d85633ef393/src/tools/gxfmt.ss#L232) uses the predictable sibling `.NAME.gxfmt-tmp`, opens it with `truncate: #t`, then renames it over the source. I reproduced this with a pre-existing `.source.ss.gxfmt-tmp`: its contents were overwritten and the file disappeared. Concurrent invocations also collide. Use an exclusively created temporary file in the destination directory. The stdlib already provides this machinery in [:std/io/tempfile](https://git.cons.io/mighty-gerbils/gerbil/src/commit/19a0a675db1c04d82e8b7a2254758d85633ef393/src/std/io/tempfile.ss#L29). 3. **[P2] Glob exclusions silently skip unintended files.** [config.ss:154](https://git.cons.io/mighty-gerbils/gerbil/src/commit/19a0a675db1c04d82e8b7a2254758d85633ef393/src/tools/gxlint/config.ss#L154) discards the slash following `**` and tries the remainder at every character position. Consequently: ```scheme (glob-match? "**/foo.ss" "barfoo.ss") ; => #t (glob-match? "**/foo.ss" "dir/barfoo.ss") ; => #t ``` Both should be false. This affects exclusions and diagnostic suppression. `**/` must consume complete directory components, including zero components. 4. **[P2] Signature parameter offsets violate the protocol.** [signature-help.ss:76](https://git.cons.io/mighty-gerbils/gerbil/src/commit/19a0a675db1c04d82e8b7a2254758d85633ef393/src/tools/gxlsp/signature-help.ss#L76) uses the negotiated document encoding for signature-label offsets. Those offsets are specifically UTF-16. For `(λ x)` with UTF-8 negotiated, I reproduced `[4,5]` for `x`; the required range is `[3,4]`. It also always sends offset pairs without checking `labelOffsetSupport`. Use UTF-16 offsets when supported and substring labels otherwise. Both requirements are described in the [official signature-help specification](https://raw.githubusercontent.com/microsoft/language-server-protocol/gh-pages/_specifications/lsp/3.17/language/signatureHelp.md). 5. **[P2] Deferred analysis has no global worker limit.** [scheduler.ss:791](https://git.cons.io/mighty-gerbils/gerbil/src/commit/19a0a675db1c04d82e8b7a2254758d85633ef393/src/tools/gxlsp/scheduler.ss#L791) creates a deferred worker per environment, and [scheduler.ss:883](https://git.cons.io/mighty-gerbils/gerbil/src/commit/19a0a675db1c04d82e8b7a2254758d85633ef393/src/tools/gxlsp/scheduler.ss#L883) starts them outside `max-live-workers`. Opening documents across enough environments can therefore start arbitrarily many processes. The PR itself estimates 400–800 MB per worker. Lower process priority does not bound memory use. Deferred work needs a bounded queue and an explicit concurrency budget. The structural changes I would require are: - **Put the reusable engine in stdlib with a usable library API.** Source positions, lexical analysis, expansion/indexing, binding facts, formatting, lint checks and rename verification should expose typed results independently of CLI or LSP transport. Keep command dispatch, protocol encoding, subprocess hosting and editor scheduling in tools. Currently even `analyze-buffer` is housed in the worker module and returns its wire representation. - **Remove the hand-maintained expander fork.** [expander-mirror.ss:3](https://git.cons.io/mighty-gerbils/gerbil/src/commit/19a0a675db1c04d82e8b7a2254758d85633ef393/src/tools/analysis/expander-mirror.ss#L3) explicitly documents 532 lines of copied expander behavior that must track core changes. It also [replaces the global reader procedure](https://git.cons.io/mighty-gerbils/gerbil/src/commit/19a0a675db1c04d82e8b7a2254758d85633ef393/src/tools/analysis/expander-mirror.ss#L516) to implement source overlays. Expose buffer/source-provider and diagnostic-collection hooks in the expander. Moving this file unchanged into stdlib would preserve the maintenance problem and introduce unsafe concurrency assumptions into a library API. - **Consolidate the scanners.** Formatter tokenization, cursor context, navigation, lexical-text detection and suppression handling each implement overlapping lexical rules. This already produces inconsistent behavior: I reproduced the cursor scanner reporting `code` inside a here-string, while the formatter lexer explicitly handles here-strings. A shared tolerant lexer should provide token spans and lexical state. - **Use shared traversal procedures or compiler visitor machinery.** Indexing, definition collection, binding classification and parameter collection each dispatch over expanded core forms. Centralize traversal and phase handling, with operation-specific callbacks or methods. The compiler already has [the `defcompile-method` framework](https://git.cons.io/mighty-gerbils/gerbil/src/commit/19a0a675db1c04d82e8b7a2254758d85633ef393/src/gerbil/compiler/method.ss#L68). Surface syntax recognition also needs consolidation; separate lists of recognized binding forms will drift as macros evolve. - **Generate repetitive wire codecs from declarations.** [payload.ss](https://git.cons.io/mighty-gerbils/gerbil/src/commit/19a0a675db1c04d82e8b7a2254758d85633ef393/src/tools/analysis/payload.ss#L199) manually maintains structures, exports, serializers and matching deserializers. A small schema macro can generate the mechanical parts while retaining explicit code for compression and special fields. This is a concrete opportunity for macros to remove duplicated knowledge. There is also straightforward stdlib duplication: | New implementation | Existing facility / change | |---|---| | [Manual LRU cache](https://git.cons.io/mighty-gerbils/gerbil/src/commit/19a0a675db1c04d82e8b7a2254758d85633ef393/src/tools/analysis/signatures.ss#L200) with timestamps and a full-table eviction scan | `:std/struct/lru` | | [Manual path simplification](https://git.cons.io/mighty-gerbils/gerbil/src/commit/19a0a675db1c04d82e8b7a2254758d85633ef393/src/tools/analysis/config-files.ss#L48), plus separate containment helpers | `:std/string/path` — `path-simplify`, `subpath?` | | [Handwritten lint option parser](https://git.cons.io/mighty-gerbils/gerbil/src/commit/19a0a675db1c04d82e8b7a2254758d85633ef393/src/tools/gxlint.ss#L110) | `:std/cli/getopt`, already used by the formatter | | [Private logger](https://git.cons.io/mighty-gerbils/gerbil/src/commit/19a0a675db1c04d82e8b7a2254758d85633ef393/src/tools/analysis/log.ss#L10) | `:std/log` | | [Redefined `string-suffix?`](https://git.cons.io/mighty-gerbils/gerbil/src/commit/19a0a675db1c04d82e8b7a2254758d85633ef393/src/tools/gxlsp/rename.ss#L69) and [substring search](https://git.cons.io/mighty-gerbils/gerbil/src/commit/19a0a675db1c04d82e8b7a2254758d85633ef393/src/tools/gxlsp/rename.ss#L1003) | Existing string procedures | The formatter and linter also duplicate their stdin-reading loops, and several components independently traverse source directories. Shared procedures with explicit policy arguments would remove that repetition. Validation: 13 test modules passed using the installed Gerbil runtime with the PR's sources loaded. The broader run encountered two source-path expectation failures and subsequently terminated with exit 137; I cannot claim a complete suite pass. The rename, temporary-file collision, glob, signature-offset and here-string findings were separately reproduced. The rename reproduction used the real handler and verifier with an in-process stand-in for worker transport. No repository source files were changed.
This pull request can be merged automatically.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u v0.19-lsp-fmt-lint:jay-v0.19-lsp-fmt-lint
git switch jay-v0.19-lsp-fmt-lint

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch v0.19-staging
git merge --no-ff jay-v0.19-lsp-fmt-lint
git switch jay-v0.19-lsp-fmt-lint
git rebase v0.19-staging
git switch v0.19-staging
git merge --ff-only jay-v0.19-lsp-fmt-lint
git switch jay-v0.19-lsp-fmt-lint
git rebase v0.19-staging
git switch v0.19-staging
git merge --no-ff jay-v0.19-lsp-fmt-lint
git switch v0.19-staging
git merge --squash jay-v0.19-lsp-fmt-lint
git switch v0.19-staging
git merge --ff-only jay-v0.19-lsp-fmt-lint
git switch v0.19-staging
git merge jay-v0.19-lsp-fmt-lint
git push origin v0.19-staging
Sign in to join this conversation.
Reviewers
No reviewers
vyzo
Labels
Clear labels   
UX

   
active development

   
backlog

   
blocker

   
bootstrap

   
bounty

   
bug

   
dependencies
Pull requests that update a dependency file

  
discussion

   
documentation

   
duplicate

   
enhancement

   
flaky test

   
help wanted

   
invalid

   
javascript
Pull requests that update javascript code

  
question

   
release

   
tendentious

   
wontfix
No labels
UX
active development
backlog
blocker
bootstrap
bounty
bug
dependencies
discussion
documentation
duplicate
enhancement
flaky test
help wanted
invalid
javascript
question
release
tendentious
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
mighty-gerbils/gerbil!1514
No description provided.