Artifact 494f79bc2179bd51ff9e3c71f1d758ba7794cfdf6fde513a4b1cf2a9cbafef73:
- File
CHANGELOG.md
— part of check-in
[60ef41b5b1]
at
2026-10-03 15:59:54
on branch trunk
— fix(blob): always inflate blob.content on load (#235) (#238)
* fix(blob): always inflate blob.content on load (#235)
blob.Load decided whether a row was stored raw by comparing len(content) with blob.size. That guess is unsound for delta rows: blob.size is the expanded artifact's length, while content is the compressed delta, and the two can be equal by coincidence. When they are, Load handed back the still-compressed bytes, and the delta applier failed with "expected integer at pos 0".
Go 1.27 replaced compress/flate's encoder, so the compressed size of any given delta shifted. The fixture in TestExpand_SingleDelta and TestIsAvailable_GroundedChain now compresses to exactly the target size (62 bytes), which exposed the bug. Any real repository could hit it under either Go version.
Fossil itself never makes this guess: content_of_blob (src/content.c:218-230) inflates blob.content unconditionally, and every writer here stores the compressed form. Load now does the same. TestLoadInflatesWhenStoredLengthEqualsSize forces the coincidence directly so it fails on the old code under any Go version.
Separately, TestDecodeRejectsOversizeDeclaredLength built its container from a 64-byte payload. Go 1.27's flate skips match-finding for inputs under 128 bytes, so that payload came out as a stored block larger than its input and tripped the helper's did-it-compress guard. The test now uses 1 KiB. The property it asserts is unchanged.
* docs(changelog): note the blob.Load inflate fix (#235) (user: [email protected], size: 25313) [annotate] [blame] [check-ins using]
Changelog
All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
[Unreleased]
Fixed
A non-forced
Extractnow checks before it changes anything (#230). It used to replace the checkout's file list first, so a refusal lost pending adds, renames and merge state and leftStatusseeing an empty checkout. It also compared disk with the target version, so a clean checkout could not switch to a version that changed a file. Extract now follows fossil'scheckout: it refuses while the current version has unsaved changes that the switch would lose, or when an untracked file sits where the target would write different content. A tracked file that is missing from disk no longer blocks a switch, as in fossil.Reading a stored artifact could return compressed bytes (#235).
blob.Loadtreated a row as stored uncompressed whenever its stored length equalledblob.size. For a delta row those are different things (the compressed delta and the full artifact's length) and can be equal by chance, so the delta step then failed. Fossil always inflatesblob.content, and so does go-libfossil now. Go 1.27's new compressor made one test fixture hit the case, but any repository could.Checkouts created by the fossil binary (
fossil open) now work throughOpenCheckout(#228).Statusfailed outright withconverting NULL to stringonvfile.mhash, because go-libfossil judged every file against that column, which fossil only fills while a merge is pending. Change detection now follows fossil's own scan: a file is compared with the artifactvfile.mridnames, adds and merge states move through the samechngedtransitions, and fossil adds, renames andorigname==pathnamerows are classified the way fossil classifies them.A non-forced
Extractno longer overwrites uncommitted work in a fossil-made checkout. The local-edit guard swallowed the same NULL scan error and skipped its check, and it never considered a merge that fossil had applied but not yet committed; both now refuse unlessForceis set.Addwritesvfile.islink, which fossil's checkout schema declares with no default, so a file added by go-libfossil to a fossil-made checkout reads back cleanly in both tools. New rows no longer populatemhash.
[0.9.0] - 2026-08-07
Fixed
- New artifacts are named with the algorithm the repository's
hash-policyselects, rather than always SHA1. Committing into a repository whose policy is sha3 -- thefossil initdefault since 2.10 -- wrote SHA1 artifact IDs, so every F-card in the new manifest carried a differently-derived hash from the one already in the parent manifest andfossil diffreported every tracked file as CHANGED even when one file had been edited. Repository integrity was never affected and canonical fossil could always read the result; the damage was unreadable diffs and a permanently mixed-hash repository, plus artifacts ashun-sha1peer would reject.
Content already stored under the other algorithm's name is reused rather
than stored a second time, unless the policy forbids it (sha3-only,
shun-sha1). Without that, a repository holding SHA1 artifacts under a
sha3 policy would churn every F-card on its next commit -- the same
every-file-CHANGED symptom, mirrored. Only a stored blob qualifies: a
phantom is a name the repository knows of but has no content for.
checkout.Manage and uv.Write read the same policy instead of each
guessing from the width of an existing name. uv.Write's guess read
project-code, which is 40 hex characters in every repository whatever the
policy, so it could only ever choose SHA1.
- Repositories created by this library are seeded with
hash-policy, so they name artifacts the wayfossil initdoes. Without the row, naming derived the algorithm from the repository's own artifacts, and a repository with no artifacts yet settled on SHA1 -- then stayed there permanently, because the first artifact it wrote was SHA1 too.
A clone is the exception and keeps no seeded policy: it adopts its source's,
which the absent row expresses by deriving the algorithm from the artifacts
the clone actually receives. Canonical fossil propagates the source's policy
value into a clone; the sync protocol carries no configuration, so this
reaches the same answer by inference. The two differ only for a source
explicitly set to sha1 that nonetheless holds a SHA3 artifact.
Added
db.HashPolicySHA1,db.HashPolicyAuto,db.HashPolicySHA3,db.HashPolicySHA3Onlyanddb.HashPolicyShunSHA1name Fossil'shash-policyvalues, so the seed path and the naming path read one definition instead of each carrying its own copy of the enum.
Changed
- Repositories created by this library now name their artifacts SHA3 rather
than SHA1. No API changed, but the same content committed into a newly
created repository produces a different artifact ID than it did in 0.8.1.
Existing repositories are unaffected: their artifacts keep their names, and
their own
hash-policycontinues to decide how new ones are named.
[0.8.1] - 2026-07-30
Fixed
- Crosslink reports the underlying cause when it fails, instead of replacing it with a generic message.
- Test fossil servers are reaped before their temporary directories are removed, and the unit-test timeout was raised; leftover servers holding files open made teardown flaky.
Changed
- Generated and local-only files are no longer tracked.
[0.8.0] - 2026-07-29
Fixed
mlink,plink,tagxrefandleafare derived the way canonical fossil derives them, in both the sweep and the commit path, including derivingleafby canonical's branch rule. mlink derivation, previously duplicated, is now unified in one place.- Tag propagation stops at branch boundaries instead of crossing them, and the three tag-apply paths share one propagation helper.
- Authenticated sync interoperates with canonical fossil.
- Clone retries a dropped exchange round instead of aborting.
- The project code is derived, and artifact visibility is written inside the storing transaction rather than after it.
- Ticket artifacts are crosslinked, and ordinary file blobs no longer emit a warning.
- Checkins cascade-linked while referencing unavailable blobs are deferred until those blobs arrive.
- Submodules are pinned to v0.7.0 rather than an orphaned pseudo-version.
Added
- The CLI selects its SQLite driver by build tag.
Performance
- Clone no longer re-seeks
blob.uuidper F-card, and no longer over-allocates delta output. - The post-sync crosslink sweep no longer re-examines the whole repository.
- Content received during sync is reused in crosslink instead of being read back.
[0.7.0] - 2026-07-27
Added
libfossil versionprints a single, stable, machine-parseable build identifier -- module version (or a-ldflags -X github.com/danmestas/go-libfossil/cli.buildVersion=...override for release builds), Go toolchain version, and platform -- and exits 0. Previouslyversionwas not a recognized command at all and fell through to the unrecognized-argument path. There is no global--versionflag:repo extractalready owns--versionas a command-scoped flag (the source version to extract), and kong's global flags are visible in every subcommand's context, so a root-level--versionwould collide with it.libfossil repo serveCLI command wires the already-publicRepo.ServeHTTPxfer HTTP server into the command surface. A repository created or cloned with this tool can now be served to a peer -- including a stockfossil clone-- without writing Go. Binds to127.0.0.1:8080by default; override with--addr host:port. Process interrupt (Ctrl-C/SIGTERM) cancels the same contextServeHTTPalready shuts down on; no new shutdown mechanism was introduced.
Fixed
- The xfer HTTP server now sends an explicit
Content-Lengthon every clone and sync reply. The full response is already materialised byEncodebefore the first byte is written, so its exact length is known for free; setting the header keepsnet/httpfrom silently switching toTransfer-Encoding: chunkedonce the buffered body passes its internal flush threshold. Release-tagged fossil ≤2.23 reads the reply length only fromContent-Length(src/http.cleaves itsiLengthnegative otherwise) and aborts with "server did not reply" on any chunked reply above ~256KB, so those clients could not clone a repository larger than that from a libfossil server. No buffering was introduced — the response was always fully in memory — so #88's per-clone bound (DefaultCloneBatchBytes) still caps each reply. The fix is unconditional rather than negotiated: because the server never streams, chunked framing offers no memory saving overContent-Length, andContent-Lengthis what canonical fossil's own server emits, so it is strictly the more compatible framing for every client (issue #101). - The message decoder now selects a body's framing from its §4 Content-Type
rather than by trying each framing and seeing which parses. A body sent as
application/x-fossilis decoded as the §4.1 compressed container; every other type, includingapplication/x-fossil-uncompressed(used by clone-v3 replies), is decoded as plain card text — the same rule canonical fossil applies. Trial-based dispatch made a decompression failure indistinguishable from "wrong framing"; a compressed body that now fails to inflate is reported as the fault it is and never re-fed to the card parser. The former unprefixed raw-zlib framing was removed: it was checked before the prefixed container on the belief that real fossil emits it, but a fossil 2.28 server never does — its pull replies are the prefixed container and its clone replies are plain text — so the framing was only ever right by accident. Removing it also retires the length-prefix/zlib-header aliasing guard, which existed solely to disambiguate the two compressed framings that trial-dispatch conflated.xfer.Decodenow takes the Content-Type; call sites that carry no header (the byte-oriented NATS transport, both ends libfossil) pass the compressed type explicitly, matching what they always emit. The HTTP server rejects a request body whose Content-Type is absent or neither §4 type with 415 Unsupported Media Type instead of decoding it as plain card text: bodies come from untrusted peers, and without a framing signal decoding either way is a guess. Real fossil always sets the header, so no conforming peer is affected. (#106) - A large artifact now clones regardless of its position within a clone round.
The server charged its batch budget before each artifact and sent the one
that crossed it whole, so a round carried
budget + one whole artifact— and when ordinary sub-budget filler preceded a large artifact, that sum exceeded what the client could decode (xfer.MaxDecompressedBytes), failing the clone with a spurious "declared container length exceeds" error. The size at which a clone failed depended on incidental filler: 8–15 MB of filler ahead of a 58.7 MB artifact all failed, while 17 MB passed by pushing the artifact into a round of its own.emitCloneBatchnow flushes the round and sends an over-bound artifact alone when adding it would cross the client's decode bound, so the ceiling is a property of the artifact alone. The2 * budgetcompile-time guard now certifies what it appears to: filler plus one budget-sized artifact always fits in a decodable round. (#109) - A clone from a libfossil server could fail with a card-syntax error
(
decode card 16303: empty line after split) naming a card the response never contained. The message decoder tried three body framings in order and treated any decompression failure as "not this framing", so a well-formed compressed body that exceeded the decoder's own size bound fell through to the uncompressed branch and the still-compressed bytes were parsed as card text — thousands of nonsense cards until one split into no fields. Worse than the misleading error, an oversize or truncated compressed body was accepted: the garbage cards decoded without error and the artifacts they carried failed their hash check downstream. A body recognizable as zlib is now decompressed or reported, never reparsed. The bound is also raised to 64 MiB, since it sat below what this implementation's own server emits in a single clone round — two libfossil peers could not reliably clone from each other even though each could clone from fossil. A compile-time guard keeps the bound at least twice the server's clone batch budget, so filler plus one budget-sized artifact always fits in a round the client can decode. (#104) - Committing a file with zero-length content panicked in
blob.Store, which treated an empty artifact -- a normal, well-known Fossil blob -- as invalid input; the panic then triggeredmanifest.Checkin's postcondition defer, which re-panicked with an unrelated "manifestRid must be positive" message that masked the real cause.blob.Storenow accepts zero-length (and nil) content, andCheckin's defer re-panics with the original panic value instead of asserting a postcondition that was never going to hold mid-unwind (#68). - A commit no longer silently carries a tracked file's last-committed
content forward when that file has been deleted from disk without
Unmanage/Remove-- it now fails clearly, naming the file, matching fossil's own behavior for a missing file that is in scope for the commit being made. A missing file outside the commit's scope (relevant only to an explicitEnqueue) is left untouched exactly as before (#79). - Clone deadline now interrupts the mid-round phantom-fill crosslink cascade, ensuring the clone does not stall with phantoms outstanding. The same deadline is now threaded into the general push/pull phantom-fill cascade for consistency.
- Sync replies are now decoded by their wire's real
Content-Typeinstead of a hardcoded assumption. The HTTP server sends the correct type on each response; call sites receiving frames without a header (the byte-oriented NATS transport) pass the type explicitly. - PGP/SSH clearsigned manifests are now accepted: the deck module strips the clearsign framing before verification.
- Manifest now emits delete rows for files omitted by the check-in, matching fossil's own behavior. Renames now resolve their parent blob from the pre-rename path to ensure the rename-source blob exists and is reachable.
Changed
- Breaking: The module path has changed from
github.com/danmestas/libfossiltogithub.com/danmestas/go-libfossil. Allreplacedirectives ingo.modhave been dropped. Consumers must update their imports and remove anyreplacedirectives pointing to the old path. - Breaking:
StatusOpts,MergeOpts, andCheckoutOptshave been removed. No function anywhere accepted any of the three, and nothing constructed one:Checkout.Status()takes zero arguments,Repo.Mergetakes positional string arguments, andCheckoutOptshad no construction site at all. A public type with no call site documents a capability that does not exist — a consumer readingMergeOpts{Strategy: ...}in the docs could reasonably conclude a strategy-selecting merge API exists; it never did.Repo.MergeandCheckout.Statuskeep their current signatures — an options-struct refactor forMergewas considered and declined.CheckoutOpts.Forcewas also a third same-namedForcefield in this package, alongside the already-removedUpdateOpts.Forceand the real, fully-wiredExtractOpts.Force; removing it also removes that readability hazard.ExtractOpts.Forceis unrelated and unaffected. - Breaking:
CloneOpts.ProjectCodeandCloneOpts.ServerCodehave been removed. Neither was ever wired to anything —Cloneaccepted both fields but never forwarded them to the internal clone path, so setting either had no effect and gave no error. The internal clone options struct has no matching fields at all, so wiring them would mean designing semantics first (is a caller-supplied code a validation assertion or an identity override?) and no caller has asked for that.CloneResult.ProjectCodeandCloneResult.ServerCodeare unrelated and unaffected — they continue to report both, populated from the remote via the clone protocol negotiation, which is the only place they meaningfully originate. - Breaking:
UpdateOpts.Forcehas been removed. It was never wired to anything —Checkout.Updateaccepted the field but never forwarded it to the internal update path, so setting it had no effect and gave no error. Deleting it is honest about what the API actually does; real forcing semantics forUpdatecan be designed and added later as a new, deliberately-wired field if a caller needs them.ExtractOpts.Forceis unrelated and unaffected — it is fully wired and unchanged. - Breaking:
Checkout.Updatenow returns(UpdateResult, error)instead of a bareerror. The internal 3-way merge already tracked which files were written, removed, and left with conflict markers; the old signature discarded all of it, so a caller could not tell a clean update from one that silently wrote<<<<<<<conflict markers into working-tree files.UpdateResult.Conflictedlists the paths that were merged but not cleanly — this is a successful update (err == nil), not an error; a genuine failure still returns a non-nilerrorwith a zero-valueUpdateResult.Checkout.Extractis unchanged. - Breaking:
Repo.Timelinenow enumerates the repository'seventtable newest-first (every event kind by default, or a single kind viaTimelineOpts.Type), matching canonicalfossil timeline. The previous behavior — a first-parent walk from a required start rid — is preserved under a new, honestly-named method,Repo.Ancestry(LogOpts). The oldRepo.Timeline(LogOpts)was actually an ancestry walk masquerading as an enumeration: it never visited a second parent or a sibling branch head, so it silently omitted any check-in that wasn't a first-parent ancestor of the given start rid. There is no deprecated shim; callers of the oldTimelineshould switch toAncestryif they want the walk, or adopt the newTimeline(TimelineOpts)if they want a full enumeration. LogEntrygains aKindfield (EventKind) identifying which ofevent.type's six kinds (ci,e,f,g,t,w) an entry is.Parentsis only populated forKind == EventKindCheckin.- Breaking:
Repo.Timelineorders by(mtime DESC, rid DESC), a total order with rid as a true tie-break at exact mtime equality — a deliberate improvement over canonical fossil's baremtime DESCwith no tie-break, which can repeat or skip rows at a page boundary. Pagination uses a new opaqueCursortype: take one from a returnedLogEntry.Cursorand pass it back asTimelineOpts.Afterto resume immediately after that entry.TimelineOpts.Before time.TimeandTimelineOpts.After FslIDare deleted, not deprecated — callers constructing either field will fail to compile.Cursor's representation is intentionally hidden — it can only be obtained from aLogEntry, never built from a timestamp and a rid by hand, because a hand-built cursor derived from a roundedtime.Timeis not guaranteed to match its row exactly, which is what reintroduces skipped or duplicated rows at a page boundary in the first place. - Breaking:
LogEntryno longer has aCursorfield, andRepo.Timelinenow returns[]TimelineEntryinstead of[]LogEntry.LogEntryisRepo.Ancestry's result type; its cursor was always the zero value, which is structurally identical to a legitimate Timeline first-page call — so feeding anAncestryentry's cursor intoTimelineOpts.Aftersilently paginated from page one forever with no error, and documenting the field as invalid onAncestryentries did nothing to stop it.TimelineEntry(which embedsLogEntryand addsCursor Cursor) is nowRepo.Timeline's result type, so the cursor lives only on the type that has one to give. Callers that readentry.Cursor,entry.UUID,entry.User, etc. off aRepo.Timelineresult need no source change — those fields are still reachable throughTimelineEntry's embeddedLogEntry. Callers that stored aRepo.Timelineresult as[]LogEntry(rather than letting:=infer the type, or using[]TimelineEntry) will fail to compile. - Performance improvements throughout the sync and manifest subsystems: the
phantom-fill crosslink cascade now batches onto a shared content cache,
db.Queriercaches prepared statements to stop per-call SQL re-parse overhead, and delta-chain walks bound their hash-collision chain to prevent quadratic behavior on low-entropy input. History-walking expansions are routed through the content cache, cutting sub-64KiB delta-chain expansion allocation ~18x and boundingApplygrowth per-command to cut ~5x allocation overhead.
[0.6.3] - 2026-05-13
Fixed
- The SQLite DSN now sets
_txlock=immediate, so writers serialize atBEGINrather than discovering the conflict partway through a transaction (#33).
[0.6.2] - 2026-05-12
Fixed
Repo.Commitnow preserves the parent check-in's tracked files, instead of dropping files the commit did not itself touch (#30).
[0.6.1] - 2026-05-12
Added
CreateOpts.ProjectCodeaccepts an explicit project code when creating a repository, rather than always generating one (#31).
[0.6.0] - 2026-05-05
Fixed
- The WAL is checkpointed on
Close, so a repository written by this library is readable by canonical fossil without a recovery pass (#28).
[0.5.0] - 2026-05-04
- Added support for whole-checkin diff via empty filePath parameter.
- Added
Repo.UUIDFromRIDfor public rid-to-uuid lookup. - Added cross-repo release automation specification and implementation.
- Added release workflow with downstream dispatch for automated releases.
[0.4.5] - 2026-04-30
Fixed
- A clone against a hub that is being written to concurrently now converges, instead of looping without making progress (#17).
[0.4.4] - 2026-04-27
Fixed
- A partial xfer no longer panics with
rid=0; the incomplete transfer is handled rather than dereferenced (#14).
[0.4.3] - 2026-04-26
- Stabilized xfer encoder and multi-round sync with deferred manifest crosslink.
- Added
libfossil versioncommand to print build identifier including module version, Go toolchain version, and platform. - Wired live documentation URL into README and hugo baseURL.
- Added SVG architecture diagram and SDK refresh.
- Launched documentation site and Cloudflare auto-deploy.
[0.4.2] - 2026-04-26
- Fixed manifest crosslink deferral to handle referenced blobs arriving later.
[0.4.1] - 2026-04-25
- Implemented xfer encoder and multi-round sync foundation with manifest crosslink.
[0.4.0] - 2026-04-25
- Added
Repo.Pullpublic wrapper with HTTP transport and hostile-input assertions. - Added Tiger Style test coverage for
Checkout.Update. - Improved documentation and test coverage for
Repo.Pull.
[0.3.0] - 2026-04-21
- Added
Repo.ReadFilefor reading tracked file content from the repository at any version.
[0.2.0] - 2026-04-20
- Added
Repo.Mergefor branch-to-branch 3-way merge operations. - Added
Repo.Difffor computing diffs between tree states.
[0.1.0] - 2026-04-20
Initial open-source release of libfossil, a pure-Go implementation of the
Fossil SCM that reads and writes the same .fossil SQLite repository format.
Added
- Repository lifecycle: create new repos and clone from existing ones.
- Working-tree operations: checkout and checkin.
- Timeline traversal over commits and events.
- Merge and rebase primitives.
- Diff and annotate (blame) over tracked content.
- Manifest parsing and content-addressed blob storage.
- Sync protocol client/server for pulling and pushing between repos.
- Observer interfaces for sync and checkout, allowing external hooks into both network sync events and working-tree state transitions.
- SQLite driver abstraction with support for both
modernc.org/sqlite(pure Go) andncruces/go-sqlite3(cgo-free, wasm-based) backends. - Deterministic simulation test harness with BUGGIFY-style fault injection for exercising concurrency and failure paths.
- OpenTelemetry observer provided as a separate submodule to keep the core dependency footprint small.
wasip1/wasmbuild target for runninglibfossilunder WASI runtimes.