Artifact aa4adb862ec92201e4eff279b8f388df565a17fd365437f4c2b012b0f7025395:
- File
internal/xfer/encode.go
— part of check-in
[a745f4f2ac]
at
2026-07-28 00:44:04
on branch trunk
— fix: make authenticated sync interoperate with canonical fossil (#207)
* fix: make authenticated sync interoperate with canonical fossil (#203)
Every credentialed sync against a real fossil server was answered "login failed", whether or not the password was right. Push always needs a login, so push was unusable against any real server, as was pull from anything that did not grant anonymous read.
The login algorithm was never the problem. Two wire-level defects were, and each on its own produces that identical, password-independent rejection.
The push and pull cards carried their two codes in the wrong order. The canonical form is `push SERVERCODE PROJECTCODE` — canonical's own client writes `blob_appendf(&send, "push %s %s\n", zSCode, zPCode)` and both ends read the project code out of the second argument (src/xfer.c). We emitted and parsed project code first. A clone therefore adopted the server's server-code as its project code, and since a login signature is salted with the project code, every signature was derived from the wrong value. The same inversion put the server code where a server validates the project code, so an authenticated push was refused with "wrong project" even once the login worked.
The nonce covered a random comment that was never transmitted. The nonce is the hash of every byte following the login card, and the server re-hashes those bytes to check it (src/xfer.c check_tail_hash) — so the comment supplying the randomness has to be on the wire, not merely in the hash. We appended it to the hashed payload only, producing a nonce the server could not reproduce; it rejected the login before comparing the password at all, which is why a wrong password looked exactly like a right one. Comment lines now have a card type, so they can be sent as well as hashed.
Two smaller faults on the same path: a clone treated the server's round-0 "not authorized to clone" as fatal, ending the session before credentials were ever offered, where canonical tolerates that error on the first cycle precisely so the client can retry with the login card it can only build once it knows the project code; and the sync path signed with SyncOpts.ProjectCode rather than the code it had already resolved from the repository, so a caller that let the repo supply its project code — as the push and pull cards allow — panicked instead of authenticating.
The password derivation itself, sha1("<project-code>/<login>/<password>"), was correct and is unchanged. It is now pinned two ways: against the hash the fossil binary actually stores in user.pw, and against a login card captured from fossil 2.28 on the wire. Canonical's rule that an already hashed 40-hex password is used as-is is now honoured too.
Regression coverage stands up a canonical fossil server with anonymous capabilities removed, so a login card is the only way in, and asserts that a credentialed clone transfers content, a wrong password is refused, and a local commit pushed under authentication lands in the server's blob table and on its timeline. Nothing in the suite authenticated against real fossil before; clone was only ever exercised against servers granting anonymous read, where the login card is never sent, which is how this shipped.
Four xfer codec expectations that pinned the inverted argument order are corrected to the canonical bytes.
* fix(sync): correct the nonce doc to match what the server recomputes
The comment described the nonce as covering the encoded non-login cards and the signature as using the stored password hash. Both readings are wrong in ways that matter to anyone changing this code: the nonce is defined over the bytes that actually follow the login card on the wire, because the server recomputes it from what it received (src/xfer.c check_tail_hash), and the signature is salted with the project code rather than derived from user.pw directly. Hashing bytes the message does not carry produces a nonce the peer cannot reproduce, which is exactly the password-independent rejection this fix was chasing. (user: [email protected], size: 10773) [annotate] [blame] [check-ins using]
package xfer import ( "bytes" "compress/zlib" "encoding/binary" "fmt" "strings" "github.com/danmestas/go-libfossil/internal/deck" ) // EncodeCard writes the wire-format representation of a Card to w. func EncodeCard(w *bytes.Buffer, c Card) error { if w == nil { panic("xfer.EncodeCard: w must not be nil") } if c == nil { panic("xfer.EncodeCard: c must not be nil") } switch v := c.(type) { case *IGotCard: return encodeIGot(w, v) case *GimmeCard: return encodeGimme(w, v) case *PushCard: return encodePush(w, v) case *PullCard: return encodePull(w, v) case *CookieCard: return encodeCookie(w, v) case *ReqConfigCard: return encodeReqConfig(w, v) case *PrivateCard: return encodePrivate(w) case *CloneCard: return encodeClone(w, v) case *CloneSeqNoCard: return encodeCloneSeqNo(w, v) case *UVGimmeCard: return encodeUVGimme(w, v) case *PragmaCard: return encodePragma(w, v) case *LoginCard: return encodeLogin(w, v) case *CommentCard: return encodeComment(w, v) case *ErrorCard: return encodeError(w, v) case *MessageCard: return encodeMessage(w, v) case *UVIGotCard: return encodeUVIGot(w, v) case *FileCard: return encodeFile(w, v) case *CFileCard: return encodeCFile(w, v) case *ConfigCard: return encodeConfig(w, v) case *UVFileCard: return encodeUVFile(w, v) case *SchemaCard: return encodeSchema(w, v) case *XIGotCard: return encodeXIGot(w, v) case *XGimmeCard: return encodeXGimme(w, v) case *XRowCard: return encodeXRow(w, v) case *XDeleteCard: return encodeXDelete(w, v) case *UnknownCard: return encodeUnknown(w, v) default: return fmt.Errorf("xfer: cannot encode %T", c) } } func encodeIGot(w *bytes.Buffer, c *IGotCard) error { w.WriteString("igot ") w.WriteString(c.UUID) if c.IsPrivate { w.WriteString(" 1") } w.WriteByte('\n') return nil } func encodeGimme(w *bytes.Buffer, c *GimmeCard) error { w.WriteString("gimme ") w.WriteString(c.UUID) w.WriteByte('\n') return nil } // encodePush writes "push <server-code> <project-code>\n". // // Server code first, project code second: canonical emits both cards as // `blob_appendf(&send, "push %s %s\n", zSCode, zPCode)` (src/xfer.c // client_sync) and parses them as `push SERVERCODE PROJECTCODE` on both ends // — the server compares aToken[2] against its own project-code and ignores // the server code, and a cloning client reads the project-code it must adopt // out of aToken[2] of the server's push reply. // // Both codes are required: canonical only honours the card at nToken==3, so a // two-token "push <code>" is silently ignored by a real Fossil server. Callers // with no cached server code send canonical's own placeholder, "x". func encodePush(w *bytes.Buffer, c *PushCard) error { if c.ProjectCode == "" { panic("xfer: encodePush: ProjectCode must not be empty") } if c.ServerCode == "" { panic("xfer: encodePush: ServerCode must not be empty") } w.WriteString("push ") w.WriteString(c.ServerCode) w.WriteByte(' ') w.WriteString(c.ProjectCode) w.WriteByte('\n') return nil } // encodePull writes "pull <server-code> <project-code>\n". // // Same arg order and same nToken==3 requirement as push; see encodePush. func encodePull(w *bytes.Buffer, c *PullCard) error { if c.ProjectCode == "" { panic("xfer: encodePull: ProjectCode must not be empty") } if c.ServerCode == "" { panic("xfer: encodePull: ServerCode must not be empty") } w.WriteString("pull ") w.WriteString(c.ServerCode) w.WriteByte(' ') w.WriteString(c.ProjectCode) w.WriteByte('\n') return nil } func encodeCookie(w *bytes.Buffer, c *CookieCard) error { w.WriteString("cookie ") w.WriteString(c.Value) w.WriteByte('\n') return nil } func encodeReqConfig(w *bytes.Buffer, c *ReqConfigCard) error { w.WriteString("reqconfig ") w.WriteString(c.Name) w.WriteByte('\n') return nil } func encodePrivate(w *bytes.Buffer) error { w.WriteString("private\n") return nil } func encodeClone(w *bytes.Buffer, c *CloneCard) error { if !c.SeqNoIsDecimal && c.Version == 0 && c.SeqNo == 0 { w.WriteString("clone\n") } else { fmt.Fprintf(w, "clone %d %d\n", c.Version, c.SeqNo) } return nil } func encodeCloneSeqNo(w *bytes.Buffer, c *CloneSeqNoCard) error { fmt.Fprintf(w, "clone_seqno %d\n", c.SeqNo) return nil } func encodeUVGimme(w *bytes.Buffer, c *UVGimmeCard) error { w.WriteString("uvgimme ") w.WriteString(c.Name) w.WriteByte('\n') return nil } func encodePragma(w *bytes.Buffer, c *PragmaCard) error { w.WriteString("pragma ") w.WriteString(c.Name) for _, v := range c.Values { w.WriteByte(' ') w.WriteString(v) } w.WriteByte('\n') return nil } func encodeLogin(w *bytes.Buffer, c *LoginCard) error { w.WriteString("login ") w.WriteString(deck.FossilEncode(c.User)) w.WriteByte(' ') w.WriteString(c.Nonce) w.WriteByte(' ') w.WriteString(c.Signature) w.WriteByte('\n') return nil } // encodeComment writes "# <text>\n". Text must be a single line: an embedded // newline would make the remainder parse as a card of its own. func encodeComment(w *bytes.Buffer, c *CommentCard) error { if strings.ContainsAny(c.Text, "\n\r") { panic("xfer: encodeComment: Text must not contain a newline") } w.WriteString("# ") w.WriteString(c.Text) w.WriteByte('\n') return nil } func encodeError(w *bytes.Buffer, c *ErrorCard) error { w.WriteString("error ") w.WriteString(deck.FossilEncode(c.Message)) w.WriteByte('\n') return nil } func encodeMessage(w *bytes.Buffer, c *MessageCard) error { w.WriteString("message ") w.WriteString(deck.FossilEncode(c.Message)) w.WriteByte('\n') return nil } func encodeUVIGot(w *bytes.Buffer, c *UVIGotCard) error { fmt.Fprintf(w, "uvigot %s %d %s %d\n", c.Name, c.MTime, c.Hash, c.Size) return nil } func encodeUnknown(w *bytes.Buffer, c *UnknownCard) error { w.WriteString(c.Command) for _, a := range c.Args { w.WriteByte(' ') w.WriteString(a) } w.WriteByte('\n') return nil } // --- Payload card encoders --- // encodeFile writes: file UUID SIZE \n CONTENT (no trailing \n) // or: file UUID DELTASRC SIZE \n CONTENT (delta variant) func encodeFile(w *bytes.Buffer, c *FileCard) error { w.WriteString("file ") w.WriteString(c.UUID) if c.DeltaSrc != "" { w.WriteByte(' ') w.WriteString(c.DeltaSrc) } fmt.Fprintf(w, " %d\n", len(c.Content)) w.Write(c.Content) // NO trailing newline return nil } // encodeCFile writes: cfile UUID USIZE CSIZE \n ZCONTENT (no trailing \n) // or: cfile UUID DELTASRC USIZE CSIZE \n ZCONTENT (delta variant) // // The ZCONTENT payload is Fossil's on-disk compressed-blob format, produced by // blob_compress() in src/blob.c: a 4-byte big-endian uncompressed-size prefix // followed by the zlib stream. This framing is mandatory for interop (issue // #152): fossil's send_compressed_file transmits a blob.content column // verbatim, and its receiver stores the cfile payload verbatim into // blob.content, then later reads it back through blob_uncompress(), which reads // that 4-byte prefix to size its output before inflating. A bare zlib payload // (no prefix) is stored unreadable, so a real fossil client cloning from us // ends up with zero usable check-ins. CSIZE therefore counts the prefix. // // The prefix value is the size of the bytes being compressed -- len(Content) -- // which for a delta card is the delta payload, not the full artifact. That is // distinct from USIZE. // // §7.2: USIZE is the sender's full reconstructed artifact size, which for a // delta card is not len(Content) -- Content there is the (smaller) delta // payload. A caller that knows the reconstructed size (the clone send path // does, from blob.size) sets c.USize; callers that don't -- every non-delta // caller, where the two quantities coincide -- leave it zero and get the old // len(Content) behaviour. func encodeCFile(w *bytes.Buffer, c *CFileCard) error { // Fossil on-disk blob format: [4-byte BE uncompressed size][zlib stream]. var payload bytes.Buffer if err := binary.Write(&payload, binary.BigEndian, uint32(len(c.Content))); err != nil { return fmt.Errorf("xfer: cfile size prefix: %w", err) } zw := zlib.NewWriter(&payload) if _, err := zw.Write(c.Content); err != nil { return fmt.Errorf("xfer: cfile zlib write: %w", err) } if err := zw.Close(); err != nil { return fmt.Errorf("xfer: cfile zlib close: %w", err) } usize := c.USize if usize == 0 { usize = len(c.Content) } w.WriteString("cfile ") w.WriteString(c.UUID) if c.DeltaSrc != "" { w.WriteByte(' ') w.WriteString(c.DeltaSrc) } fmt.Fprintf(w, " %d %d\n", usize, payload.Len()) w.Write(payload.Bytes()) // NO trailing newline return nil } // encodeConfig writes: config NAME SIZE \n CONTENT \n // Note the trailing \n after CONTENT. func encodeConfig(w *bytes.Buffer, c *ConfigCard) error { fmt.Fprintf(w, "config %s %d\n", c.Name, len(c.Content)) w.Write(c.Content) w.WriteByte('\n') // trailing newline return nil } // encodeUVFile writes: uvfile NAME MTIME HASH SIZE FLAGS \n CONTENT // When deleted (0x0001) or content-omitted (0x0004), no payload follows. func encodeUVFile(w *bytes.Buffer, c *UVFileCard) error { fmt.Fprintf(w, "uvfile %s %d %s %d %d\n", c.Name, c.MTime, c.Hash, c.Size, c.Flags) if !uvFileOmitsContent(c.Flags) { w.Write(c.Content) } // NO trailing newline return nil } func encodeSchema(w *bytes.Buffer, c *SchemaCard) error { fmt.Fprintf(w, "schema %s %d %s %d %d\n", c.Table, c.Version, c.Hash, c.MTime, len(c.Content)) w.Write(c.Content) w.WriteByte('\n') return nil } func encodeXIGot(w *bytes.Buffer, c *XIGotCard) error { if c.Table == "" { panic("encodeXIGot: Table must not be empty") } if c.PKHash == "" { panic("encodeXIGot: PKHash must not be empty") } fmt.Fprintf(w, "xigot %s %s %d\n", c.Table, c.PKHash, c.MTime) return nil } func encodeXGimme(w *bytes.Buffer, c *XGimmeCard) error { if c.Table == "" { panic("encodeXGimme: Table must not be empty") } if c.PKHash == "" { panic("encodeXGimme: PKHash must not be empty") } fmt.Fprintf(w, "xgimme %s %s\n", c.Table, c.PKHash) return nil } func encodeXRow(w *bytes.Buffer, c *XRowCard) error { if c.Table == "" { panic("encodeXRow: Table must not be empty") } if c.PKHash == "" { panic("encodeXRow: PKHash must not be empty") } fmt.Fprintf(w, "xrow %s %s %d %d\n", c.Table, c.PKHash, c.MTime, len(c.Content)) w.Write(c.Content) w.WriteByte('\n') return nil } func encodeXDelete(w *bytes.Buffer, c *XDeleteCard) error { if c.Table == "" { panic("encodeXDelete: Table must not be empty") } if c.PKHash == "" { panic("encodeXDelete: PKHash must not be empty") } fmt.Fprintf(w, "xdelete %s %s %d %d\n", c.Table, c.PKHash, c.MTime, len(c.PKData)) w.Write(c.PKData) w.WriteByte('\n') return nil }