Jev Is a Reward Model Sold as Product

Di Zhang, a PhD candidate at Fudan University working on LLM reasoning, published a September 21 breakdown that recasts TypeSafe’s Jev. He argues it is not a chat model at all, but a reward model turned into the interface.

That inversion explains the name RLCD.

Zhang defines it simply as multiway preference modeling plus probability calibration. More formally, he calls RLCD a schema-conditioned Plackett-Luce objective, with Jev adding typed outputs and parallel inference on top. The reward model no longer hides behind a generator. It becomes the model.

The lineage starts with scalar rewards. A conventional outcome or process reward model scores a context x and candidate a as r(x,a) in real numbers. Zhang notes a 0.8 means little across problems or checkpoints. It was always relative, useful only as r(x,a1) greater than r(x,a2).

LLaMA-Berry made that explicit. Its Pairwise Preference Reward Model, or PPRM, asks is a1 better than a2 and models it as Bradley-Terry, P(a1 over a2 | x) = exp u(x,a1) / (exp u(x,a1)+exp u(x,a2)), or sigma(u1 minus u2). It was trained on almost 7.8 million mathematical solution pairs and optimized with DPO, using Enhanced Borda Count downstream for MCTS search. The conceptual move matters more than the search. Reward modeling becomes preference probability.

Plackett-Luce generalizes that to K candidates. Assign each a utility u_i = u(x,a_i), then P(a_i | x,A) = exp u_i / sum exp u_j. For K equals 2 this collapses to Bradley-Terry, so PPRM is the binary case. With a full ranking, the likelihood factorizes as product over t of exp u_pi_t / sum_j>=t exp u_pi_j. With a single correct choice y, the loss is minus log exp u_y / sum exp u_j. Zhang calls this the mathematical center of RLCD.

Normalization still is not calibration. A softmax sums to one, but a reported 0.8 should be right 80 percent of the time, or P(Y equals hat Y | hat P equals p) approx p. TypeSafe’s own primer promises that contract for Jev, and Zhang treats calibration as a second objective that RLHF often ignores. A model can stay systematically overconfident under RLHF and still score well because the reward never prices confidence directly.

His calibrator is the Brier score. For binary Noul, BS(p,y) = (p minus y) squared, so a 0.8 that is wrong costs 0.64 versus 0.04 when right, sixteen times more. For K-way Choice it extends to one half sum (p_i minus indicator[i equals y]) squared. Top-1 accuracy cannot separate a 0.55 from a 0.99 correct pick. Brier can. He cites the Murphy decomposition, BS equals REL minus RES plus UNC, where reliability measures calibration, resolution measures separation of easy and hard cases, and uncertainty is dataset base rate. Training can minimize log loss, then tune temperature T on validation Brier, p_i = exp(u_i / T) / sum exp(u_j / T), to adjust sharpness without changing order.

Jev then exposes that distribution directly. Instead of prompt to generator to candidate to reward model, the path is state plus candidate schema to calibrated decision distribution. No explanation has to be generated and parsed. Three primitives share the same object P(typed outcome | state, question, candidate set). Noul is a binary Bradley-Terry true or false, Choice is a Luce distribution over K unordered alternatives, Score is a distribution over ordered levels. Each returns the pick plus the full vector and confidence.

The utility comes from a decision head. Zhang details the released Jevre design. The encoder runs over state, question and every candidate under a tree mask, mean-pools normalized hidden states bar h_S, bar h_Qf, bar h_Cf,i, then forms q_f = W_q bar h_S + W_q bar h_Qf and k_f,i = W_k bar h_Cf,i. Utility is scaled dot product u_f,i = q_f transpose k_f,i / sqrt(r) with r equals 512. Softmax within the question yields p_f,i. Candidate names arrive as text at runtime, so the same head scores new schemas without adding classifiers. Images flow through bar h_S, the head stays unchanged.

Parallelism follows from layout, not magic. Autoregressive models factor P(y | x) as product P(y_t | x, y less than t). Jev only needs utilities and normalization. It packs Z = [S; Q1; C1,1 ... C1,K1; Q2 ...] and enforces M_tree_ij equals 0 if v(j) is ancestor of v(i), else minus infinity. A question sees state and itself. A candidate sees state, its question and itself, never siblings. One pass through a shared head plus typed schema decoding returns all decisions.

The limitation is explicit. Ranking picks the best candidate, calibration decides whether software should execute, defer or escalate. One without the other leaves automation stuck. Zhang says RLCD separates them, and the product bet is that software wants probabilities it can act on, not sentences it must interpret.