`silvic-core`

silvic-core provides silvic’s parameter handling, binning, objectives, metrics, split search and LightGBM 3.3.5 model text. It grows trees leaf by leaf using quantized integer histograms. Training uses the Backend interface with this crate’s CpuBackend, the Metal and CUDA backends of silvic-gpu, or the AWS Neuron backend of silvic-neuron. Histograms reduce across ranks through a Collective, using LocalCollective in one process or the TCP ring of silvic-net across processes.

let params = Params::from_json(r#"{"objective": "lambdarank", "num_iterations": 100}"#)?;
let train_set = BinnedDataset::construct(&input, &params, &mut LocalCollective)?;
let model = train(&params, &train_set, &[], &mut CpuBackend::new(), &mut LocalCollective, &mut |_| Ok(()))?;
std::fs::write("model.txt", model.to_lightgbm_string())?;

Public API

Params::from_json(text): LightGBM 3.3.5 parameters by name and alias, from a JSON object of numbers, strings, booleans and lists.
BinnedDataset::construct(input, params, collective): Bins one rank’s rows: a DenseMatrix (f32 or f64, row- or column-major, borrowed) with labels, weights, query sizes and categorical columns. Bin mappers come from a sample allgathered over every rank, so every rank bins alike; invalid input on any rank fails every rank with the same error.
BinnedDataset::construct_with_reference(input, reference, collective): Bins a validation set with the training set’s mappers.
train(params, train_set, valid, backend, collective, on_iteration): Grows num_iterations trees and returns the Model; every rank returns the same one. on_iteration sees each iteration’s evaluations, and report.model() the model so far, to checkpoint; an error from it stops training.
train_from(params, init_model, train_set, valid, backend, collective, on_iteration): Adds num_iterations trees to init_model. Its trees are replayed onto the scores in f32 as training added them, and sampling is keyed by the absolute iteration, so resuming a checkpoint on the same rows and bins gives the uninterrupted model, byte for byte. A model binned differently, LightGBM’s included, continues approximately.
model.to_lightgbm_string(), Model::from_lightgbm_string(text): LightGBM 3.3.5 model text, written byte for byte as LightGBM writes it; LightGBM 3.x and 4.x models read back to the same bytes.
model.predict(matrix, raw_score, num_trees): LightGBM’s predictions bit for bit, NaN, zeros and unseen categories included, in parallel over rows.
CpuBackend::new(), LocalCollective: The CPU backend, parallel over rayon’s pool, and the collectives of a single process.
trait Backend, trait Collective: What silvic-gpu and silvic-net implement.

Also public SilvicError, Result, model.feature_importance(kind), and the modules config, data, objective, metric, quantize, tree and backend; test_util behind the test-util feature.

Parameters

LightGBM 3.3.5’s names and their aliases, scikit-learn’s included: n_estimators, min_child_weight, reg_alpha, subsample, colsample_bytree and the rest.

objective regression: regression (L2), binary with sigmoid, or lambdarank with label_gain, lambdarank_truncation_level and lambdarank_norm.
Trees: num_leaves, max_depth, min_data_in_leaf, min_sum_hessian_in_leaf, lambda_l1, lambda_l2, min_gain_to_split, learning_rate.
Sampling: bagging_fraction, bagging_freq, feature_fraction, their seeds, and seed, which draws the others as LightGBM does.
Bins and categoricals: max_bin (at most 255), min_data_in_bin, bin_construct_sample_cnt, max_cat_to_onehot, max_cat_threshold, cat_l2, cat_smooth, min_data_per_group.
Metrics: metric: l2, rmse, binary_logloss, auc, ndcg with eval_at; the objective’s own by default.
use_quantized_grad true: Trains on integer histograms of gradients rounded to num_grad_quant_bins (4) levels, with stochastic_rounding. LightGBM defaults to false.
quant_train_renew_leaf true: Leaf values from exact sums of the float gradients. LightGBM defaults to false.
device_type auto: Read by the Python package, which builds the backend; gpu_device_id and num_threads as in LightGBM.

An unknown key is an error, not a warning, and so is any of LightGBM’s features outside silvic set to other than its default: is_unbalance, scale_pos_weight, early_stopping_round, monotone_constraints, linear_tree, extra_trees, path_smooth, max_delta_step, zero_as_missing, num_class, and boosting other than gbdt. Keys that only tune how LightGBM runs, such as force_row_wise and deterministic, are accepted and change nothing.

Determinism

Each row’s gradient and hessian round to small integers by stochastic rounding, and bagging draws its rows, from a counter-based hash of the seed, the iteration and the row’s global index. The same row rounds the same way on every backend and on whichever rank holds it. Histograms are sums of those integers, exact in any order, so allreducing them across ranks is exact, and leaf values come from fixed-point sums of the float gradients, exact as well. The same data and parameters give the same model on every run, and on any number of ranks when bin_construct_sample_cnt covers every row, so all ranks bin from the same sample.

Against LightGBM

Bins: LightGBM 3.3.5’s FindBin: for the same sample, every threshold LightGBM splits on is a silvic bin boundary, bit for bit. feature_pre_filter is not applied.
Models and metrics: Model text byte for byte LightGBM 3.3.5’s; metrics equal LightGBM 4.7’s to 1e-12.
Training: Quantized by default, with silvic’s own random draws and f32 training scores: models of the same kind and accuracy as LightGBM’s, not the same trees.
Zero hessians: A leaf without hessian is never split and an empty child is rejected, so quantized lambdarank trains on queries without a relevant item, where LightGBM 4.7 aborts.
Queries: Any size: LightGBM’s 10000-position discount table is extended by its formula.