Hyperparameters

Training adjusts the weights of the classifier, but some settings are fixed before training begins and shape how it proceeds. These are the hyperparameters: the number of weights, how features are scaled, and how fast and how strongly the optimiser adjusts the weights.

The hyperparameters are part of the model variant selected in model on the SpamClassifier singleton, found in the WebUI under Settings › Spam Filter › Classifier. The defaults suit most servers. The one setting with a practical trade-off is the size of the model.

A change has a cost. Changing a hyperparameter changes the meaning of the weights already learned, so the existing weights no longer fit the new model. After a change, train the model again from the retained samples, with a SpamFilterMaintenance task whose maintenanceType is retrain. Samples are kept for holdSamplesFor, 180 days by default, so the retrained model learns from up to that much history (Training).

Model size

numFeatures in parameters sets the number of weights of the model, as a power of two. The range runs from 16 (65536 weights) to 28 (about 268 million weights). The default is 20, about one million weights.

The size is a trade-off between accuracy and resources. Every feature is hashed to one of these weights, so a larger model makes it less likely that two unrelated features share a weight. The price is memory on every node and space in the blob store, both of which double with each step of numFeatures.

Feature scaling

Raw feature counts would let the length of a message and the repetition of a word distort the result. Before a message is classified, the counts of its features are therefore scaled in two steps.

The first step, featureLogScale, replaces each count with its logarithm (log(1 + count)), so that a word repeated many times does not dominate the message. It is enabled by default.

The second step, featureL2Normalize, scales the feature vector of the message to unit length, so that long and short messages are classified on the same scale. It is enabled by default.

Optimiser

The optimiser is the algorithm that adjusts the weights after each sample. The FTRL-Proximal optimiser has four parameters, also in parameters:

Field	Effect
alpha	Base learning rate: how far each sample moves the weights.
beta	Smoothing of the per-feature learning rates in the early stage of learning.
l1Ratio	L1 regularisation: how strongly weights of rarely useful features are driven to zero.
l2Ratio	L2 regularisation: how strongly large weights are reduced.

These parameters interact, and their effect on accuracy can only be judged by measuring the classifier on the server’s own mail. Change them only together with such a measurement.

Cuckoo feature hashing

Cuckoo feature hashing gives each feature several candidate weights, and a second, smaller model learns which candidate each feature uses. The FtrlCcfh variant therefore has two sets of the same fields: parameters for the main model, and indicatorParameters for the smaller one. The default size of the smaller model is smaller than that of the main model.

The following object body selects cuckoo feature hashing with a larger main model:

{
  "model": {
    "@type": "FtrlCcfh",
    "parameters": {"numFeatures": "22"},
    "indicatorParameters": {"numFeatures": "18"}
  }
}