calibration
Define the Calibration class
Classes
| Name | Description |
|---|---|
| Calibration | A class to handle calibration of FPsim objects. Uses the Optuna hyperparameter |
Calibration
calibration.Calibration(
pars,
calib_pars=None,
weights=None,
fit_exposure_age=True,
fit_exposure_parity=True,
fit_spacing_pref=True,
smoothness_weight=0.5,
exposure_age_bounds=None,
exposure_parity_bounds=None,
burn_in=0,
verbose=True,
keep_db=False,
**kwargs,
)A class to handle calibration of FPsim objects. Uses the Optuna hyperparameter optimization library (optuna.org).
Supports calibration of scalar parameters (e.g. exposure_factor, prob_use_intercept, prob_use_trend_par, fecundity bounds) as well as the exposure_age curve — a 13-knot age-specific exposure multiplier fitted with a smoothness penalty to prevent sharp jumps between adjacent knots.
During calibration, the location’s make_calib_pars() is temporarily neutralized so that the optimizer’s trial values take effect instead of being overridden by the location’s existing calibrated parameters.
Note: running a calibration does not guarantee a good fit! You must ensure that you run for a sufficient number of iterations, have enough free parameters, and that the parameters have wide enough bounds. for more information.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| pars (dict) | simulation parameters, should include ‘location’ and ‘n_agents’ | required | |
| calib_pars (dict) | scalar parameters to calibrate, format dict(key=[best, low, high]) | required | |
| weights (dict) | a custom dictionary of weights for each calibration target | required | |
| fit_exposure_age (bool) | whether to calibrate the exposure_age curve (default: True) | required | |
| smoothness_weight (float) | weight of the smoothness penalty for exposure_age (default: 0.5) | required | |
| exposure_age_bounds (dict) | custom bounds per knot age, format {age: (low, high)} (default: EXPOSURE_AGE_BOUNDS) | required | |
| burn_in (int) | number of years from start_year to ignore when computing fit (default: 0) | required | |
| n_trials (int) | the number of trials per worker (default: 100) | required | |
| n_workers (int) | the number of parallel workers (default: maximum CPUs) | required | |
| total_trials (int) | if n_trials is not supplied, calculate by dividing this by n_workers | required | |
| name (str) | the name of the database (default: ‘fpsim_calibration’) | required | |
| db_name (str) | the name of the database file (default: ‘fpsim_calibration.db’) | required | |
| keep_db (bool) | whether to keep the database after calibration (default: false) | required | |
| storage (str) | the location of the database (default: sqlite) | required | |
| verbose (bool) | whether to print details of the calibration | required | |
| kwargs (dict) | additional Optuna configuration | required |
Returns
| Name | Type | Description |
|---|---|---|
| A Calibration object |
Examples::
# Calibrate scalar params + exposure_age curve
calib_pars = dict(
exposure_factor = [1.0, 0.5, 3.0],
prob_use_intercept = [-1.0, -3.0, 0.0],
prob_use_trend_par = [0.0, -0.15, 0.05],
fecundity_low = [0.7, 0.5, 0.9],
fecundity_high = [1.5, 1.0, 2.5],
)
weights = dict(mcpr=3, asfr=3, total_fertility_rate=3, age_first_stats=4)
calib = fp.Calibration(
pars=dict(location='senegal', n_agents=5000),
calib_pars=calib_pars,
weights=weights,
)
calib.calibrate()
# Scalar-only calibration (no exposure_age curve)
calib = fp.Calibration(pars=pars, calib_pars=calib_pars,
weights=weights, fit_exposure_age=False)
calib.calibrate()Methods
| Name | Description |
|---|---|
| calibrate | Actually perform calibration. Neutralizes the location’s make_calib_pars(), |
| configure_optuna | Update Optuna configuration, if required |
| make_study | Make a study, deleting one if it already exists |
| parse_study | Parse the study into a data frame |
| plot_all | Plot every point: warning, very slow! |
| plot_best | Plot only the points with lowest mismatch |
| plot_stride | Plot a fixed number of points in order across the results |
| plot_trend | Plot the trend in best mismatch over time |
| remove_db | Remove the database file if keep_db is false and the path exists. |
| run_exp | Create and run an experiment |
| run_trial | Define the objective for Optuna. Suggests scalar parameters and optionally |
| run_workers | Run multiple workers in parallel |
| set_optuna_defaults | Create a (mutable) dictionary with default global settings |
| to_json | Convert the data to JSON |
| validate_pars | Ensure parameters are in the correct format. Two formats are permitted: |
| worker | Run a single worker |
calibrate
calibration.Calibration.calibrate(
calib_pars=None,
weights=None,
verbose=None,
**kwargs,
)Actually perform calibration. Neutralizes the location’s make_calib_pars(), runs Optuna optimization, reconstructs the exposure_age array if fitted, and restores the original make_calib_pars() when done.
configure_optuna
calibration.Calibration.configure_optuna(**kwargs)Update Optuna configuration, if required
make_study
calibration.Calibration.make_study()Make a study, deleting one if it already exists
parse_study
calibration.Calibration.parse_study()Parse the study into a data frame
plot_all
calibration.Calibration.plot_all()Plot every point: warning, very slow!
plot_best
calibration.Calibration.plot_best(best_thresh=2)Plot only the points with lowest mismatch
plot_stride
calibration.Calibration.plot_stride(npts=200)Plot a fixed number of points in order across the results
plot_trend
calibration.Calibration.plot_trend(best_thresh=2)Plot the trend in best mismatch over time
remove_db
calibration.Calibration.remove_db()Remove the database file if keep_db is false and the path exists.
run_exp
calibration.Calibration.run_exp(calib_pars, return_exp=False, **kwargs)Create and run an experiment
run_trial
calibration.Calibration.run_trial(trial)Define the objective for Optuna. Suggests scalar parameters and optionally exposure_age knots, exposure_parity knots, and spacing_pref suppression.
run_workers
calibration.Calibration.run_workers()Run multiple workers in parallel
set_optuna_defaults
calibration.Calibration.set_optuna_defaults()Create a (mutable) dictionary with default global settings
to_json
calibration.Calibration.to_json(filename=None)Convert the data to JSON
validate_pars
calibration.Calibration.validate_pars()Ensure parameters are in the correct format. Two formats are permitted: either a dict of arrays or lists in order best-low-high, e.g.::
calib_pars = dict(
exposure_factor = [1.0, 0.5, 1.5],
maternal_mortality_factor = [1, 0.75, 3.0],
)Or the same thing, as a dict of dicts::
calib_pars = dict(
exposure_factor = dict(best=1.0, low=0.5, high=1.5),
maternal_mortality_factor = dict(best=1, low=0.75, high=3.0),
)worker
calibration.Calibration.worker()Run a single worker