Skip to main content

Failure model

OpenDC can be used to simulate failures. Host failure is an important aspect of data center operations. A failure model decides when hosts fail, how long a failure lasts, and how many hosts it affects. At the moment, OpenDC only models start-stop failures. When a host fails, all its tasks are interrupted and the host can not be used until the failure ends. A task that has failed maxNumFailures times (see Experiment) is terminated.

There are four types of failure models selected by the type parameter:

TypeDescription
noneNo failures. This is the default.
prefabA built-in model based on measured failures.
customSamples each failure from three distributions.
traceReplays the failures in a failure trace.

Custom models​

A custom model samples every failure from three distributions:

FieldDescription
interArrivalThe time between two failures, in hours.
durationHow long a failure lasts, in hours.
hostFractionThe fraction of hosts that fail, clamped to between 0 and 1.

Each field is one of the following distributions:

TypeFields
constantvalue
exponentialmean
gammashape, scale
log-normalscale, shape
normalmean, std
paretoscale, shape
uniformlower, upper
weibullalpha, beta

Custom models are random, so their results depend on the seed of the run.

{
"type": "custom",
"interArrival": { "type": "exponential", "mean": 24.0 },
"duration": { "type": "log-normal", "scale": 1.0, "shape": 0.5 },
"hostFraction": { "type": "uniform", "lower": 0.1, "upper": 0.5 }
}

Trace models​

FieldTypeDefaultDescription
sourcetrace referencerequiredThe failure trace.
startPointnumber0.0Where in the trace to start, as a fraction between 0 and 1. The failures before it are replayed at the end.
repeatbooleantrueStart again from the beginning when the trace ends.

A failure trace is a Parquet file with one row per failure:

ColumnTypeDescription
failure_intervalint64The time since the previous failure, in milliseconds.
failure_durationint64How long the failure lasts, in milliseconds.
failure_intensitydoubleThe fraction of hosts that fail.
{
"type": "trace",
"source": { "type": "named", "name": "failure_traces/my_failures.parquet" }
}

Prefab models​

{ "type": "prefab", "prefabName": "G5k06Exp" }

The prefabs are fitted to the traces of The Failure Trace Archive. The name combines a trace with a distribution:

  • Traces: G5k06, Lanl05, Ldns04, Microsoft99, Nd07cpu, Overnet03, Pl05, Skype06, Websites02
  • Distributions: Exp (exponential), Wbl (Weibull), LogN (log-normal), Gam (gamma)

For example, Lanl05Wbl uses a Weibull distribution fitted to the LANL 2005 trace. Every prefab affects a uniformly random fraction of the hosts.

The table below lists the parameters of each prefab. The availability intervals give the time between two failures, and the unavailability intervals the duration of a failure, both in hours.

The parameters of the prefab failure models