Chapter 7. Event-Based Simulation with OESjs
The JavaScript-based simulation framework OESjs Core1 implements the Object Event Simulation (OES) paradigm, representing a general Discrete Event Simulation approach based on object-oriented modeling and event scheduling.
The code of an OESjs Core1 simulation consists of (1) the OESjs Core1 framework files in the folder framework, (2) general library files in the lib folder and (3) the following files to be created by the simulation developer:
- For each object type ObjT, a JS code file ObjT.js.
- For each event type EvtT, a JS code file EvtT.js.
- A simulation.js file defining further parts of the simulation, such as statistics variables and the initial state.
OESjs Core1 supports three forms of simulations:
Standalone scenario simulations, which are good for getting a quick impression of a simulation model, e.g., by checking some simple statistics.
Simple simulation experiments, which are defined as a set of replicated simulation scenario runs, providing summary statistics like mean, standard deviation, minimum/maximum and confidence intervals for each statistics variable defined in the underlying model.
Parameter variation experiments, for which a set of experiment parameters with value sets are defined such that each experiment parameter corresponds to a model parameter. When an experiment is run, each experiment parameter value combination defines an experiment scenario, which is run repeatedly, according to the specified number or replications for collecting statistics.
OESjs Core1 allows to define two or more simulation scenarios for a given model. While an experiment type is defined for a given model, an experiment of that type is run on top of a specific scenario.
Using a simulation library like OESjs Core1 means that only the model-specific logic has to be coded (in the form of object types, event types, event routines and other functions for model-specific computations), but not the general simulator operations (e.g., time progression and statistics) and the environment handling (e.g., user interfaces for statistics output).
The following sections present the basic concepts of the OESjs Core1 simulation library.
You can download OESjs Core1 in the form of a ZIP archive file from the OES GitHub repo. After extracting the archive on your local disk, you can run any of its example models by going to its folder and loading its index.html file into your browser. You can create your own model by making a copy of one of the example model folders and using its code files a s starting point.
Since an OESjs simulation includes a JS worker file for running the simulator in its own thread separately from the main (user interface) thread, it cannot be run from the local file system without changing the browser's default configuration (due to the web security policy CORS).
For developing OESjs simulations on your computer, you should use Firefox because its security settings can be easily configured such that it allows loading JS worker files directly from the local file system by disabling the flag "strict_origin_policy" specifically for file URLs:
Enter "about:config" in the Firefox search bar.
Search for "security.fileuri.strict_origin_policy".
Disable this policy by changing its value from true to false.
This creates only a small security risk because the web security policy called "CORS" is only disabled for file URLs, but not for normal URLs.
For other browsers, like Chrome, you need to install a local HTTP server and load your simulation's index.html file from that local server, or run it via the JS development tool WebStorm (which has a built-in local server), because the only option for loading JS worker files from the local file system in Chrome would be to disable the CORS policy completely (see how to disable CORS in Chrome), but that would create a more severe security risk and is therefore not recommended.
7.1. Simulation Time
A simulation model has an underlying time model, which can be either discrete time, when setting
sim.model.time = "discrete";
or continuous time, when setting
sim.model.time = "continuous";
Choosing a discrete time model means that time is measured in steps (with equal durations), and all temporal random variables used in the model need to be discrete (i.e., based on discrete probability distributions). Choosing a continuous time model means that one has to define a simulation time granularity, as explained in the next sub-section.
In both cases, the underlying simulation time unit can be either left unspecified (e.g., in the case of an abstract time model), or it can be set to one of the time units "ms", "s", "min", "hour", "day", "week", "month" or "year", as in
sim.model.timeUnit = "hour";
Typical examples of time models are:
An abstract discrete model of time where time runs in steps without any concrete meaning:
sim.model.time = "discrete";
A concrete discrete model of time in number of days:
sim.model.time = "discrete"; sim.model.timeUnit = "day";
A concrete continuous model of time in number of seconds:
sim.model.time = "continuous"; sim.model.timeUnit = "s";
2.1.1. Time Granularity
A model's time
granularity is the time delay until the next moment, such that the
model does not allow considering an earlier next moment. This is captured by
the simulation parameter nextMomentDeltaT used by the simulator for
scheduling immediate events with a minimal delay. When a simulation model is
based on discrete time, nextMomentDeltaT is set to 1, referring to
the next time point. When a simulation model is based on continuous time,
nextMomentDeltaT is set to the default value 0.001, unless the model
parameter sim.model.nextMomentDeltaT
is explicitly assigned in
the simulation.js file.
2.1.2. Time Progression
An important issue in simulation is the question how the simulation time is advanced by the simulator. The OES paradigm supports next-event time progression and fixed-increment time progression, as well as their combination.
An OESjs-Core1 model with fixed-increment time
progression has to define a suitable periodic time event type, like
EachSecond
or EachDay
in the form of an exogenous
event type with a recurrence function returning the value 1. Such a model
can be used for
- modeling continuous state changes (e.g., objects moving in a continuous space), or
- making a discrete model that abstracts away from explicit events and uses only implicit periodic time events ("ticks"), which is a popular approach in social science simulation.
Examples of discrete event simulation models with fixed-increment time progression and no explicit events are the Schelling Segregation Model and the Susceptible-Infected-Recovered (SIR) Disease Model.
7.2. Simulation Models
2.2.1. Model Variables and Functions
In the simple model of a service desk discussed in the previous section, we define one (global) model variable, queueLength, one model function, serviceTime(), and two event types, as shown in the following class diagram:
Notice that this model does not define any object
type, which implies that the system state is not composed of the states of
objects, but of the states of model variables, here it consists of the state
of the model variable queueLength. The discrete random variable for
modeling the random variation of service durations is implemented as a model
function serviceTime
shown in the Global Variables and
Functions class. It samples integers between 2 and 4 from the empirical
probability distribution Frequency{ 2:0.3, 3:0.5, 4:0.2}. The model
can be coded with OESjs-Core1 in the following way:
// (global) model variable sim.model.v.queueLength = 0; // (global) model function sim.model.f.serviceTime = function () { var r = math.getUniformRandomInteger( 0, 99); if ( r < 30) return 2; // probability 0.30 else if ( r < 80) return 3; // probability 0.50 else return 4; // probability 0.20 };
You can run this Service-Desk-0 model from the project's GitHub website. An example of a run of this model is shown in the following simulation log:
Step | Time | System State | Future Events |
---|---|---|---|
0 | 0 | queueLength: 0 | CustomerArrival@1 |
1 | 1 | queueLength: 1 | CustomerDeparture@4, CustomerArrival@4 |
2 | 4 | queueLength: 1 | CustomerDeparture@6, CustomerArrival@7 |
3 | 6 | queueLength: 0 | CustomerArrival@7 |
4 | 7 | queueLength: 1 | CustomerDeparture@11, CustomerArrival@13 |
5 | 11 | queueLength: 0 | CustomerArrival@13 |
6 | 13 | queueLength: 1 | CustomerDeparture@15, CustomerArrival@19 |
7 | 15 | queueLength: 0 | CustomerArrival@19 |
... | ... | ... | ... |
49 | 114 | queueLength: 0 | CustomerArrival@117 |
50 | 117 | queueLength: 1 | CustomerArrival@118, CustomerDeparture@119 |
51 | 118 | queueLength: 2 | CustomerDeparture@119, CustomerArrival@119 |
52 | 119 | queueLength: 2 | CustomerArrival@121, CustomerDeparture@123 |
53 | 121 | queueLength: 3 | CustomerDeparture@123, CustomerArrival@124 |
54 | 123 | queueLength: 2 | CustomerArrival@124, CustomerDeparture@126 |
55 | 124 | queueLength: 3 | CustomerArrival@125, CustomerDeparture@126 |
56 | 125 | queueLength: 4 | CustomerDeparture@126, CustomerArrival@128 |
57 | 126 | queueLength: 3 | CustomerArrival@128, CustomerDeparture@128 |
58 | 128 | queueLength: 3 | CustomerArrival@129, CustomerDeparture@131 |
59 | 129 | queueLength: 4 | CustomerDeparture@131, CustomerArrival@133 |
60 | 131 | queueLength: 3 | CustomerArrival@133, CustomerDeparture@135 |
61 | 133 | queueLength: 4 | CustomerDeparture@135, CustomerArrival@137 |
62 | 135 | queueLength: 3 | CustomerArrival@137, CustomerDeparture@137 |
63 | 137 | queueLength: 3 | CustomerArrival@139, CustomerDeparture@141 |
64 | 139 | queueLength: 4 | CustomerDeparture@141, CustomerArrival@142 |
65 | 141 | queueLength: 3 | CustomerArrival@142, CustomerDeparture@144 |
66 | 142 | queueLength: 4 | CustomerDeparture@144, CustomerArrival@147 |
67 | 144 | queueLength: 3 | CustomerArrival@147, CustomerDeparture@148 |
68 | 147 | queueLength: 4 | CustomerDeparture@148, CustomerArrival@148 |
69 | 148 | queueLength: 4 | CustomerArrival@149, CustomerDeparture@151 |
70 | 149 | queueLength: 5 | CustomerDeparture@151, CustomerArrival@151 |
... | ... | ... | ... |
2.2.2. Object Types
Object types are defined in the form of classes. Consider the object type ServiceDesk defined in the following Service-Desk-1 model:
While
queueLength
was defined as a global variable in the
Service-Desk-0 model, it is now defined as an attribute of the object
type ServiceDesk:
class ServiceDesk extends oBJECT { constructor({ id, name, queueLength}) { super( id, name); this.queueLength = queueLength; } static serviceTime() { var r = math.getUniformRandomInteger( 0, 99); if ( r < 30) return 2; // probability 0.3 else if ( r < 80) return 3; // probability 0.5 else return 4; // probability 0.2 } } ServiceDesk.labels = {"queueLength":"qLen"}; // for the log
Notice
that, in OESjs, object types are defined as subtypes of the pre-defined
class oBJECT
, from which they inherit an integer-valued
id
attribute and an optional name
attribute. When
a property has a label
(defined by the class-level (map-valued)
property labels
), it is shown in the simulation log.
You can run this simulation model from the project's GitHub website.
2.2.3. Event Types
In OES, there is a distinction between two kinds of events:
- events that are caused by other event occurrences during a simulation run;
- exogenous events that seem to happen spontaneously, but may be caused by factors, which are external to the simulation model.
Here is an example of an exogenous event type definition in OESjs-Core1:
class CustomerArrival extends eVENT { constructor({ occTime, serviceDesk}) { super( occTime); this.serviceDesk = serviceDesk; } onEvent() { ... } ... }
The definition of the CustomerArrival event type includes a
reference property serviceDesk, which is used for referencing the
service desk object at which a customer arrival event occurs. In OESjs,
event types are defined as subtypes of the pre-defined class
eVENT
, from which they inherit an attribute
occTime
, which holds the occurrence time of an event. As opposed to
objects, events do normally not have an ID, nor a name.
Each event
type needs to define an onEvent
method that implements the
event rule for events of the defined type. Event rules are discussed below.
Exogenous events occur periodically. They are therefore defined with a recurrence function, which provides the time in-between two events (often in the form of a random variable). The recurrence function is defined as a class-level ("static") method:
class CustomerArrival extends eVENT { ... static recurrence() { return math.getUniformRandomInteger( 1, 6); } }
Notice that the recurrence function of CustomerArrival
is coded with the library method math.getUniformRandomInteger
, which allows sampling from
discrete uniform probability distribution functions.
In the case of an exogenous event type definition, a createNextEvent method has to be defined for assigning event properties and returning the next event of that type, which is scheduled by invoking the recurrence function for setting its ocurrenceTime and by copying all participant references (such as the serviceDesk reference).
class CustomerArrival extends eVENT { ... createNextEvent() { return new CustomerArrival({ occTime: this.occTime + CustomerArrival.recurrence(), serviceDesk: this.serviceDesk }); } static recurrence() {...} }
When an OE simulator processes an exogenous event e of type E, it automatically schedules the next event of type E by invoking the createNextEvent method on e, if it is defined, or, otherwise by duplicating e and resetting its occurrence time by invoking E.recurrence().
For an exogenous event type, it is an
option to define a maximum number of event occurrences by setting the static
attribute maxNmrOfEvents
, as in the following example:
CustomerArrival.maxNmrOfEvents = 3;
The second event type of the Service-Desk-1 model, Departure, is an example of a type of caused events:
class CustomerDeparture extends eVENT { constructor({ occTime, serviceDesk}) { super( occTime); this.serviceDesk = serviceDesk; } onEvent() { ... } }
A caused event type does neither define a recurrence function nor a createNextEvent method.
2.2.4. Event Rules
An event rule for an
event type defines what happens when an event of that type occurs, by
specifying the caused state changes and follow-up events. In OESjs, event
rules are coded as onEvent
methods of the class that implements
the event type. These methods return a set of events (more precisely, a set
of JS objects representing events).
Notice that in the DES literature, event rule methods are called event routines.
For instance, in the
CustomerArrival
class, the following event rule method is
defined:
class CustomerArrival extends eVENT { ... onEvent() { var followupEvents=[]; // increment queue length due to newly arrived customer this.serviceDesk.queueLength++; // update statistics sim.stat.arrivedCustomers++; if (this.serviceDesk.queueLength > sim.stat.maxQueueLength) { sim.stat.maxQueueLength = this.serviceDesk.queueLength; } // if the service desk is not busy if (this.serviceDesk.queueLength === 1) { followupEvents.push( new CustomerDeparture({ occTime: this.occTime + ServiceDesk.serviceTime(), serviceDesk: this.serviceDesk })); } return followupEvents; } }
The context of this event rule method is the event that triggers the
rule, that is, the variable this
references a JS object that
represents the triggering event. Thus, the expression
this.serviceDesk
refers to the service desk object associated
with the current customer arrival event, and the statement
this.serviceDesk.queueLength++
increments the
queueLength attribute of this service desk object (as an immediate
state change).
The following event rule method is defined in the
CustomerDeparture
class.
class CustomerDeparture extends eVENT { ... onEvent() { var followupEvents=[]; // decrement queue length due to departure this.serviceDesk.queueLength--; // update statistics sim.stat.departedCustomers++; // if there are still customers waiting if (this.serviceDesk.queueLength > 0) { // start next service and schedule its end/departure followupEvents.push( new CustomerDeparture({ occTime: this.occTime + ServiceDesk.serviceTime(), serviceDesk: this.serviceDesk })); } return followupEvents; } }
2.2.5. Event Priorities
An OES model may imply the possibility of several events occurring at the same time. Consequently, a simulator (like OESjs) must be able to process simultaneous events. In particular, simulation models based on discrete time may create simulation states where two or more events occur at the same time, but the model's logic requires them to be processed in a certain order. Defining priorities for events of a certain type helps to control the processing order of simultaneous events.
Consider an example model based on discrete time with three exogenous event types StartOfMonth, EachDay and EndOfMonth, where the recurrence of StartOfMonth and EndOfMonth is 21, and the recurrence of EachDay is 1. In this example we want to control that on simulation time 1 + i * 21 both a StartOfMonth and an EachDay event occur simultaneously, but StartOfMonth should be processed before EachDay, and on simulation time 21 + i * 21 both an EndOfMonth and an EachDay event occur simultaneously, but EndOfMonth should be processed after EachDay. This can be achieved by defining a high priority, say 2, to StartOfMonth, a middle priority, say 1, to StartOfMonth, and a low priority, say 0, to EndOfMonth.
Event priorities are defined as class-level properties of event classes in the event type definition file. Thus, we would define in StartOfMonth.js:
StartOfMonth.priority = 2;
and in EachDay.js:
EachDay.priority = 1;
and finally in EndOfMonth.js:
EndOfMonth.priority = 0;
2.2.6. Library Methods for Sampling Probability Distribution Functions
Random variables are implemented as methods that sample specific probability distribution functions (PDFs). Simulation frameworks typically provide a library of predefined parametrized PDF sampling methods, which can be used with one or several (possibly seeded) streams of pseudo-random numbers.
The OESjs simulator provides the following predefined parametrized PDF sampling methods:
Probability Distribution Function | OESjs Library Method | Example |
Uniform | uniform ( lowerBound, upperBound) |
rand.uniform( 0.5, 1.5) |
Discrete Uniform | uniformInt ( lowerBound,
upperBound) |
rand.uniformInt( 1, 6) |
Triangular | triangular ( lowerBound, upperBound,
mode) |
rand.triangular( 0.5, 1.5, 1.0) |
Frequency | frequency ( frequencyMap) |
rand.frequency({"2":0.4, "3":0.6}) |
Exponential | exponential ( eventRate) |
rand.exponential( 0.5) |
Gamma | gamma ( shape, scale) |
rand.gamma( 1.0, 2.0) |
Normal | normal ( mean, stdDev) |
rand.normal( 1.5, 0.5) |
Pareto | pareto ( shape) |
rand.pareto( 2.0) |
Weibull | weibull ( scale, shape) |
rand.weibull( 1, 0.5) |
The OESjs library rand.js supports both unseeded and
seeded random number streams. By default, its PDF sampling methods are based on an unseeded
stream, using Marsaglia’s high-performance random number generator xorshift that is built into the
Math.random
function of modern JavaScript engines.
A
seeded random number stream, based on David Bau's seedable random number
generator seedrandom,
can be obtained by setting the scenario parameter
sim.scenario.randomSeed
to a positive integer
value.
Additional streams can be defined and used in the following way:
var stream1 = new Random( 1234); var stream2 = new Random( 6789); var service1Duration = stream1.exponential( 0.5); var service2Duration = stream2.exponential( 1.5);
Avoid using JavaScript's built-in
Math.random
in simulation code. Always userand.uniform
, or one of the other sampling functions from the rand.js library described above, for generating random numbers.Otherwise, using a random seed does not guarantee reproducible simulation runs!
7.3. Simulation Scenarios
For obtaining a complete executable simulation scenario, a simulation model has to be complemented with simulation parameter settings and an initial system state.
In general, we may have more than one simulation scenario for a simulation model. For instance, the same model could be used in two different scenarios with different initial states.
An OESjs simulation scenario consists of
- a simulation model;
- simulation parameter settings, such as setting a value for
durationInSimTime
andrandomSeed
; and - an initial state definition.
An empty template for the simulation.js file has the following structure:
// ***** Simulation Model ******************* sim.model.time = "..."; // discrete or continuous sim.model.timeIncrement = ...; // optional sim.model.timeUnit = "..."; // optional (ms|s|min|hour|day|week|month|year) sim.model.v.aModelVariable = ...; // (developer-defined) model variables sim.model.f.aModelFunction = ...; // (developer-defined) model functions sim.model.p.aModelParameter = ...; // (developer-defined) model parameters sim.model.objectTypes = [...]; // (developer-defined) object types sim.model.eventTypes = [...]; // (developer-defined) event types // ***** Simulation Parameters ************** sim.scenario.durationInSimTime = ...; sim.scenario.randomSeed = ...; // optional // ***** Initial State ********************** sim.scenario.setupInitialState = function () { // Initialize model variables ... // Create initial objects ... // Schedule initial events ... }; // ***** Ex-Post Statistics ***************** sim.model.statistics = {...};
We briefly discuss each group of scenario information items in the following sub-sections.
2.3.1. Model Parameters
While model variables are state variables whose values are changed as an effect of an event occurrence, model parameters are not part of the dynamic state of the simulated system, but are rather used for providing values that can only be read during a simulation run. The main purpose of model parameters is to allow parameter variation experiments.
2.3.2. Simulation Scenario Parameters
A few simulation parameters are predefined as attributes of the simulation scenario. The most important ones are:
- durationInSimTime - this attribute allows defining the duration of a simulation run; which runs forever when this attribute s not set;
- randomSeed: Setting this optional parameter to a positive integer allows to obtain a specific fixed random number sequence (generated by a random number generator). This can be used for performing simulation runs with the same (repeated) random number sequence, e.g., for testing a simulation model by checking if expected results are obtained.
2.3.3. Initial State
Defining an initial state means:
- assigning initial values to global model variables, if there are any;
- defining which objects exist initially, and assigning initial values to their properties;
- defining which events are scheduled initially.
A setupInitialState
procedure takes care of these
initial state definitions. A global model variable is initialized in the
following way:
sim.scenario.setupInitialState = function () { // Initialize model variables sim.model.v.queueLength = 0; // Create initial objects ... // Schedule initial events ... };
An initial state object is created by instantiating an object type of the simulation model with suitable initial property values, as shown in the following example:
sim.scenario.setupInitialState = function () { // Initialize model variables ... // Create initial objects const serviceDesk1 = new ServiceDesk({id: 1, queueLength: 0}); // Schedule initial events ... };
Notice that object IDs are positive integers.
Instead of
assigning a fixed value to a property like
queueLength
for defining an object's initial state, as in
queueLength: 0
, we can also assign it an
expression, as in queueLength:
Math.round(12/30)
.
An initial event is scheduled (or added to the Future Events List), as shown in the following example:
sim.scenario.setupInitialState = function () { // Initialize model variables ... // Create initial objects const desk1 = new ServiceDesk({id: 1, queueLength: 0}); // Schedule initial events sim.schedule( new CustomerArrival({occTime:1, serviceDesk: desk1})); };
Initial objects or events can be parametrized with the help of model parameters.
2.3.4. Defining Alternative Scenarios with Different Initial States
For running a model on top of
different initial states, one can define a list of scenarios, each with its
own setupInitialState
procedure:
sim.scenarios[1] = { scenarioNo: 1, title: "Scenario with two service desks", setupInitialState: function () { // Create initial objects var sD1 = new ServiceDesk({id: 1, queueLength: 0}), sD2 = new ServiceDesk({id: 2, queueLength: 0}); // Schedule initial events sim.FEL.add( new CustomerArrival({occTime: 1, serviceDesk: sD1})); sim.FEL.add( new CustomerArrival({occTime: 2, serviceDesk: sD2})); } }; sim.scenarios[2] = {...}
Before running a simulation, a specific scenario can be chosen in the user interface.
Do not set model parameters in the
setupInitialState
procedure! This would interfere with parameter variation experiments in which the same parameter(s) are used.
7.4. Statistics
In scientific and engineering simulation projects the main goal is getting estimates of the values of certain variables or performance indicators with the help of statistical methods. In educational simulations, statistics can be used for observing simulation runs and for learning the dynamics of a simulation model.
For collecting statistics, suitable statistics variables have to be defined, as in the following example:
sim.model.setupStatistics = function () { sim.stat.arrivedCustomers = 0; sim.stat.departedCustomers = 0; sim.stat.maxQueueLength = 0; };
Statistics variables have to be updated in onEvent methods. For instance, the variables arrivedCustomers and maxQueueLength are updated in the onEvent method of the CustomerArrival event class:
class CustomerArrival extends eVENT { ... onEvent() { ... // update statistics sim.stat.arrivedCustomers++; if (this.serviceDesk.queueLength > sim.stat.maxQueueLength) { sim.stat.maxQueueLength = this.serviceDesk.queueLength; } ... } }
In certain cases, a statistics variable can only be computed at the end of a simulation run. For this purpose, there is the option to define a computeFinalStatistics procedure:
sim.model.computeFinalStatistics = function () { // percentage of business days without stock-outs sim.stat.serviceLevel = (sim.time - sim.stat.nmrOfStockOuts) / sim.time * 100; };
After running a simulation scenario, the statistics results are shown in a table:
arrivedCustomers | 289 |
departedCustomers | 288 |
maxQueueLength | 4 |
7.5. Simulation Experiments
There are different types of simulation experiments. In a simple experiment, a simulation scenario is run repeatedly by defining a number of replications (iterations) for being able to compute average statistics.
In a parameter variation experiment, several variants of a simulation scenario (called experiment scenarios), are defined by defining value sets for certain model parameters (the experiment parameters), such that a parameter variation experiment run consists of a set of experiment scenario runs, one for each combination of parameter values.
An experiment type is defined for a given simulation model and an experiment of that type is run on top of a given simulation scenario for that model.
When running an experiment, the resulting statistics data are stored in a database, which allows looking them up later on or exporting them to data analysis tools (such as Microsoft Excel and RStudio)
Simple Experiments
A simple experiment type is defined with a
sim.experimentType
record on top of a model by defining (1) the
number of replications and (2) possibly a list of seed
values, one for each replication. The following code shows an example
of a simple experiment type definition:
1 2 3 4 5 | sim.experimentType = { title: "Simple Experiment with 10 replications, each running for 1000 time units (days)", nmrOfReplications: 10, seeds: [123, 234, 345, 456, 567, 678, 789, 890, 901, 1012] }; |
Running this simple experiment means running the underlying scenario 10 times, each time with another random seed, as specified by the list of seeds. The resulting statistics are composed of the statistics for each replication complemented with summary statistics listing averages, standard deviations, min/max values and 95% confidence intervals, as shown in the following example:
Experiment Results | |||
---|---|---|---|
Replication | Statistics | ||
arrivedCustomers | departedCustomers | maxQueueLength | |
1 | 285 | 283 | 7 |
2 | 274 | 274 | 6 |
3 | 285 | 285 | 4 |
4 | 287 | 286 | 5 |
5 | 284 | 284 | 6 |
6 | 300 | 299 | 4 |
7 | 288 | 286 | 5 |
8 | 286 | 284 | 4 |
9 | 286 | 285 | 4 |
10 | 295 | 293 | 6 |
Average | 287 | 285.9 | 5.1 |
Std.dev. | 6.848 | 6.506 | 1.101 |
Minimum | 274 | 274 | 4 |
Maximum | 300 | 299 | 7 |
CI Lower | 282.9 | 281.9 | 4.4 |
CI Upper | 291 | 289.6 | 5.7 |
When no seeds are defined, the experiment is run with implicit random seeds using JavaScript's built-in random number generator, which implies that experiment runs are not reproducible.
Parameter Variation Experiments
A parameter variation experiment is defined with (1) a number of replications, (2) a list of seed values (one for each replication), and (3) one or more experiment parameters.
An
experiment parameter must have the same name as the model parameter to which
it refers. It defines a set of values for this model parameter, either using
a values
field or a combination of a startValue
and endValue
field (and stepSize
for a non-default
increment value) as in the following example.
The following code shows an example of a parameter variation experiment definition (on top of the Inventory-Management simulation model):
1 2 3 4 5 6 7 8 9 10 11 | sim.experimentTypes[1] = { id: 1, title: "Parameter variation experiment for exploring reorderInterval and targetInventory", nmrOfReplications: 10, seeds: [123, 234, 345, 456, 567, 678, 789, 890, 901, 1012], parameterDefs: [ {name:"reviewPolicy", values:["periodic"]}, {name:"reorderInterval", values:[2,3,4]}, {name:"targetInventory", startValue:80, endValue:100, stepSize:10}, ] }; |
Notice that this experiment definition defines 9 experiment scenarios resulting from the combinations of the values 2/3/4 and 80/90/100 for the parameters reorderInterval and targetInventory. Running this parameter variation experiment means running each of the 9 experiment scenarios 10 times (each time with another random seed, as specified by the list of seeds). The resulting statistics, as shown in the following table, is computed by averaging all statistics variables defined for the given model.
Experiment Results | ||||
---|---|---|---|---|
Experiment scenario | Parameter values | Statistics | ||
nmrOfStockOuts | lostSales | serviceLevel | ||
0 | periodic,2,80 | 21.8 | 180.7 | 97.82 |
1 | periodic,2,90 | 7.4 | 55.9 | 99.26 |
2 | periodic,2,100 | 2.1 | 15.8 | 99.79 |
3 | periodic,3,80 | 86.6 | 855.6 | 91.34 |
4 | periodic,3,90 | 40.6 | 377.5 | 95.94 |
5 | periodic,3,100 | 16.3 | 139.8 | 98.37 |
6 | periodic,4,80 | 171.5 | 2067.5 | 82.85 |
7 | periodic,4,90 | 110.6 | 1238.3 | 88.94 |
8 | periodic,4,100 | 63.8 | 661.4 | 93.62 |
Storage and Export of Experiment Results
In OESjs-Core1, an experiment's output statistics data is stored in a browser-managed database using JavaScript's IndexedDB technology. The name of this database is the same as the name of the simulation model. It can be inspected with the help of the browser's developer tools, which are typically activated with the key combination [Shift]+[Ctrl]+[I]. For instance, in Google's Chrome browser, one has to go to Application/Storage/IndexedDB.
The experiment statistics database consists of three tables containing data about (1) experiment runs, (2) experiment scenarios, and (3) experiment scenario runs, which can be exported to a CSV file.
7.6. Using the Simulation Log
The OESjs-Core1 simulator can generate a simulation log, which allows to inspect the evolving states of a simulation run. Inspecting the simulation log can help to understand the dynamics of a model, or it can be used for finding logical flaws in it.
The contents of the simulation
log can be controlled by defining labels for those object
properties that are to be displayed in the log. For instance, in the case of
the Service-Desk-1 model, a label "qLen" is defined for the
queueLength
property of ServiceDesk objects by
setting
ServiceDesk.labels = {"queueLength":"qLen"};
This results in the following simulation log:
Step | Time | System State | Future Events |
---|---|---|---|
0 | 0 | Service-Desk-1{ qLen: 0} | CustomerArrival@1 |
1 | 1 | Service-Desk-1{ qLen: 1} | CustomerDeparture@5, CustomerArrival@6 |
2 | 5 | Service-Desk-1{ qLen: 0} | CustomerArrival@6 |
3 | 6 | Service-Desk-1{ qLen: 1} | CustomerArrival@7, CustomerDeparture@10 |
4 | 7 | Service-Desk-1{ qLen: 2} | CustomerDeparture@10, CustomerArrival@10 |
5 | 10 | Service-Desk-1{ qLen: 2} | CustomerArrival@12, CustomerDeparture@13 |
6 | 12 | Service-Desk-1{ qLen: 3} | CustomerDeparture@13, CustomerArrival@16 |
7 | 13 | Service-Desk-1{ qLen: 2} | CustomerArrival@16, CustomerDeparture@16 |
8 | 16 | Service-Desk-1{ qLen: 2} | CustomerDeparture@19, CustomerArrival@21 |
9 | 19 | Service-Desk-1{ qLen: 1} | CustomerArrival@21, CustomerDeparture@23 |