Running Models & Scenarios in Cosmic Frog

When a Cosmic Frog model has been built and scenarios of interest have been created, it is usually time to run 1 or multiple scenarios. This documentation covers how scenarios can be kicked off, and which run parameters can be configured by users.

Model Run Screen

When ready to run scenarios, user can click on the green Run button at the right top of the Cosmic Frog screen:

This will bring up the Run Settings screen:

1. We are on the Run Settings screen.
2. On the left-hand side of the screen a list of all scenarios contained in the model is shown. Users can select 1 or multiple of these to be run.
3. To facilitate finding specific scenarios in the list, text can be typed into the search box to filter the scenario list down to scenarios that contain the typed text in their name.
4. Clicking on this icon with the horizontal bars allows the user to sort the scenario list. Two sort options are available:
   1. Default: the scenarios are listed in the order they were added to the model (i.e. the same order as they are in in the Scenarios module).
   2. A-Z Name: this sorts the scenario list in alphabetical order. Clicking on this sort option again results in sorting the list in reverse alphabetical order.
5. These are the 5 icons associated with each of the Cosmic Frog engines. From left to right: network optimization (Neo), simulation (Throg), inventory (Dendro), Greenfield (Triad), and transportation optimization (Hopper). These icons are shown here for reference only.
6. The checkbox at the top of the scenario list can be used to check / un-check all scenarios that are showing in the list simultaneously.
7. Each scenario has an icon in front of it to show which engine will be used to run this scenario. These match the ones discussed under bullet number 5. The 2 scenario icons outlined in green are a Greenfield and a network optimization scenario, respectively. The other scenario icon outlined in green further down the list is an inventory scenario.
8. At the bottom of the scenario list the number of selected scenarios is listed. Here none are selected yet.
9. User can select the Resource Size that they deem most appropriate for the scenarios to be run with. Larger resource sizes have more CPUs and more RAM available; however, they also have higher billing factors. Therefore, the aim is usually to choose a resource size as small as possible to limit the number of hours used for running the scenarios, while still being large enough for the scenarios to not run out of memory. More guidance on Resource Size selection and assessment can be found in this help article “Resource Size Selection (Neo)”.
10. For each engine, run parameters are available for configuration by users. We will go through these for each engine in the following sections of this documentation. Note that no parameters are available here for Greenfield runs; parameters to run Greenfield scenarios are all specified in the Greenfield Settings input table, which are covered in this “Greenfield Settings Explained” Help Center article.
11. When ready to start running the selected scenario(s), user can click on the Run button. Note that it is now greyed out and unavailable as no scenarios have been selected. It will be available and green when 1 or more scenarios are selected. Should user decide to not run any scenarios at this time, they can close the Run Settings screen without kicking off any runs in 2 ways:
    1. By clicking on the Cancel button. If changes were made to any parameters, these changes will not be saved. In other words, the parameter values will be reverted to those that were there when the Run Settings screen was opened.
    2. By clicking on the x-icon at the right top of the Run Settings screen. If changes were made to any parameters, these changes will be saved.

Please note that:

• Scenarios which are run with the same engine will use the same set of parameters when kicked off simultaneously. Should there be a need to run scenarios with different parameters, user currently needs to kick them off in separate batches.
• Scenarios that are run simultaneously, will currently all use the same selected resource size. Should there be a need to run scenarios using different resource sizes, user currently needs to kick them off in separate batches. More to come in future releases!
• Users can check the progress of all runs in Optilogic’s Run Manager application:

1. To open the Run Manager, click on the Run Manager icon with the vertical bars on the left-hand side of the screen while on the Optilogic platform.
2. If the Run Manager application is not showing on the left-hand side of the screen while on the Optilogic platform, find the icon with the 3 dots and click on it to start showing all Optilogic applications. Then click on Run Manager.
3. There are 4 jobs listed here which, from bottom to top, are the overall job “General Demo_V3 (neo)” of kicking off network optimization (Neo) scenarios in the model named General Demo_V3. Three scenarios have been kicked off and are currently still running as their State = “running”: Sc1d_Site selection, Sc2_CoPacker, and Sc3_Prebuild Inventory.
4. There are 4 jobs listed here which, from bottom to top, are the overall job “General Demo_V3 (triad)” of kicking off Greenfield (Triad) scenarios in the model named General Demo_V3. Three scenarios have been kicked off and have finished running as their State = “done”: Sc1a_Greenfield 1 New Site, Sc1b_Greenfield 2 New Sites, and Sc1c_Greenfield 3 New Sites.
5. There are 2 jobs listed here which, from bottom to top, are the overall job “General Demo_V3 (dendro)” of kicking off inventory (Dendro) scenarios in the model named General Demo_V3. One scenario has been kicked off and has not completed successfully as its State = “error”: Sc9a_Dendro_Clean.
6. When selecting an individual job by clicking on the circle to the left of it, additional details of the run can be viewed on the right-hand side (not shown in screenshot). This includes job info, job (error) logs, job resource usage metrics, etc.
7. Not shown in the screenshot above, but if a user wants to end a run for example in case it was kicked off accidentally or it has been running for a long time without finishing, user can right-click on the job and choose “Cancel Job” from the context menu.

While we will discuss all parameters for each technology in the next sections of the documentation, users can find explanations for each of them within Cosmic Frog too: when hovering over a parameter, a tooltip explaining the parameter will be shown. In the next screenshot, user has expanded the Termination section within the Optimization (Neo) parameters section and is hovering with the mouse over the “MIP Relative Gap Percentage” parameter, which brings up the tooltip explaining this parameter:

When selecting one or multiple scenarios to be run with the same engine, the corresponding technology parameters are automatically expanded in the Technology Parameters part of the Run Settings screen:

1. 3 scenarios are selected in the scenario list; they are all optimization (Neo) scenarios.
2. When one or multiple optimization scenarios are selected in the list, the Optimization section in the Technology Parameters part of the Run Settings screen is automatically expanded.
3. User can manually expand/collapse these parameter sections by clicking on the greater than icon to expand, and the downward caret icon to collapse.
4. In the Optimization section of the Technology Parameters part of the Run Settings screen, there is 1 general parameter (Check Infeasibility) and 4 additional categories of optimization parameters: Termination, Network Output Reporting, and Troubleshooting. These sections are all collapsed at the moment; we will cover each and their parameters in the next sections of this documentation.
5. Note that the run button has now turned green as 3 scenarios have been selected in the scenario list.

Optimization (Neo) Parameters

When enabled, the Check Infeasibility parameter will run an infeasibility diagnostic on the model in order to identify any cause(s) of the scenario being infeasible rather than optimizing the scenario. For more information on running infeasibility diagnostics, please see this Help article.

Optimization (Neo) – Termination Parameters

1. Solve Time Limit (Minutes) – specify the maximum amount of time in minutes the scenario is allowed to run. If a feasible solution has been found within this time (regardless of the MIP Relative Gap), it will be returned. If no feasible solution has been found, the scenario run will end without returning results. The default is blank, meaning scenarios will run until a solution is found within the MIP Relative Gap Percentage, see the next bullet. In case needed, long running scenarios can manually be stopped by cancelling the job through the Run Manager application, as explained above.
2. MIP Relative Gap Percentage – once the difference (gap) between the best bound and currently best-found solution is less than this defined gap, the solver terminates, and the solution is returned as the solution. To set a value of 1%, enter 0.01. The default is set to 0.0001, meaning 0.01%.

Optimization (Neo) – Network Parameters

1. Lane Creation Rule – this parameter dictates how the transportation policies and sourcing policies (Customer Fulfillment Policies, Replenishment Policies, and Procurement Policies) input tables are used to create the lanes (origin-destination pairs) that are considered during the optimization. Options are:
   1. Transportation Policy Lanes Only: lanes are only created based on the origin-destination pairs specified in the Transportation Policies input table. This is the default setting for this parameter.
   2. Sourcing Policy Lanes Only: lanes are only created based on the origin-destination pairs specified in the Sourcing Policies input tables (which are: Customer Fulfillment Policies, Replenishment Policies, and Procurement Policies).
   3. Intersection: lanes are created for those origin-destination pairs that exist in both the Transportation Policies and Sourcing Policies input tables.
   4. Union: lanes are created for all origin-destination pairs that exist in the Transportation Policies and Sourcing Policies input tables.
2. Max Number Of Sources To Consider For Customers – when set, individual customers cannot source from more than this number of sources in total throughout the model horizon and over all products together. The sources are evaluated based on distance. For example, if 5 sources are set up for a certain customer in the model, and this parameter is set to 2, then out of those 5 sources, only the 2 that are closest in distance to the customer are considered as sources. The default is blank.
3. Max Number Of Sources To Consider For Facilities – when set, individual facilities cannot source from more than this number of sources in total throughout the model horizon and over all products together. The sources are evaluated based on distance. For example, if 3 sources are set up for a certain facility in the model, and this parameter is set to 1, then out of those 3 sources, only the 1 that is closest in distance to the facility is considered as a source. The default is blank.
4. Apply Max Number Of Sources By Location Only – when values are set for the previous 2 parameters, this parameter specifies if those max number of sources parameters are applied for each product-mode-period combination at that location or over all product-mode-period mode combinations together. When this parameter is off (the default), the max number of sources parameters will limit the number of sources for each destination across all product-mode-period combinations. When this parameter is turned on, the max number of sources parameters will limit the number of sources for each unique destination-product-mode-period combination.
5. Allow Cross Period Flows – turn this parameter on when flows that depart in one period and arrive in a later one need to be allowed in the model. I.e. (some) transportation times are longer than the period lengths. By default, this parameter is off, and flows will depart and arrive in the same period regardless of the transport times specified in the model.
6. Open Close At Most Once – when turned on (the default), facilities and work centers which can be opened and/or closed during the model horizon are not allowed to change status more than once during the model horizon. When turned off, facilities and work centers are allowed to change between open and close states an unlimited number of times during the model horizon.

Optimization (Neo) – Output Reporting Parameters

1. Print Detailed User Defined Variable Summary – when turned on, the Optimization User Defined Variable Summary and Optimization User Defined Variable Term Summary output tables are populated with all outputs related to user defined variables, including records with value of 0. When turned off (the default) these output table are only populated with non-0 values.
2. Print Constraint Summary – when turned on, the Optimization Constraint Summary output table will be populated, whereas when this parameter is turned off (the default) it will not be populated. As this table can contain a lot of records, it can be helpful to turn it off and only turn it on for troubleshooting purposes or for select scenarios to review how close some constraints are to maxing out for example.

Optimization (Neo) – Troubleshooting Parameters

1. Feasibility tolerance – since numbers have finite precision on digital computers, sometimes constraints cannot be met exactly. Therefore, the feasibility tolerance sets a limit on the amount of violation allowed on a constraint while still considering it “satisfied”. The default value is 0.000001, and for most models users will not need to change this. Should a user suspect a model turns infeasible due to slight constraint violation, user can test this out by relaxing the feasibility tolerance and setting it to a higher value, e.g. try 0.00001 first, then 0.0001, etc.
2. Write LP (linear program) File – when turned on, this will produce a file containing the linear programming formulation of the model that is being optimized. Experienced users may be able to review this for troubleshooting purposes, and Optilogic’s support team may ask for this when troubleshooting any models. The LP-file can be found in Explorer > My Files > debug_data > model_name > scenario_name > NEO.lp. By default, this option is turned off. The following screenshot shows where the NEO.lp file can be found when running a scenario with this parameter turned on:

1. 1. While in any application on the Optilogic platform, user can open the Explorer by clicking on the greater than icon at the top left of the screen. It will then turn into a less than icon, which clicked will hide the Explorer.
   2. To quickly find the NEO.lp file, we can search for “debug” in the search box.
   3. The file we’re looking for is located in:
      1. The folder “My Files”
      2. The sub-folder “debug_data”
      3. The sub-folder with the name of the model, in this case “General Demo_V3”
      4. The sub-folder with the name of the scenario, in this case “Sc1d_Site selection”
   4. The NEO.lp file in this folder is the file containing the linear programming formulation of this scenario.
   5. The input_solver sub-folder contains the .csv files that are generated when the next parameter “Write Input Solver” is turned on, see the next bullet.
3. Write Input Solver – when turned on, this will write all the data of the input tables that are used during the scenario run into .csv-files. This means that following have been applied before writing these files: scenario changes, excluded records are removed, and records using groups/named table filters are enumerated into records for the individual group/filter members (for groups used in constraints tables only where the group behavior field is set to enumerate). The .csv-files can be found in Explorer > My Files > debug_data > model_name > scenario_name > input_solver, see the screenshot above. By default, this option is turned off.
4. Relative Constraint Tolerance Percentage – constraints of type “Fixed With Tolerance” are relaxed by this amount. The default of 0.02 means a relaxation of 2%. For example, if a constraint of type “Fixed With Tolerance” is set to a constraint value of 500, then with a Relative Constraint Tolerance Percentage of 0.02 (2%), values from 490 up to 510 are accepted as satisfying the constraint.
5. Allow Lazy Constraints – when this option is turned on, constraints that are computationally expensive are at first turned off during the optimization and are added back in later in the process, so they are still respected in the final solution. By default, this option is turned off as it can slow the optimization down. Users are advised to turn it on when a performance issue is detected on a model, in which case it may be able to speed up the optimization.
6. Logical Constraints – if a model is poorly scaled (for example due to having both very small and very large demand quantities or cost numbers in it), turning this option on may prevent numerical issues leading to infeasibility of the model. Models can run longer when this option is used, however. By default, this option is turned off.

Simulation (Throg) Parameters

1. Number Of Replications – for simulation models that use variability in their inputs, multiple replications where different values are used for the variable inputs can be run. This can inform users on how robust a network is: out of all the replications run, how many show a well-functioning network in terms of cost, service and risk and how many do not? Running 20-30 replications is generally a good number to start with, keeping in mind that the more variable the input data is, the more replications should be used.
2. Run Replications in Parallel – when turned on, multiple replications will be run simultaneously rather than sequentially.
3. Log Progress – when turned on, a record will be printed in the job log of the simulation run for each day of the simulation horizon:

1. 1. Select the simulation run of interest in the Run Manager application on the Optilogic platform and choose to view the Job Log by clicking on the 4^th icon at the top right of the screen.
   2. The current date and time are printed for each record printed in the Job Log.
   3. The current simulation time is printed, a record for each day of the simulation is included.

This can help users troubleshoot issues. For example, if a simulation run takes longer than expected, users can review the Job Log to see if it gets stuck on any particular day during the modelling horizon.

4. Confidence Interval Percent – the confidence interval used for calculating half-width statistics. The default is set to 0.95 (95%).

Simulation (Throg) – Output Reporting Parameters

1. Inventory Reporting – choose whether and, if so, in what way the Simulation Inventory On Hand Report output table will be populated after a simulation run. Options are:
   1. End of Day Only: a record is printed for each facility-product combination at the end of each day.
   2. All Changes Only: a record is printed for facility-product combinations every time the inventory level changes.
   3. End of Day and All Changes: records are printed for each facility-product combination at the end of each day and also every time the inventory level changes. This is the default setting.
   4. None – the Simulation Inventory On Hand Report output table is not populated.
2. Print Order Report – when turned on (the default), the Simulation Order Report output table will be populated after a simulation run. This table tracks orders at all levels of the supply chain, e.g. production orders, replenishment orders, and customer orders. Users can see when a production order dropped, started, and was completed. For replenishment orders, users can see when they dropped, and when they were filled. If an order is (partially) cancelled, that will be indicated in this table too.
3. Print Shipment Report – when turned on (the default), the Simulation Shipment Report output table will be populated after a simulation run. All product movements are captured in this table, including what time the shipment departed, when it arrived, and what the due date was (if applicable).
4. Print Process Report – when turned on (the default), the Simulation Process Report output table will be populated after a simulation run. If any production, (un)loading), or (de)stocking processes are used in the simulation model, the details of when which process was used for what amount of product and time can be found in this output table.
5. Print Production Report – when turned on (the default), the Simulation Production Report output table will be populated after a simulation run. All productions of the simulation run are detailed in this output table, including start and end time, quantities, and cost associated with the production.
6. Queue Depth Reporting – choose whether and, if so, in what way the Simulation Queue Depth Report output tables (for Work Centers, Lanes, Lane Mode combinations, and Order Fulfillments) will be populated after a simulation run. There can be queues where items are waiting for Work Centers, Lanes, or Lane-Mode combinations to become available. These queue depth reports list how many items there are in these queues at certain times. Options are:
   1. End of Day Only: a record is printed for each queue at the end of each day.
   2. All Changes Only: a record is printed for queues every time the queue depth changes.
   3. End of Day and All Changes: records are printed for each queue at the end of each day and also every time the queue depth changes.
   4. None – the Simulation Queue Depth Report output tables are not populated. This is the default setting.
7. Print Work Center Throughput Report – this parameter is not currently used by the simulation engine.
8. Print Failed Sourcing Report – this parameter is not currently used by the simulation engine.

Inventory (Dendro) Parameters

The inventory engine in Cosmic Frog (called Dendro) uses a genetic algorithm to evaluate possible solutions in a successive manner. The parameters that can be set dictate how deep the search space is and how solutions can evolve from one generation to the next. Using the parameter defaults will suffice for most inventory problems, they are however available  to change as needed for experienced users.

1. Number Of Generations – the number of generations the genetic algorithm will go through and calculate. The default value is 5.
2. Number Of Genes Per Generation – This is the population size, i.e. the number of candidate solutions maintained in the population throughout the algorithm. The default value is 20.
3. Number Of Offsprings – this specifies how many offsprings are considered from the 2^nd until the last generation. This is the number of new candidate solutions created during a step (generation) of the genetic algorithm. The N best solutions from the current population and these offsprings are kept, where N is the population size parameter (Number Of Genes Per Generation, see previous bullet) and then the algorithm moves on to the next step. The default value is 20.
4. Mutation Probability – the chance that a gene factor will mutate from one generation to the next. The default is 10.
5. Max Values To Mutate – the maximum number of gene factors that are allowed to mutate from one generation to the next. The default value is 5.
6. Crossover Probability – the chance that crossover will occur between genes, e.g. genes are swapped. The default value is 90.
7. Number Of Crossovers – the number of crossover points. The default value is 2.
8. Random Seed – set the seed of the random number generator of the algorithm to start generating the initial population in an area where optimal solutions are likely to be found. When left blank (the default value), the initial population is generated randomly.
9. Fitness Score Calculator – available to enable using a customized approach and class name in the code (Optilogic can provide advice and the options for this). When this is left blank (the default value), the data from the Output Factors input table is used.

Transportation (Hopper) Parameters

1. Run Load Balancing – if weekly demand needs to be balanced over a week, the Load Balancing Demand and Load Balancing Schedules input tables can be used to set this up. If both the Shipments table and the Load Balancing Demand/Schedules tables are populated, by default the Shipments table will be used and the Load Balancing Demand/Schedules tables will be ignored. To switch to using the Load Balancing Demand/Schedules tables (and ignoring the Shipments) table, the Run Load Balancing toggle needs to be switched to on (toggle to the left and grey is off; to the right and blue is on).

Please feel free to contact Optilogic Support at support@optilogic.com in case of questions or feedback.