8.8 Completing the Project
A major aspect of the use of simulation is convincing the key decision makers concerning the recommendations of the study. Developing an animation for the model is especially useful in this regard. For the SMTesting model, an animation is available within the file. The animation (Figure 8.34) animates the conveyor, the usage of the resources, queues, and has a number of variables for monitoring the system. In addition, user defined views were defined for easily viewing the animation and the model logic.
Figure 8.34: Overview of the simulation animation
A written report is also a key requirement of a simulation project. (Chung 2004) provides some guidelines for the report and a presentation associated with a project. An outline for a simulation project report is provided here.
Executive Summary
Provide a brief summary of the problem, method, results, and recommendation. It should be oriented towards the decision maker.
A good executive summary should have the following characteristics: Essential facts of problem provided; methods of analysis summarized succinctly; recommendations clearly tied to results, clear decision recommended; essential trade-offs provided in terms of key performance measures; decision maker knows what to do after reading the summary.
Introduction
- Provide a brief overview of the system and problem context. Orient the reader as to what should be expected from the report.
System and Problem Description
- Describe the system under study. The system should be described in such a way that an analyst could develop a model from the description. Describe the problem to be solved. Indicate the key performance metrics to be used to make the decision and the major goals/objectives to be achieved. For example, the contest problem description given in this chapter is an excellent example of describing the system and the problem.
Solution and Modeling Approach
Describe the modeling issues that needed to be solved. Describe the inputs for the model and how they were developed. For each major modeling issue you should describe briefly your conceptual approach. Relate your description to the system description. Provide activity diagrams or pseudo-code and describe how this relates to your conceptual model. Describe the major outputs from the model and how they are collected.
Describe the overall flow of the model and relate this to your conceptual model. Give an accounting of important modeling logic where the detail is necessary to help the reader understand a tricky issue or important modeling issue.
State relevant and important assumptions. Discuss their potential effect on your answers and any procedures you took to test your assumptions.
Describe any verification/validation procedures. Describe your up-front analysis. You might approximate parts of your model or do some basic calculations that give you a ballpark idea of the range of your outputs. If you have data from the system concerning expected outputs, then you should compare the simulation output to the system output using statistical techniques. For a more detailed discussion of these techniques, please see the excellent discussion in (Balci 1998).
Experimentation and Analysis
Describe a typical simulation run. Is the model a finite horizon or steady state model? Analyzed by replication method or batch means? If it is a steady state simulation, how did you handle the initial transient period? How did you determine the run length? The number of replications? Describe or reference any special statistical methods that you used so that the reader can understand your results.
Describe your experimental plan and the results. What are your factors and the levels? What are you varying in the model? What are you capturing as output? Describe the results from your experiments. Use figures and tables and refer to them in your text to explain the meaning of the results. Discuss the effect of assumptions, sensitivity analysis and their effect on the model’s results.
Conclusions and Recommendations
- Give the answers to the problem. Back up your recommendations based on your results and analysis. If possible, give the solution in the form of alternatives. For example: “Assuming X, we recommend Y because of Z” or “Given results X, Y, we recommend Z”
References
Appendix
- Supporting materials. Detailed computer model documentation.
In addition to the above material, a simulation report might also contain plans for how to implement the recommendations. A presentation covering the major issues and recommendations from the report should also be prepared. You can think of the presentation as a somewhat expanded version of the executive summary of the report. It should concentrate on building credibility for your analysis, the results, and the recommendations. You should consider using your animation during the presentation as a tool for building credibility and understanding for what was modeled.
Besides the model, written report, presentation, and animation, it is extremely useful to provide model/user documentation. To assist with some of the documentation, you might investigate the Model Documentation Report available on the Tools menu. This tool produces an organized listing of the modules within the model. However, this tool is no substitute for fully describing the model in a language independent manner within your report. Even if non-simulation oriented users will not be interacting with the model, it is very useful to have a users guide to the model of some sort. This document should describe how to run the model, what inputs to modify, and what outputs to examine, etc. Often a model built for a single use is re-discovered and used for additional analysis. A users guide becomes extremely useful in these situations.