============================================================= SOS: Scripts for Organizing 'Speriments (formerly: Scripts for Organizing Simulations) version 1.1.0 ============================================================= Welcome to the README! The purpose of this file is to introduce the user to the (rather large) set of scripts contained in this directory. SOS is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You can find the text of the GNU General Public License in the file 'LICENSE' in this directory. Author information can be found at the end of this 'README' file. =============== 1. Introduction =============== The purpose of this set of scripts is to ease the process of running a large number of experiments and working with the resulting data (including plotting). The work required to do such things manually can be quite large, especially taking into account that the number of experiments that need to be run in order to obtain representative data is often big. Building experiment files for every run and then processing the results of each run can be quite a painful task. Also, there has to be a way to easily organize and manage that large set of experiments. For that purpose, we introduced three new concepts: family, experiment and experiment group. This set of scripts was originally called SOS and designed to be used with Java implementations of SSF, but has since been made more generic to work with any set of experiments performed on a computer. If you've never heard of SSF, that's ok. You can read section 7 of this README file if you want to know what it is, and then ignore it unless you're interested in discrete-event simulation. The SSF-specific scripts can be found in a subdirectory called 'SSF'. Instructions for using SOS with SSF can be found in section 7 of this file. Experiments and MySQL Database ------------------------------ We have chosen to use the MySQL database to store the experiment results as well as some of the scripts. MySQL was chosen over other database servers because: 1) It's free. 2) It (almost completely) supports the SQL standard. 3) It's open source. Why involve a database? ----------------------- We have chosen to involve the MySQL database for the following reasons: 1) SQL is a powerful language for data manipulation. After the experiment is complete, a user usually wants to manipulate the data in various ways. Relational databases, including SQL, provide the user with lots of freedom for data manipulation, far more than anything we could have hoped to implement ourselves. 2) MySQL can be accessed remotely, by different users on different machines. Therefore, several users can share the results of their experiments through a central repository. Also, a single user can run experiments and access data from multiple locations. =============== 2. Installation =============== In order to use SOS you must do the following: - Make sure you have access to a MySQL server, using MySQL version 4.0 or higher, and enough privileges on that server to create and modify a database. The MySQL server doesn't have to be run locally. If your MySQL server is earlier than version 4, you will not be able to use the script 'rmexpr.pl'. The MySQL server can be downloaded from http://www.mysql.com/. - Perl must be installed on the local machine along with the following Perl modules. - DBI - DBD::mysql, version 2.10 or higher (I am finding the version numbers rather confusing for this module. It seems that DBD::msql and DBD::mysql were combined at some point, but still have different version numbers.) - Getopt (probably included in your perl installation) All the modules can be downloaded from http://www.cpan.org/. - In order to generate plots you must have gnuplot installed. Gnuplot can be downloaded from http://www.gnuplot.info/. Each user must: --------------- Unpack the archive file in the directory which you would like to use for your experiments. Modify the 'Makefile' so that the PERL variable is set to the full pathname of your perl interpreter followed by any command-line options you may need so that it can locate the necessary modules. Run 'make install'. This will change the first line of each perl script in the distribution so that the perl interpreter command is correct. It will then copy 'globals.pl.default' to 'globals.pl' and copy 'mkplot_vars.pl.template' to 'mkplot_vars.pl'. The original versions of the perl scripts which are changed by 'make install' are saved in files ending with '.orig'. Once you have verified that the modification was correct you can use the command 'rm *.orig extractors/*.orig families/*/*.orig' to remove the orginial files. Modify the appropriate variables in 'globals.pl'. It should contain information about the user, database operation, and location of the directories crucial for correct behavior of the scripts, among other things. You may later wish to add other variables to 'globals.pl' which are specific to his experiments. Run the mkenv.pl script to initialize your MySQL database with your environment information. The user is advised to become familiar with MySQL commands and the formats of the tables in the database before proceeding. A recommended start is to add the sample family, experiment group, and extractor to the database. Then run the sample set of experiments, read section 6, and connect and explore the database. The sample experiments and their results can be removed from the database when they are no longer useful. ============== 3. Definitions ============== Family ------ One of the ways to run large number of experiments is to create a template experiment file and replace the parameters with real values, one for each unique experiment. The notion of 'family' captures the concept of defining the template based on some set of parameters. Each family in our workspace is defined by the unique family name, a template script which generates the command line(s) by which the experiment is executed, and the set of parameters (contained in the 'schema.pl' and 'simschema.pl' files). Experiment ---------- An experiment is the result of replacing all the parameters in the template with real values and running the necessary command(s). Experiment group ---------------- A group of experiments. While the concepts of family and experiment are quite clear, the concept of the experiment group might not seem so straightforward. We'll try to clarify: Usually the user who is creating and running the experiments doesn't want to redefine the values for the template parameters for each new experiment. One may want to fix values of some parameters, while varying the values for other parameters. For example, a user simulating the behavior of a mesh of routers may want to fix the size of the mesh to a specific value, and vary the value for the random number seed between 1 and 30. In doing so, the user has specified a group of experiments constituting all possible combinations of values of the parameters. In this example, there are only two parameters: mesh size, which has just one value, and random number seed, which has 30 values. (One could argue that there's really only one parameter here, since mesh size is fixed. However, that's not important to make the point here.) Each one of the possible combinations of parameter assignments constitutes a single experiment in the group. For example, imagine the following family of experiments: a line of BGP routers with BGP sessions established between the neighboring routers. For this experiment family, we could define the following parameters: 'size': the number of routers in the line 'linkd': a link delay on channels between each two routers, in seconds 'seed': a random number seed We can create one experiment group by fixing the value for 'size' to 5, varying the value for 'linkd' from 1 to 3 seconds, and varying the value of 'seed' from 10 to 12. Thus, we define the experiment group: size=5 linkd=[1-3] seed=[10-12] This experiment group contains 9 experiments: size=5;linkd=1;seed=10 size=5;linkd=1;seed=11 size=5;linkd=1;seed=12 size=5;linkd=2;seed=10 ... Experiment parameters --------------------- Experiment parameters are parameters which are experimentally "interesting". They are likely to have different values for each experiment within a group. In the above example, 'size', 'linkd', and 'seed' could be considered experiment parameters; however, if you don't plan to change the size of the mesh you might want to consider 'size' a simulation parameter instead. Simulation parameters --------------------- Simulation parameters are those which are necessary for the proper execution of the simulation experiment, but not otherwise valuable to the experiment. Simulation parameters cannot take on ranges within an experiment group; each such parameter must be bound to a single value. The 'size' parameter above could be a simulation parameter, as could a configuration input filename, the number of units of time you wish to simulate, or even the name of the program to run. Extractors ---------- Extractors are scripts, written by the user, which extract the interesting values from the output of experiments and insert them in the MySQL database. One or more extractors may be bound to each family or experiment group. ============================================= 4. Using the scripts / Performing experiments ============================================= These scripts provide the user with the environment for creating, testing and running experiments, as well as manipulating their results. Once you have installed SOS, follow the instructions in this section to create and run experiments. Examples of all user-generated scripts are provided in the distribution. --------- Test runs --------- To test the correctness of the family and experiment group scripts before they are added to the database, use 'test_run.pl'. This will run each experiment in the specified group, displaying each set of parameters and each command as it is executed. Extractors are not tested by this script, and no data is taken from or entered in the database. ----------------- Defining a family ----------------- To define a family, you must write 3 Perl scripts. The default names and locations of the files are: /families//prepe.pl /families//schema.pl /families//simschema.pl ------ schema.pl and simschema.pl scripts ------ First, identify all the experiment parameters and simulation parameters required for the experiments. Experiment parameters must be specified in a file, typically called 'schema.pl', and simulation parameters must be specified in a second file, typically called 'simschema.pl'. These are both Perl scripts. The 'schema.pl' script should both be composed using the following format: ::= {'\n' } ::= $schema '{' '}' = ::= '"' type= ; desc= '"' ; The script must end with "1;" to prevent error messages when it is included in other scripts using a "require" statement. The 'simschema.pl' script uses the same format, replacing "schema" with "simschema". example: $schema{size} = "type=INTEGER;desc=size of the mesh"; $schema{ssld} = "type=VARCHAR(5);desc=use sender side loop detection (true or false)"; --------------- prepe.pl script --------------- After composing these two scripts, the user should write a script which takes as arguments a list of names and values, an id number for the experiment, the name of a directory for temporary data, and the name of the logfile. It generates the list of commands and their command-line arguments which make up an experiment. This script is typically called 'prepe.pl'. It generates a string, written to STDOUT, which contains the list of commands separated by ';'s. To help in processing the arguements, two Perl subroutines have been provided in the file 'prepe-lib.pl', 'prepe-lib.pl' can be found in the main SOS directory. It is recommended that all 'prepe.pl' files include this code near the beginning: # ------------------- BEGIN PERL BLOCK ------------------- # require "schema.pl"; require "simschema.pl"; require "globals.pl"; require "$workfolder/prepe-lib.pl"; ($spec, $expr_id, $datadir, $logfile) = process_args(@ARGV); $values = process_schemas($spec, \%schema, \%simschema); # -------------------- END PERL BLOCK -------------------- # Note that the files 'schema.pl', 'simschema.pl', and 'globals.pl' are assumed to be in the same directory as 'prepe.pl' and that they are called by their default names. This is because the scripts will have been stored in the database and will be written to the same directory as 'prepe.pl', in files with those names, by SOS when the experiment is executed. "process_args" and "process_schemas" are the subroutines provided in 'prepe-lib.pl'. Invoking the first verifies that the script was invoked with the proper arguments, and assigns those arguments to perl variables. The second takes the names and values in the list, $spec, and creates and associative array, to be referenced by $values. It verifies that each parameter in the family's definition will have a value, and then associates the name of the parameter with its value for the current experiment. SSF users will probably want the 'prepe.pl' script to generate DML code. These users should see section 7. ------------------------------------------------ Once an experiment group is defined you may use the script 'test_run.pl' to test the family and experiment group scripts before defining extractors and adding the family to the database. ------------------------------------------------ Extractors may be bound to a family at the time that the family is created or at a later time, using the bind.pl script. See the section "Defining an extractor". Note that extractors bound to a family will be used only if no extractor is bound to the experiment group used. Once the 3 required scripts have been written and tested and any desired extractors have been defined, run the script 'mkfamily.pl', using the syntax described in section 5. If at any time you wish to remove a family, use the script 'rmfamily.pl'. This would most likely be useful when first designing your experiments or when your entire project is complete. Because the contents of scripts are stored in the MySQL database, once the family has been created, any changes to the 3 required scripts will require the family to be removed and created again. ---------------------------- Defining an experiment group ---------------------------- After the family has been defined, the user should define one or more experiment groups. Each experiment group is specified in a pair of files, usually called 'vals.expr' and 'vals.sim'. The default location of the files is: /families/// 'vals.expr' contains a value, a list of values, or a range of values for each experiment parameter. It should be written in following format: ::= {'\n' } ::= = ::= | A range of values is specified in the following format: ::= [ {,}] ::= | - Furthermore, the file 'vals.sim', with a value for each simulation parameter, should be created using the following format: ::= {'\n' } ::= = ------------------------------------------------ It is now recommended that you use the script 'test_run.pl' to test the family and experiment group scripts before defining extractors and adding the experiment group to the database. ------------------------------------------------ Define any extractors which will be used specifically with this experiment group, or if you wish, you may add extractors later using the bind.pl script. Binding extractors to an experiment group will prevent extractors bound only to the family from being used. Use the mkexpr.pl script to add the experiment group to the MySQL database. Use the script 'rmexpr.pl' to remove an experiment group if it contains an error or if you no longer want a record of that group. Because the contents of scripts are stored in the MySQL database, once an experiment group has been created, any changes to the 'vals.expr' or 'vals.sim' files require the family to be removed and created again. --------------------- Defining an extractor --------------------- After an experiment is run there is typically some form of output saved to a file or files. The job of an extractor is to process this output in a user-defined way and store desired results in a database table. Users will probably want to write their own extractors suited to their particular experiments, following the examples provided in the 'extractors' directory. This directory is the default location for extractor scripts. Each extractor is specified by two Perl scripts. The first script, called .pl by default, does the actual processing of the experiment output and writes the results to the database. The second script, called _def.pl, contains the definitions of the fields produced by the extractor. -------- Definition script -------- The definition script must be written in the following format: ::= @fields = '(' [] ')' ; ::= {'\n' ','} '\n' ::= '"' name= ; type= ; desc= [; KEY] '"' ; example: @fields = ( "name=sent_updates;type=INTEGER;desc=number of updates sent;KEY", "name=rcvd_updates;type=INTEGER;desc=number of updates received;KEY" ) ; If the 'KEY' entry is present for a given field, it means that the field should be a primary key in the result table. This is crucial in order to prevent multiple entries for the same experiment to be written in the same table, in case the same experiment is run more than once. The 'KEY' entry should only be present for the fields that should be unique within a set of results for ONE experiment run. -------- Processing script -------- The following code should be at the beginning of each extractor's processing script: # ------------------- BEGIN PERL BLOCK ------------------- # require "globals.pl"; use DBI; use Getopt::Long; # SET TO NAME OF THIS EXTRACTOR IN EXTRACTOR TABLE $extrname = "sample"; $Getopt::Long::ignorecase = 0; GetOptions("v" => \$verbose_flag); $family_name = $ARGV[0]; $expr_name = $ARGV[1]; $expr_id = $ARGV[2]; $logid = $ARGV[3]; #the output of this script will be stored #in the table DBI:mysql:$dbname:$family_name\_$extrname\_results #connect to the database for this experiment $dbh = DBI->connect("DBI:mysql:$dbname:$mysqlhost:$mysqlport", $mysqluser,$mysqlpasswd); # all rows are the same, so id doesn't matter; we really only want one # row, but there's no way to get just one when they're all identical! $sth = $dbh->prepare("SELECT * FROM ${family_name}_simdata where id=$expr_id"); $sth->execute; $ref = $sth->fetchrow_hashref; $sth = $dbh->prepare("SELECT * FROM ${family_name}_exprdata where id=$expr_id"); $sth->execute; $e_ref = $sth->fetchrow_hashref; $mytempdir = "$tempfolder/$family_name/$expr_name"; # $is = beginning of MySQL command to insert data into database. $is = "INSERT INTO $family_name\_$extrname\_results VALUES ($expr_id,\"$expr_name\""; # -------------------- END PERL BLOCK -------------------- # Again, the file 'globals.pl' is required, but the user does not need to create it in the same directory as the extractor. Upon running an experiment, all the necessary scripts are loaded from the database and saved to the same directory. Each row in the results tables must have a unique experiment key and a unique experiment log key. These will be passed as arguments to the perl script at assigned to the variables $expr_id and $logid in the code above. The $extrname variable should be set to the name of the user's extractor. (In the above code the name is set to "sample".) The results of the experiment should be stored in a single row of the table '$extrname\_$family_name\_results' in the following order: 1. experiment id ($expr_id) 2. experiment group name ($expr_name) (This column does not hold results of the experiment and is never needed to select only the rows specific to a single experiment group, but we have found that it makes some common queries significantly easier.) 3. fields specified in the extractor definition file, in the same order as they are listed in that file 4. log id ($logid) The values of the parameters in the experiment as well as the output may be useful for computing some of the results. Therefore, the above code creates 2 associative arrays: $ref and $e_ref. $ref associates the name of each simulation parameter with its value for the current experiment. Likewise, $e_ref associates each experiment parameter with its value. The extractor's processing script will most likely end with the following code: # ------------------- BEGIN PERL BLOCK ------------------- # # end of MySQL command $is .= ",$logid)\n"; # execute MySQL command $dbh->do($is); # -------------------- END PERL BLOCK -------------------- # ----------------------------------- Once the scripts have been written, the extractor should be created using the 'mkextr.pl' script. Then the extractor may be bound to a family or experiment group. If the family or experiment group has already been created, use the 'bind.pl' script. If you wish to change the behavior of an extractor, as expressed in the processing script, rerun 'mkextr.pl' to save the changes in the database. If the definition script also changes, 'bind.pl' must be run to bind the modified extractor to the family or experiment group. Use the script 'rmextr.pl' to remove an extractor and the result tables filled by the extractor. See section 5 for details of what the script does. ----------------------- Running the experiments ----------------------- Once the database contains all the information about a set of experiments, the rest is quite simple. By invoking the 'do_run.pl' script with appropriate parameters (see section 5), the user initiates the process of running the experiments, processing outputs of those experiments, and writing the results to the result tables in the database. The results of all the experiment runs are stored in the MySQL database. The user now has the power of SQL process those results. One can either connect to the database manually, or use the script 'sql_run.pl' to execute a sequence of SQL statements on the data. ---------------- Generating plots ---------------- The script mkplot.pl parses a MySQL table, generates a data file and a gnuplot script, and calls gnuplot. It generates a plot in the graphics format specified in 'globals.pl'. This script is useful for those who do not like creating data files and entering gnuplot commands by hand. The file 'mkplot_vars.pl.template' contains the variables which can be modified in order to change the style of the plot. If you want something other than the generic plot, make a copy of 'mkplot.vars.pl.tempate'. The default name is 'mkplot_vars.pl', but you may wish to have multiple versions of the file with different names. Unlike other user-generated scripts, this is not stored in the MySQL database. The 'mkplot.pl' script is not considered to be part of the main SOS package, and it is assumed that the user may want to make changes to the file every time a plot is generated. See section 5 for a description of the command-line arguments to mkplot.pl. ================================= 5. Scripts and their descriptions ================================= bind.pl: usage: bind.pl [] specifies the family of experiments specifies the experiment group specifies the extractor Description: Binds an extractor to a family or an experiment group. "Binding" in this sense means that the extractor will be run after each experiment run in the family or group is complete. If the experiment group is specified it is bound to the group, adding an entry to the table 'extractor_family_binding'; otherwise, it is bound to the family, adding an entry to 'extractor_family_binding'. In addition, the result table, '$family-name_$extr-name_results' is created by this script. do_run.pl: usage: do_run.pl [-c ] [-frsvV] specifies the family name specifies the experiment name (belonging to that family) -c range indicates which experiments to run; for example, -c 1,10-20 in an experiment group with >=20 experiments -f force experiment group to continue running even if after an experiment returns an error. -r replace existing entries in the result table as well as any existing temp files -s save all temp files after the experiment -v verbose flag. if present, steps in execution of this script will be displayed -V very verbose flag. if present, steps in execution of this script and each of the extractors will be displayed Description: This is the script that actually runs all the experiments belonging to a specific experiment group. After each run is complete, extractors bound to the experiment group are run. If no extractors are specified for the experiment group, extractors bound to the family are run. The output of each extractor is stored in the database, in the result tables. All the temporary files that are created during the process of running experiments are stored in a temporary directory which is removed when the all of the runs are complete unless the '-s' option is specified. If the temp directory exists before do_run.pl is invoked, and the '-r' option is not specified, the user is given the options of replacing the temp directory, using the files in the existing temp directory at his own risk, or exiting. globals.pl: Usage: to be used by other scripts, not independently Description: File where all the global variables should be specified. This file describes the simulation environment for a specific machine and user. mkenv.pl: Usage: mkenv.pl Description: Stores the 'globals.pl' file into the database, keyed by the $USER and $HOSTNAME variables of the Unix environment. The table in which it stores this information is called 'userenv'. mkexpr.pl: usage: mkexpr.pl [-ev ] [-sv ] [-x extractor-list] specifies the family name specifies the experiment group name -ev specifies the experiment parameter values file -sv specifies the simulation parameter values file -x specifies the list of extractors (separated by commas) to bind to the experiment group Description: Stores the information about the new experiment group into the database. A row is inserted in the table '$family_experiments' containing the values specified by the experiment group. Then, all of the ranges of values are unfolded, and each combination of values is inserted in the table '$family_exprdata' as a separate row. mkextr.pl: usage: mkextr.pl [-f ] [-s ] [-ds desc-string>] [-df desc-file] specifies the extractor name -f specifies the name and location of the extractor's processing script -s specifies the location of the extractor's definition file -ds specifies the description of this extractor -df specifies the file with the description of this extractor Description: Stores the information about the extractor in the database. The table 'extractor' contains the each extractor's name, the Perl script file that does the extraction, and the description of the extractor specified by $desc-string or $desc-file. The table 'extractor_output_schemas' contains the schema of the output of this extractor (that is, the names, types and descriptions of all the fields this extractor produces, as defined by the definition file). mkfamily.pl: usage: mkfamily.pl [-x ] [-p ] [-s ] [-ss ] [-ds ] [-df desc-file] specifies the family name -x specifies the list of extractors (separated by commas) to be used with this family -p specifies the location of the 'prepe.pl' file for this family -s specifies the location of the 'schema.pl' file for this family -ss specifies the location of the 'simschema.pl' file for this family -ds specifies the string describing this family -df specifies the file with the description for this family Description: Stores the information about a family of experiments in the database. Table 'family' contains the family name, the 'prepe.pl' file and a short description of the family. The table 'family_schemas' contains the names, types, and descriptions of each experiment parameter, and the table 'family_simschemas' contains the same information for each simulation parameter. In addition, 'mkfamily.pl' creates the log tables, 'expr_log' and 'do_run_log' if they do not already exist, and the experiment tables, '$family-name_experiments' and '$family-name_exprdata'. If extractors are listed it also invokes the script 'bind.pl', which binds the extractors to the family (see bind.pl). mkplot.pl: usage: mkplot.pl [-p ] [-s ] [-v ] [-g] specifies the file where the MySQL output table is stored specifies the prefix for the gnuplot data and spec files, as well as the gnuplot output file -p specifies the path to where the files should be stored -s x1:y1[:z1]{,xn:yn[:zn]} specifies which columns in the MySQL table to plot. Columns x1, y1, and optionally z1 specify the first data set. xn, yn and optionally zn specify data set n. -v specifies a script file containing variables modified by the user to create a customized plot -g grid flag. If present, data will be plotted as a grid (only useful in the case of a 3D plot) Description: Processes the MySQL output that is stored either in tab-separated form, as generated by the sql_run.pl script, or in standard MySQL form, as displayed when MySQL is run interactively. It then generates a data file and a gnuplot specification file, based on the selection string. If no selection is specified, it is assumed that all the columns should be plotted using a single line or grid (if 3 columns are listed), using a default style. The gnuplot-var-file, if specified, should be a Perl script which, when included in the mkplot.pl script, will change the style of the plot. If this option is not specified the file 'mkplot_vars.pl' is automatically read. If the grid flag is present, and a plot is a 3D plot, data will be formatted in such a way that a "grid-like" plot is produced. In the end, the gnuplot specification file will be fed to the gnuplot program and the plot image will be produced in the file '.', where is the value of the $gnuext variable found in 'globals.pl'. rmexpr.pl: usage: rmexpr.pl specifies the family specifies the experiment group Description: Removes all tables from the database which are specific to the experiment group. This script also removes entries from all tables except log tables containing information about or results from this experiment group. rmextr.pl: usage: rmextr.pl specifies the extractor Description: First, it removes bindings between the extractor and any family or experiment group, getting confirmation from the user for each family or experiment group. When the binding is removed, the results table is also removed. If the extractor is no longer bound to any family or experiment group its description is removed from the database. rmfamily.pl: usage: rmfamily.pl specifies the family Description: Removes all tables from the database which are specific to the family. This script also removes entries from all tables except log tables containing information about or results from this family of experiments. sql_run.pl: usage: sql_run.pl -f [-d ] sql_run.pl -s [-d ] -f specifies a file containing an SQL statement -s specifies an SQL statement -d which MySQL database to use Description: Runs the specified sequence of SQL statements and prints the results to the standard output. Data will be printed out in tab-separated mode. It also prepends the text of the SQL statements to the output as comments. test_run.pl: usage: test_run.pl [-c] specifies the 'prepe.pl' file specifies the 'schema.pl' file specifies the 'simschema.pl' file specifies the 'globals.pl' file specifies the '.expr' experiment parameter values file specifies the '.sim' simulation parameter values file -c|-noc do/don\'t clean up all temp files after the experiment [default -c] Description: Convenience script. It does the same job as the 'do_run.pl' script, except it does not run any extractors. This script should be used for testing purposes, before the family and the experiment are stored into the database (see 'mkfamily.pl' and 'mkexpr.pl'). ====================================== 6. MySQL tables and their descriptions ====================================== do_run_log: Information about each run of a 'do_run.pl' script is stored here. That information includes the unique id, family name, experiment name, and the version number as defined in 'globals.pl'. expr_log: Information about each experiment run is stored here. The information includes a unique log id, unique experiment id, unique do_run_log id and a run time for that particular experiment. extractor: This table describes an extractor. The extractor's name, script file and a description are stored in this table. extractor_output_schemas: Table that describes the output schema for the extractor. Field names, types and descriptions of each of the field produced by the extractor are stored in this table. extractor_experiment_binding: Binding of experiment groups to extractors are stored here. This table contains the following fields: family name, experiment group name and extractor name. When an experiment group is bound to an extractor it follows that after each run of an experiment belonging to that group, the corresponding extractor will be run. extractor_family_binding: The binding of a family with an extractor is specified in this table. The table contains family_name-extractor_name pairs. When a family is bound to an extractor it follows that after each run of an experiment belonging to that family, the corresponding extractor will be run (but only in case that experiment group the experiment belongs to is not already bound to an extractor). family: Information about the family of the experiments is stored here. That information includes the family name, a 'prepe.pl' script file and a description of the family. family_schemas: This table describes the family experiment parameter schema. Field names, types and descriptions of those fields are stored in this table. When the 'do_run.pl' script is invoked, 'schema.pl' file is recreated from this table (you might have wondered why the 'schema.pl' is not stored in the database). family_simschemas: This table describes the family simulation parameter schema. Field names, types and descriptions of those fields are stored in this table. When the 'do_run.pl' script is invoked, 'simschema.pl' file is recreated from this table. X_experiments Information about all experiment groups belonging to family 'X' is stored in this table. The table contains the values for the each of the experiment parameter schema fields. This value can be either a single value, or the list of values. This table is the image of the 'vals.expr' file. X_exprdata This table contains instances of experiments, belonging to family 'X'. Experiments are created from table 'X_exprdata' (in this case), by creating all the possible combinations of values in that table. Each experiment has a unique id. X_simdata This table contains values for the simulation parameters for each experiment belonging to family 'X'. (Note that simulation parameter values are fixed for an experiment group--no ranges allowed as with experiment parameters.) X_Y_results The result table for the family 'X' and the extractor 'Y'. The information about each experiment gathered by extractor 'Y' is stored here. Also, the experiment id, the experiment group name, and the log id for that experiment run are stored in each row of the table (see tables X_exprdata' and 'expr_log'). userenv User environment table. The file 'globals.pl' is stored here, keyed by the $USER and $HOSTNAME variables of the Linux environment. ========================= 7. Using SOS with SSF ======================= Scalable Simulation Framework, or SSF, is an API for high-performance telecommunications simulators. Currently no fewer than four simulators implement this API. Among them are the Dartmouth SSF (DaSSF), a C++ simulator, and Raceway, a Java simulator. SSFNet is a popular add-on for Raceway, and it includes several modules for simulating network hardware and protocols. For full information on Raceway and SSFNet, visit ssfnet.org. SSFNet uses a simple language called DML (Domain Modeling Language) as the means for a user to configure a model. SOS includes some scripts specifically for those who are running experiments using SSFNet. Such SSFNet-specific files are stored in the SSF subdirectory. These files may also useful to those using other SSF-compliant simulators as well. When using SOS with SSFNet, a different DML file is required for each individual experiment. Building each DML file by hand can be very cumbersome, and thus you would typically want the prepe.pl script to generate the DML automatically. To help with generating DML on the fly, a library of useful subroutines called mkdml-lib.pl is available. To use the subroutines in this library, include the following line of code near the beginning of the prepe.pl file: require "$workfolder/SSF/mkdml-lib.pl"; For a good explanation of how to use mkdml-lib.pl to generate DML, see the text at the beginning of that file itself. See also the example experiment family called ssf_sample. ========== 8. Authors ========== Timothy G. Griffin AT&T Research Srdjan Petrovic Dartmouth College Anna Poplawski Dartmouth College BJ Premore Dartmouth College Copyright (C) 2001-2002 Timothy G. Griffin, Srdjan Petrovic, Anna Poplawski, BJ Premore