Manage Tester

Managing a Tester System

A tester system is one where the assignment testing occurs, most often by running a virtual machine and testing the assignment inside that. Assignments are sent from a storer system, they are placed in a queue for the tester and then picked up from the queue and tested/checked (inside an environment such as a virtual machine). Results from the testing/checking phase are sent out back to the storer.

Overview

A tester system for vmchecker consists of two sets of components: executors and queue managers.

An executor is a process that starts up the testing environment (most often a virtual machine), grabs an assignment and a test archive, kick off the checking phase and grabs back the results. An executor manages a given kind of environment (for example VMware, VirtualBox, KVM and so on). There is usually only a single executor type on a given tester system, for a particular type of environment (i.e. virtual machine implementation/hypervisor).

There is a queue manager for each class. Each class uses a Unix account and stores information about the executor and queue manager. A queue manager would handle check requests for a given class and kick of the executor for the class virtual machine.

Unix Account Folder Structure

Each class requires a Unix account. Inside the Unix account one stores the folder structure for queue managers and virtual machines for the class. Typically you would store the queue manager configuration in a tester/ subfolder and the virtual machines in a vm/ subfolder. A sample folder listing (for the asm account on the checker.cs.pub.ro tester system) is shown below:

asm@checker:~$ tree --charset ASCII -L 3 -F
.
|-- tester/
|   |-- callback_sftp.log
|   |-- config
|   |-- queue/
|   |-- queue-manager.log
|   `-- tmpunzip/
`-- vm/
    `-- 2015/
        |-- iocla-linux/
        `-- iocla.zip

In the folder listing, the vm/ subfolder stores the iocla-linux/ virtual machine and the tester/ subfolder stores the queue manager configuration files:

• config is the actual configuration file for the queue manager
• the queue/ subfolder is the assignment queue; assignments sent out from the storer are placed in this folder
• the queue-manager.log file stores logging messages output by the queue manager
• the tmpunzip/ subfolder is used for unpacking the current assignment
• the callback_sftp.log file stores logging information from when results from an assignment are sent back to the storer system

Setting Up a New Class

We assume you've already setup an Unix account for the storer system and for the tester system and you've made SSH public key-based authentication available, by following the instructions here.

Non-Privileged Actions

On the tester Unix account, create a folder for the virtual machines (we recommend naming it vm/) and copy and unpack the virtual machine(s) in that folder. Important: If using VMware these virtual machines need to have VMware Tools installed. You will then fire up vmware or ask the administrator for that to add a snapshot for the virtual machine. The path to the virtual machine is to be used in the virtual machine configuration section on the storer.

In order to create the folder for the queue manager, you would use the vmchecker-init-course command as shown below.

user@checker:~$ mkdir tester
user@checker:~$ cd tester/
user@checker:~/tester$ vmchecker-init-course tester
A new default vmchecker configuration file was written to /home/asm/tester/config. Update it before running any vmchecker code.
INFO:vmchecker.initialise_course: -- tester init done setting up paths and db file.
user@checker:~/tester$ ls -l
total 16
-rw-r--r-- 1 user user 7520 Dec 28 19:21 config
drwxr-xr-x 2 user user 4096 Dec 28 19:21 queue
drwxr-xr-x 2 user user 4096 Dec 28 19:21 tmpunzip

The tester/ subfolder now stores the required files for running the queue manager.

Before running the queue manager, we need to configure it by updating the tester/config file. In that file you need only update the first attributes; an example of configuring them is shown below:

asm@checker:~/tester$ head config
[vmchecker]

root = /home/asm/tester
repository = /home/asm/vmchecker-storer/repo
CourseName = Introducere in utilizara calculatorului si limbaj de asamblare
[...]

In the configuration above:

• root is the path to the queue manager folder on the tester
• repository is the path to the class folder on the storer
• CourseName is the name of the class