Test procedures for the handler

Introduction

This document gives an overview of test procedures for the CCP4i project database handler "dbccp4i" and the underlying code. The tests are divided into different types:

More tests may be added in future.

Manual Test Procedures

The following procedures outline manual tests that can be performed to check that the software exhibits the correct behaviours.

Identifier Name Description Procedure & Expected Outcomes
#1 Only one handler runs per user Only one instance of the handler should run for each user of the system. Starting a subsequent instance of the handler should result in control being passed to the existing one, and the subsequent instance should then stop.

On UNIX/Linux:

  1. Start a handler in one terminal window using the "dbccp4i" command

    You should see the startup message from the handler:

    ================================================================
    DB Handler started at 2006-10-30 15:06:12.258985
    Version:  $Revision: 1.8 $
    ================================================================
    
  2. Start a second handler in another terminal in the same way

    You should see the following message from the second handler:

    There is another handler running on port 4090.
    
  3. Do "ps -ef | grep python"

    Only one python handler process should be running:

    pjx      22602 15780  2 15:09 pts/11   00:00:00
             python /home/pjx/BIOXHIT/Bioxhit_db/dbccp4i/dbccp4i/dbccp4i.py
    
#2 Multiple clients Two clients operating on behalf of the same user should be able to connect to the same handler, and modify and read the same data.

On UNIX/Linux:

  1. Start a handler process using the "dbccp4i" command

    You should see the startup message from the handler:

    ================================================================
    DB Handler started at 2006-11-02 13:54:58.231302
    Version:  $Revision: 1.8 $
    ================================================================
    
  2. Start a console process in another window

    You should see the console start up message and list of commands.

  3. Start a second console process in yet another window

    You should see the console start up message and list of commands.

  4. Open the same project in both consoles by doing OpenDatabase in each

    The project should be opened, e.g.:

    % OpenDatabase JUNK
    OpenDatabase JUNK
    Result:
    OK
    
  5. Create a new job in the first console

    A new job should be created and the id number returned, e.g.:

    % NewRecord JUNK
    NewRecord JUNK
    Result:
    3
    

    Broadcast messages should be received in each console:

    % ******************************************************
    Received broadcast message:
    
    Update JUNK 3
    ******************************************************
    
  6. List the jobs in the second console

    The new job id number should also appear in the list, e.g.:

    % ListJobs JUNK
    ListJobs JUNK
    Result:
    1 2 3
    
  7. In the second console, set the TASKNAME, TITLE and STATUS data items for the new job

    % SetData JUNK 3 TASKNAME test TITLE "This is a test" STATUS REPORTED
    SetData JUNK 3 TASKNAME test TITLE "This is a test" STATUS REPORTED
    Result:
    1
    

    Broadcast messages should be received in each console:

    % ******************************************************
    Received broadcast message:
    
    Update JUNK 3
    ******************************************************
    
  8. In the first console, check the values of the TASKNAME, TITLE and STATUS data items for the new job

    % GetData JUNK 3 TASKNAME STATUS TITLE
    GetData JUNK 3 TASKNAME STATUS TITLE
    Result:
    test REPORTED {This is a test}
    

Tcl Test Scripts

The scripts are in the $BIOXHIT_TOP/dbccp4i/test/ directory.

Identifier Name Description & Outcomes Bugzilla No
Script#1 rude_client.tcl

Tests for handler failure on unexpected client disconnection

Usage: tclsh rude_client.tcl

If the test succeeds then the handler should still be running correctly after the client disconnects.

#1879
Script#2 dirlock_client.tcl

Tests for dealing with a lock on user's directories.def file when client attempts to register.

Usage: tclsh dirlock_client.tcl

If the test succeeds then the handler should deal correctly with the lock it encounters.

#1860
Script#3 opendb_client.tcl

Tests for dealing with existing lock on project database when client attempts to open it.

Usage: tclsh opendb_client.tcl <project_name>

If the test succeeds then the script should report the list of jobs found in the project.

#1890
Script#4 check_saves.tcl

Independently monitors saves of the database.def file for a project

Usage: tclsh check_saves.tcl <project_name>

This monitoring script is intended to be used in conjunction with the console application. It checks periodically on the last modification time of the project in question, and reports when the file has been updated (i.e. rewritten). Ideally writes should only happen when the database content has been updated by the console (or other) application.

Not assigned
Script#5 openclose_client.tcl

Attempts to close unopened or previously closed projects, and to open projects that have already been opened.

Usage: tclsh openclose_client.tcl <project_name>

The script attempts to perform the following actions:

  • Close the project before opening it
  • Open the project, and then open it again
  • Close the project, and then close it again

The script reports the success or failure for each of these actions, which should all fail if the tests are successful.

#1888
Script#6 bigrequest_client.tcl

Operates in one of two modes. In the first (default) mode, it opens the specified project and then submits requests which are longer each time, until the handler fails to return consistent output, giving a rough estimate of the maximum size of the request that the system is able to process through the Tcl client API.

In the second mode the user specifies a maximum number of jobs that the system must process, and the script reports whether this number can be successfully processed.

Usage: tclsh bigrequest_client.tcl <project_name> [-limit <n> ] [ -adjust_timeout ]

In the default mode, the script attempts to perform the following actions:

  • Open the project and acquire the id for the last job in that project
  • Submit a GetDescription request asking for the descriptions for a list of jobs consisting of that job repeated one hundred times, and check that the number of descriptions returned is equal to the number of job ids.
  • Increase the list of jobs by 100 and repeat until failure occurs.

The script reports the size (in terms of numbers of jobs) of the failing request.

If -limit is specified then the script tries a single request with a list of <n> jobs, and reports whether or not this was successfully processed.

Notes:

This test is of limited use in ascertaining the maximum size of requests that the system can deal with.

  • In either mode the maximum number of jobs is not a precise value, and may vary depending on the length of the data values associated with the job that is chosen for the test - essentially it is a ballpark indication only.
  • The upper limit is also dependent on the timeout that is chosen for the client API; for large requests the default timeout of 5000ms is too short for the transaction to be completed. This can lead to discrepancies in the maximum number of jobs obtained using the two different modes.
    Specifying the -adjust_timeout option attempts to adjust the timeout period to compensate.
    It is possible that other factors such as machine load might also make a difference to the maximum request size observed.

For reference: with the default timeout in the iterative mode, the maximum request size on one particular machine. is around 16300 jobs. In the same mode with the adjusted timeout option, the maximum request size obtained is significantly higher (41400 jobs).

#1921
Script#7 test_dbxml.tcl

Loads the functions from dbxml.tcl in the ClientAPI directory and attempts various operations to test the functions.

Usage: tclsh test_dbxml.tcl

The script attempts to perform the following actions:

  • Wraps a number of requests into XML using WrapDbRequest
  • Unwraps a number of XML responses using UnwrapDbResponse
  • Unwraps an XML broadcast using UnwrapDbBroadcast

Note that this script tests the functions directly without invoking the handler or the client API.

Not assigned
Script#8 ccp4icompat_client.tcl

Attempts to simulate the effect of a second process (e.g. CCP4i) interfering with the project lock and database files while the handler is also accessing the project.

Usage: tclsh ccp4icompat_client.tcl <project_name>

The script attempts to perform the following actions:

  • Open the database for the specified project
  • Override the lock on the project database
  • Update the modification time on the project database

At each step the script attempts to get data from the project to verify that the behaviour is as expected.

#2089

Python Test Modules

The modules are in the $BIOXHIT_TOP/dbccp4i/test/ directory.

Module name Description Usage
ccp4ipy_tests.py Test functions for the ccp4i.py module

Usage:

% python ccp4ipy_tests.py
test_ccp4i.py

Test functions for the ccp4i.py module built using the Python unittest module.

This is a fairly comprehensive set of tests of the directories and database classes for a variety of situations.

Usage:

% python test_ccp4i.py

CVS Id: $Id: tests.html,v 1.8 2007/02/09 15:52:24 pjx Exp $