Test procedures for the handler

Introduction

This document gives an overview of test procedures for the CCP4i project database handler "dbccp4i" and the underlying code. The tests are divided into different types:

More tests may be added in future.

Manual Test Procedures

The following procedures outline manual tests that can be performed to check that the software exhibits the correct behaviours.

Identifier Name Description Procedure & Expected Outcomes
#1 Only one handler runs per user Only one instance of the handler should run for each user of the system. Starting a subsequent instance of the handler should result in control being passed to the existing one, and the subsequent instance should then stop.

On UNIX/Linux:

  1. Start a handler in one terminal window using the "dbccp4i debug" command

    You should see the startup message from the handler:

    ================================================================
    DB Handler started at 2006-10-30 15:06:12.258985
    Version:  $Revision: 1.14 $
    ================================================================
    
  2. Start a second handler in another terminal in the same way

    You should see the following message from the second handler:

    There is another handler running on port 4090.
    
  3. Do "ps -ef | grep python"

    Only one python handler process should be running:

    pjx      22602 15780  2 15:09 pts/11   00:00:00
             python /home/pjx/BIOXHIT/Bioxhit_db/dbccp4i/dbccp4i/dbccp4i.py
    
#2 Multiple clients Two clients operating on behalf of the same user should be able to connect to the same handler, and modify and read the same data.

On UNIX/Linux:

  1. Start a handler process using the "dbccp4i debug" command

    You should see the startup message from the handler:

    ================================================================
    DB Handler started at 2006-11-02 13:54:58.231302
    Version:  $Revision: 1.14 $
    ================================================================
    
  2. Start a console process in another window

    You should see the console start up message and list of commands.

  3. Start a second console process in yet another window

    You should see the console start up message and list of commands.

  4. Open the same project in both consoles by doing OpenDatabase in each

    The project should be opened, e.g.:

    % OpenProject JUNK
    OpenProject JUNK
    Result:
    OK
    
  5. Create a new job in the first console

    A new job should be created and the id number returned, e.g.:

    % NewJob JUNK
    NewJob JUNK
    Result:
    3
    

    Broadcast messages should be received in each console:

    % ******************************************************
    Received broadcast message:
    
    Update JUNK 3
    ******************************************************
    
  6. List the jobs in the second console

    The new job id number should also appear in the list, e.g.:

    % ListJobs JUNK
    ListJobs JUNK
    Result:
    1 2 3
    
  7. In the second console, set the TASKNAME, TITLE and STATUS data items for the new job

    % SetData JUNK 3 TASKNAME test TITLE "This is a test" STATUS REPORTED
    SetData JUNK 3 TASKNAME test TITLE "This is a test" STATUS REPORTED
    Result:
    1
    

    Broadcast messages should be received in each console:

    % ******************************************************
    Received broadcast message:
    
    Update JUNK 3
    ******************************************************
    
  8. In the first console, check the values of the TASKNAME, TITLE and STATUS data items for the new job

    % GetData JUNK 3 TASKNAME STATUS TITLE
    GetData JUNK 3 TASKNAME STATUS TITLE
    Result:
    test REPORTED {This is a test}
    
#3 ccp4i/dbhandler compatibility When running both dbhandler(including its clients) and ccp4i, ccp4i should be able to take control of the database while the handler should be able to read database and reflect the database update which made by ccp4i, but handler shouldn't be able to write to the database.

On UNIX/Linux:

  1. Start a dbconsole

    You should see the console start up message and list of commands.

    open a project in console by doing OpenDatabase projectname

    The project should be opened, e.g.:

    % OpenProject TEST
    Added item INPUT_FILES_STATUS to the db
    Added item OUTPUT_FILES_STATUS to the db
    Warning: directory in file is /home/wy45/projects/TEST/CCP4_DATABASE
    OpenProject TEST
    Result:
    1
    
  2. Start a ccp4i and open the same project.

    You should see a warning lock message, click 'Override Lock', the jobs should appear in the job window in ccp4i.

  3. Attempt to update data from console.

    In console, do SetData TEST 1 STATUS RUNNING

    This operation should failed. e.g.:

    % SetData TEST 1 STATUS RUNNING
    SetData TEST 1 STATUS RUNNING
    Result:
    0
    

    however, the console should be able to read the data. do GetData TEST 1 STATUS

    you should see the following:

    % GetData TEST 1 STATUS
    GetData TEST 1 STATUS
    Result:
    FINISHED
    
  4. Update database from ccp4i

    In ccp4i, edit the job 1, change status to FAILED

    console should receive broadcast message.

    % Warning: directory in file is /home/wy45/projects/TEST/CCP4_DATABASE
    ******************************************************
    Received broadcast message:
    
    TEST is changed by other process, the database is reloaded with readonly mode.
    ******************************************************
    

    in console, do GetData TEST 1 STATUS

    % GetData TEST 1 STATUS
    GetData TEST 1 STATUS
    Result:
    FAILED
    
  5. Exit both console and ccp4i.
  6. Exit console by doing exit

    % exit
    Disconnecting before exit, please wait...
    ==========================================================
     Finished
    ==========================================================
    wy45@WY45PORT:~/work/Bioxhit_db/dbccp4i/application> ======================================================================
    DB Handler stopped at 2007-03-09 14:57:11.859556
    ======================================================================
    

    Exit ccp4i by clicking 'Exit' button.

  7. Start a dbviewer.

    You should see the display of one of the existing project.

  8. Start a ccp4i and open the same project.

    You should see a warning lock message, click 'Override Lock', the jobs should appear in the job window in ccp4i.

  9. Update database from ccp4i

    In ccp4i, edit the job 1, change status to FINISHED

    In dbviewer, you should see it automatically refresh the display and the job number 1 status change to green.

  10. Exit both ccp4i and dbviewer.

  11. Start ccp4i.

  12. Start a console, Open the same project.

    e.g OpenProject TEST.

    % OpenProject TEST
    Added item INPUT_FILES_STATUS to the db
    Added item OUTPUT_FILES_STATUS to the db
    OpenProject TEST
    Result:
    0
    
    Open the project with readonly mode by doing OpenDatabase TEST -readonly.
    % OpenProject TEST -readonly
    Added item INPUT_FILES_STATUS to the db
    Added item OUTPUT_FILES_STATUS to the db
    OpenProject TEST -readonly
    Result:
    1
    
  13. Repeat step 3 & 4.

  14. Start a dbviewer, repeat step 8.

  15. Exit ccp4i, dbconsole & dbviewer.

Tcl Test Scripts

The scripts are in the $BIOXHIT_TOP/dbccp4i/test/ directory.

Identifier Name Description & Outcomes Bugzilla No
Script#1 rude_client.tcl

Tests for handler failure on unexpected client disconnection

Usage: tclsh rude_client.tcl

If the test succeeds then the handler should still be running correctly after the client disconnects.

#1879
Script#2 dirlock_client.tcl

Tests for dealing with a lock on user's directories.def file when client attempts to register.

Usage: tclsh dirlock_client.tcl

If the test succeeds then the handler should deal correctly with the lock it encounters.

#1860
Script#3 opendb_client.tcl

Tests for dealing with existing lock on project database when client attempts to open it.

Usage: tclsh opendb_client.tcl <project_name>

If the test succeeds then the script should report the list of jobs found in the project.

#1890
Script#4 check_saves.tcl

Independently monitors saves of the database.def file for a project

Usage: tclsh check_saves.tcl <project_name>

This monitoring script is intended to be used in conjunction with the console application. It checks periodically on the last modification time of the project in question, and reports when the file has been updated (i.e. rewritten). Ideally writes should only happen when the database content has been updated by the console (or other) application.

Not assigned
Script#5 openclose_client.tcl

Attempts to close unopened or previously closed projects, and to open projects that have already been opened.

Usage: tclsh openclose_client.tcl <project_name>

The script attempts to perform the following actions:

  • Close the project before opening it
  • Open the project, and then open it again
  • Close the project, and then close it again

The script reports the success or failure for each of these actions, which should all fail if the tests are successful.

#1888
Script#6 bigrequest_client.tcl

Operates in one of two modes. In the first (default) mode, it opens the specified project and then submits requests which are longer each time, until the handler fails to return consistent output, giving a rough estimate of the maximum size of the request that the system is able to process through the Tcl client API.

In the second mode the user specifies a maximum number of jobs that the system must process, and the script reports whether this number can be successfully processed.

Usage: tclsh bigrequest_client.tcl <project_name> [-limit <n> ] [ -adjust_timeout ]

In the default mode, the script attempts to perform the following actions:

  • Open the project and acquire the id for the last job in that project
  • Submit a GetDescription request asking for the descriptions for a list of jobs consisting of that job repeated one hundred times, and check that the number of descriptions returned is equal to the number of job ids.
  • Increase the list of jobs by 100 and repeat until failure occurs.

The script reports the size (in terms of numbers of jobs) of the failing request.

If -limit is specified then the script tries a single request with a list of <n> jobs, and reports whether or not this was successfully processed.

Notes:

This test is of limited use in ascertaining the maximum size of requests that the system can deal with.

  • In either mode the maximum number of jobs is not a precise value, and may vary depending on the length of the data values associated with the job that is chosen for the test - essentially it is a ballpark indication only.
  • The upper limit is also dependent on the timeout that is chosen for the client API; for large requests the default timeout of 5000ms is too short for the transaction to be completed. This can lead to discrepancies in the maximum number of jobs obtained using the two different modes.
    Specifying the -adjust_timeout option attempts to adjust the timeout period to compensate.
    It is possible that other factors such as machine load might also make a difference to the maximum request size observed.

For reference: with the default timeout in the iterative mode, the maximum request size on one particular machine. is around 16300 jobs. In the same mode with the adjusted timeout option, the maximum request size obtained is significantly higher (41400 jobs).

#1921
Script#7 test_dbxml.tcl

Loads the functions from dbxml.tcl in the ClientAPI directory and attempts various operations to test the functions.

Usage: tclsh test_dbxml.tcl

The script attempts to perform the following actions:

  • Wraps a number of requests into XML using WrapDbRequest
  • Unwraps a number of XML responses using UnwrapDbResponse
  • Unwraps an XML broadcast using UnwrapDbBroadcast

Note that this script tests the functions directly without invoking the handler or the client API.

Not assigned
Script#8 ccp4icompat_client.tcl

Attempts to simulate the effect of a second process (e.g. CCP4i) interfering with the project lock and database files while the handler is also accessing the project.

Usage: tclsh ccp4icompat_client.tcl <project_name>

The script attempts to perform the following actions:

  • Open the database for the specified project
  • Override the lock on the project database
  • Update the modification time on the project database

At each step the script attempts to get data from the project to verify that the behaviour is as expected.

#2089
Script#9 demanding_client.tcl

Attempts to simulate the effect of a client that generates large numbers of new jobs and other changes to a project in a short space of time (for example, an automated script).

Usage: tclsh demanding_client.tcl <project_name> <task_name>

The script attempts to perform the following actions:

  • Open the database for the specified project
  • Creates ten new jobs with the specified taskname and modifies the data associated with each

The script can be edited to adjust the time interval between the creation of new jobs. Multiple instances of the script can be run simultaneously to test the effect of multiple demanding clients, e.g.:

#!/bin/sh -e
#
# Run many instances of the demanding client
tclsh demanding_client.tcl JUNK fake1 &
tclsh demanding_client.tcl JUNK fake2 &
tclsh demanding_client.tcl JUNK fake3 &
tclsh demanding_client.tcl JUNK fake4 &
tclsh demanding_client.tcl JUNK fake5 &
tclsh demanding_client.tcl JUNK fake6 &

Warning this script is very messy - it will leave large numbers of "fake" jobs in the project that it is run in. Open the project in CCP4i and use the "Delete/Archive..." options to remove this jobs in bulk.

Not assigned

Python Test Modules

The modules are in the $BIOXHIT_TOP/dbccp4i/test/ directory.

Module name Description Usage
ccp4ipy_tests.py Test functions for the ccp4i.py module

Usage:

% python ccp4ipy_tests.py
dbsocket_crash.py

This script simulates a client application that is designed to crash. In this situation the Python client API should also terminate.

(It creates a test situation for reproducing the conditions for bug #2003)

Usage:

% python dbsocket_crash.py <project> <job_id>
test_addfile.py

This script simulates a client application adding an input file name to a job multiple times. This shouldn't cause a proble,

(It creates a test situation for reproducing the conditions for bug #2662)

Usage:

% python test_addfile.py <project> <job_id> <filename>
test_ccp4i.py

Test functions for the ccp4i.py module built using the Python unittest module.

This is a fairly comprehensive set of tests of the directories and database classes for a variety of situations.

Usage:

% python test_ccp4i.py
test_dbsocket.py

Test XML wrapping and unwrapping functions for the dbsocket.py module, built using the Python unittest module.

Usage:

% python test_dbsocket.py

CVS Id: $Id: tests.html,v 1.14 2007/10/10 15:16:07 pjx Exp $