dbCCP4i.tcl: Tcl Database Handler Description

Version of 17th February 2006

Background
dbCCP4i usage
Available commands

0. Background

A single server runs at any one time (for a given user). If you try to start a second instance then the new process will hand you off to the first and close down.

Once the server is running you can send requests to it. These requests are wrapped in XML of the form:

<db_request>
<command>command</command>
<argument>arg1</argument>
<argument>arg2</argument>
...
<request_id>id</request_id>
</db_request>

The args are specific to the command. The id is some string without whitespace that the connecting application can use to match up the responses that the server sends back.

The server response is in XML of the form:

<db_response>
<status>status</status>
<result>message/result</result>
<response_id>id</response_id>
</db_response>

The response elements are:

status	The status reflects the success or failure of the request being processed by the handler. It is possible for the handler to return status OK even though the request itself may result in a null value or error for the command. Usually the status will be "OK" or "ERROR"
message/result	Dependent on the command - for example, if the command was a request for data then this will hold the actual data. If the status is ERROR then this will probably be an error message.
id	The id string supplied by the client when making the request

Note: the status reflects the success or failure of the request being processed by the handler. It is possible for the handler to return status OK even though the request itself may result in a null value or error for the command.

The server can also send "broadcast messages" to clients. Broadcasts are not in direct response to a client request, they are triggered by a change in the database status and are intended to notify clients that a change has taken place (for example, if a new job has been registered). The broadcasts are also in XML, of the form:

<db_broadcast>
<message>message</message>
</db_broadcast>

where message is a text string detailing the changes.

1. dbCCP4i usage

dbCCP4i supports two command line options:

Usage: dbccp4i [-port <no>]
       dbccp4i -shutdown

dbccp4i starts up an instance of the server, which then opens up a port to listen for connections. Specifying the -port option tells the server to send back a notification saying which port it is using. The message is the string of the form "Server started on port <portno>".

dbccp4i -shutdown will shut down any server that is currently running, and then close down itself.

2. Available commands

2.0 Background

As described above, commands are sent to the server in the form of XML wrapping. The following sections list the commands without the XML, in the form:

command arg1 arg2 ...

2.1 Commands for interacting with the handler

DbRegister <user> <pid>: Register client with the server (essentially, log in). This needs to be done before any other commands can be issued.
DbRequestStatus: Get the status of the handler. Should return "Active".
DbDisconnect: Disconnect client from server (essentially, log out).
DbShutdown: Issue command to shut down the server.

2.2 Commands for manipulating project data

DbOpen <project> <db_dir> [-grablock]: Open a project with alias <project> and CCP4_DATABASE directory <db_dir>. This will fail if the project is currently locked, either by another client also connected to the handler ("internal lock") or by some other process ("external lock" or "file lock"), unless the -grablock option is also specified.

If the project doesn't exist then this command will create a new project. (This is probably undesirable and an additional command should be added instead.)
DbClose <project>: Close an open project with alias <project>.
DbNewRecord <project>: Create a new record (=job) in <project>, and return the record id.
DbItemExists <project> <n> <item>: Check that <item> exists for record <n> in <project>.
DbGetData <project> <n> <item> [<item2> [ ... ]]: Return the value(s) of one or more data <item>s from record <n> in <project>.
DbSetData <project> <n> <item> <value> [<item2> <value2> [ ... ]]: Set the <value>(s) of one or more <item>(s) in record <n> in <project>.
DbSelectRecord <project> <item> <pattern>: Return a list of record ids in <project> for which the regular expression <pattern> matches the value of <item>.
DbDeleteRecord <project> <n>: Remove record <n> from <project>.
DbReturnRecord <project> <record_list> <item_list> <format>: Given a list of record ids <record_list>, return a list of descriptions which are generated from the supplied lists of data items in <item_list> and <format>.

2.3 Commands for handling running jobs

This is a peculiarity of CCP4i at present. Once started, running jobs talk directly to the handler rather than to the main CCP4i process in order to report progress and update the record that they are associated with.

The procedure is: the main CCP4i process "opens" the record in question for writing purposes, using DbRegisterJob. The running job connects to the handler each time it needs to update the database, using DbReceive. When a job sends a termination notice then the handler "closes" the record and notifies the original CCP4i process so that it can perform clean up or review.

There are obviously some issues here, for example: 1) what happens to records that are never closed (e.g. running job fails without informing the handler)? 2) what happens if the initiating CCP4i process has finished in the meantime?

So this setup is not ideal and should probably be overhauled in future.

DbRegisterJob <project> <n>: Register record <n> in <project> as being "open", so that new processes can access and write to that record.
DbReceive UpdateRecord|AddOutputFile <project> <n> args ...: A request from a running job to update the record for an "open" job. This can be a request to update the status of the running job, or to add additional output files.