| | |
- array
- database
-
- projectDB
- subjobDB
- directories
- history
-
- extendedhistory
- lockfile
-
- dblockfile
class array |
| |
CCP4i array class.
The array class provides functionality for dealing with CCP4i
arrays, which are typically loaded from and save to .def files.
It includes methods for readinng and writing CCP4i .def files,
and allowing manipulation of data inbetween.
An array consists of 'parameters', which have associated values
and may optionally also have associated types. Parameters can
be either 'scalar' (in which case the parameter name is a string
consisting of alphanumeric characters and the underscore) or
'indexed' (in which case the parameter name consists of a string
followed by a comma an then an index component).
Conventionally parameter names are uppercased. Examples of
scalar parameters include: NRUNS HKLIN1 REFINE_MODE.
Examples of indexed parameters include: RUN,0 DERIV,4 ATNAM,1_5
The index component consists of either a single integer (single
index), or a pair of integers separated by an underscore (double
index).
In addition to parameters, the array object also keeps track of
'paramater prototypes' (also just referred to as 'prototypes').
For scalar parameters, the prototype has the same name as the
parameter and there is essentially no difference between the two.
For indexed parameters, the prototype is the zero'th indexed
equivalent. For example: for the parameter DERIV,4 the
parameter prototype would be DERIV,0 while for ATNAM,1_5 it
would be ATNAM,0 (NB not ATNAM,0_0). RUN,0 is already a
parameter prototype.
Parameter prototypes are significant for indexed parameters for
two reasons:
1. The type of an indexed parameter is attached to the prototype
2. The value attached to the prototype is usually used as a
default value for new indexed parameters based on that
prototype.
Prototypes are significant for all parameters because only
parameters that have prototypes are written to the output
.def file.
|
| |
Methods defined here:
- __contains__(self, key)
- __delitem__(self, key)
- __getitem__(self, key)
- __init__(self, initfile='')
- __setitem__(self, key, value)
- addparameter(self, parameter, index='', type='', value='')
- Add a new parameter to the array object.
This method is used to add parameters to the array
object. If the parameter doesn't exist then a new entry
will be created to store the specified 'value'.
If 'parameter' is a scalar parameter, or if it is a zero
indexed parameter (either explicitly, or when 'index' is
set to zero), then it will also be added to the list of
parameter prototypes (if the prototype doesn't already
exist). The 'type' will be associated with the new
prototype definition.
NB: indexed parameters that are not already parameter
prototypes will NOT have prototypes added for them - this
may be significant if the array object is saved to file,
since by default parameters with no prototype are not
written out.
- delparameter(self, parameter, *indices)
- getequivalent(self, parameter1, parameter2)
- getheaderdata(self, pattern)
- # Methods for extracting data from the header
- getvalue(self, parameter, *indices)
- has_parameter(self, parameter, *indices)
- incr(self, parameter, increment=1)
- isparameter(self, parameter)
- listindexed(self, parameter, include_prototype=False)
- newvalue(self, parameter, *indices)
- parameter_index(self, parameter)
- parameter_isindexed(self, parameter)
- parameter_root(self, parameter)
- parameters(self)
- read(self, filename)
- Reads the contents of a def-format file into the array.
The file format can be either key-value pairs, or
key-type-value triplets. If a value in the file is of the
form '\$name' then 'name' is assumed to be an environment
variable which is substituted via a call to the GetEnvVar
function.
- replacevalue(self, parameter, *indices)
- setvalue(self, parameter, *indices)
- typeof(self, parameter)
- write(self, filename, application='', version='', defname='', user='', project='', include_types=False, include_prototypes=False, include_nonprototyped=False)
- Write the contents of the array object to a def file.
Optionally a number of values can be provided which will be
written into the header lines of the def file, specifically:
'application' and 'version' specify a name and version number
for the application writing the file, i.e.:
#CCP4I VERSION <application> <version>
The application name defaults to 'CCP4iPy' if none is supplied.
'defname' specifies the name of the file, i.e.:
#CCP4I SCRIPT DEF <defname>
If defname is not supplied then it defaults to the basename
of the file being written out.
'user' is the user name, i.e.:
#CCP4I USER <user>
If this is not supplied then it defaults to the value returned
by the GetUserId function.
'project' is the name of a CCP4 project, i.e.:
#CCP4I PROJECT <project>
If this is not supplied then this header line will not appear.
Additionally, writing the actual parameters to the file are
affected by a number of optional 'include_...' arguments:
include_types: if True then the type information will
also be written to the file; by default
type information is not written.
include_prototypes: if True then the parameter prototypes
for indexed parameters (i.e. the
0th-indexed parameters) will also be
written; by default the prototypes for
indexed parameters are not written.
include_nonprototyped: if True then all stored parameters will
be written; by default only parameters
with prototypes are written out.
- writeparameter(self, parameter, f, include_types=False)
- Write an array parameter to a def file.
The calling function must supply the complete parameter name (i.e.
including index, if appropriate) and an initialised file object f.
If include_types is True then the parameter type is also written.
|
class database |
| |
CCP4i def file project database class.
Creates an instance of a CCP4i project database object associated
with a particular project name and a project directory. If the
project doesn't exist then the create method must be used to make
a new project database.def file; use the open method to load
data from an existing database.
|
| |
Methods defined here:
- __del__(self)
- del(project)
Saves data, releases lock and deletes the object.
- __init__(self, project, directory, dbdir='', template='')
- Internal: initialise a new project database object.
The name of the project and the path to the corresponding
project directory must be supplied.
Optionally a database directory can be specified using the
'dbdir' argument - this specifies where the object will look
for or write the persistent storage.
By default the data items in the file are taken from the
'template' def file in $CCP4/ccp4i/etc; this can be over-ridden
by specifying a template def file via the optional 'template'
argument.
No data is loaded until either the open or create methods
are invoked.
- __len__(self)
- len(project)
Returns the number of jobs stored in the project, or
zero if the project is not loaded.
- __nonzero__(self)
- Returns True if the project is loaded, False otherwise.
- __repr__(self)
- Returns the project name.
- addfile(self, job, ftype, dirs, filename, alias)
- Add a file to the list of files for a job.
The job is specified by an id number. 'ftype' specifies
whether the file should be associated as an 'input' or
'output' file.
'filename' specifies the name of the file to be added
to the can be either a full path or a relative path,
and 'alias' specifies a project alias to be associated
with the file. The alias is only used to resolve
relative paths in the filename, and is ignored if a
full path is supplied for the file being added.
'dirs' is a ccp4i directories object.
Returns True if the operation was successful, False
if there was a problem. Note that if the file name
already appears in the list of files then this is
not an error.
- addinputfile(self, job, dirs, filename, alias)
- Add a file to the list of input files for a job.
This function wraps the addfile method: 'job' is a
job number, 'dirs' is a directories object, 'filename'
is an absolute or relative path, and 'alias' is an
optional project name which is used to resolve
relative paths but is otherwise ignored.
The method returns True on success and False otherwise.
- addoutputfile(self, job, dirs, filename, alias)
- Add a file to the list of output files for a job.
This function wraps the addfile method: 'job' is a
job number, 'dirs' is a directories object, 'filename'
is an absolute or relative path, and 'alias' is an
optional project name which is used to resolve
relative paths but is otherwise ignored.
The method returns True on success and False otherwise.
- auto_refresh(self)
- Return the value of the auto_refresh flag.
The 'auto_refresh' feature means that read attempts on a
'stale' database automatically cause the data to be
refreshed from file in read-only mode. See the
set_auto_refresh method for more information.
- close(self)
- Close an open project.
This saves any unsaved data to file, releases the lock on the
database.def file, and clears the data from the project object,
returning True on success.
The project must have first been loaded via an open or create
operation, otherwise False is returned.
- create(self)
- Make a new project.
Creating a new project includes creating the project directory
(if it doesn't already exist), and creating the database
subdirectory and database.def file. The project is then opened
and the project object is returned.
The create operation will fail (returning False) if the project
already exists.
- deletejob(self, job)
- Remove an existing job record from the project.
The job with the specified id number will be removed from the
project, also removing all associated data, returning True on
success. The operation will fail if the database is not loaded,
if the object has lost the lock on the database.def file, or
if a job with the specified id number is not found.
- describe(self, joblist, itemlist, formatlist)
- Return a list of formatted strings populated with job data.
For each job id number in the joblist, the describe method
retrieves the values of each of the data items in the itemlist
and creates a description string by placing those values into
fields with character widths supplied in the formatlist.
It returns a list of these strings, with one list item per job.
Allowed data items can be any stored in the project database
plus JOB_ID (the id number of the job), which is an implicit
data item.
- describejob(self, job, itemlist, formatlist)
- Return a formatted string populated with job data.
For the specified job id number, the describejob method
retrieves the values of each of the data items in the itemlist
and creates a description string by placing those values into
fields with character widths supplied in the formatlist.
Allowed data items can be any stored in the project database
plus JOB_ID (the id number of the job), which is an implicit
data item.
- exists(self)
- Test if the project exists.
Returns True if the project directory and database.def file both
exist, and False otherwise.
- getDbFile(self)
- Return the database file for the project.
- getDbItems(self)
- Return a list of the data items stored for all jobs.
- getDbdir(self)
- Return the database directory of the project.
- getProjectname(self)
- Return the name of the project.
- getdata(self, job, item)
- Retrieve the value of a data item stored for a job.
The job is specified by an id number. If the job doesn't
exist, or if the item is otherwise inaccessible as
indicated by a call to the itemexists method, then the
operation raises an IndexError exception.
Otherwise, the specific value of the data item for the
specified job id is returned (or the 'generic' value,
if a specific value was not found).
If the database is not readable then 'None' is returned.
- getmessage(self)
- Retrieve last internal error message.
If an operation fails then the client application can
access the last error message using this method. Invoking
getmessage clears the error message.
- hasjob(self, job)
- Checks if a job with the specified id exists in the database.
Given a job id, returns True if the database contains a job
with that id and False if not.
- haslock(self)
- Test whether the project is locked by this process.
Returns True if the project database.def file has a lockfile
owned by this process, and False otherwise.
- isloaded(self)
- Test whether the project data has been loaded.
Returns True if the database object has been populated from
the file on disk.
- islocked(self)
- Test whether the project is locked.
Returns True if the project database.def file has a lockfile
from any process, and False otherwise.
- isreadable(self)
- Test if it is possible to get data from the object.
Returns True if the database object has been loaded,
and has data that is at least as recent as the persistent
storage on disk.
- isstale(self)
- Check if the data in the object is older than on disk.
Return True if the modification time of the resource
file is more recent than the last save.
If so then this suggests that the data in this object may
not accurately mirror the data in the persistent
storage.
- iswriteable(self)
- Test if it is possible to modify and save the data.
Returns True if the database object has been loaded, owns
the lock on the persistent storage on disk, and has data
that is at least as recent as the persistent storage.
- itemexists(self, job, item)
- Check for the existence of a data item for a particular job.
Returns True if the data item is found for the specified job
id number. If the item is not found for the specified job id,
but was defined in the template database.def file, then this
method also returns True. Otherwise, itemexists returns False,
indicating that the item is not accessible for the specified
job.
itemexists will also return False in the event that the project
is not loaded.
- listfiles(self, job, ftype, dirs)
- Return a list of files for a job.
The job is specified by an id number. 'ftype' specifies
whether the file list should be the associated 'input' or
'output' files.
'dirs' is a ccp4i directories object.
Returns a list with the full filenames for each of the
files associated with the job, or a blank list if there
are no files.
- listinputfiles(self, job, dirs)
- Return a list of input files for a job.
The job is specified by an id number,'dirs' is a ccp4i
directories object.
Returns a list with the full filenames for each of the
files associated with the job, or a blank list if there
are no files.
- listjobs(self)
- Return the list of all jobs in the project.
The list is an unsorted list of job id numbers, or an empty list
if the project is not currently loaded.
- listoutputfiles(self, job, dirs)
- Return a list of output files for a job.
The job is specified by an id number,'dirs' is a ccp4i
directories object.
Returns a list with the full filenames for each of the
files associated with the job, or a blank list if there
are no files.
- newjob(self, taskname='', status='STARTING', title='')
- Create a new job record in the project.
Creates a new entry in the current project with the current date
and returns the id number for the new job. Optionally the
taskname (TASKNAME data item), the status (STATUS item) and/or
title (TITLE item) may also be set when the job is created.
If no taskname is specified then it is set to 'none', if no status
is specified then it is set to 'STARTING'. (Note that blank values
of the taskname can crash CCP4i and cause corruption of the
database.)
The operation will fail and return -1 if project is not loaded,
or if the current object has lost the lock on the database.def
file.
- njobs(self)
- Return the value of the NJOBS data item.
For historical reasons, the NJOBS data item records the current
highest job id number used in the project, and is not generally
the actual number of jobs in the project. To get the actual
number of jobs in an object project, do len(project).
Returns -1 if the project has not been correctly loaded.
- open(self, grablock=False, readonly=False, strict=False)
- Open a project.
Read the data from the project database and return the project
object.
The open operation will fail (return False) if the project is
locked, if the project is already loaded, or if the project
doesn't exist. Set the 'grablock' argument to True to override
any existing lock and force loading of the database.
Specifying 'readonly' as True populates the database object
from the file on disk without attempting to lock the resource.
However it will clear an existing lock, so the process may
still own the lock even if it is in 'read-only' mode.
If the 'strict' flag is true then header information about
the project name and database directory are checked against
the internal values, and warnings issued if there is a
discrepancy.
- refresh(self, grablock=False, readonly=True)
- Reload the data from the file into a database object.
Use the 'refresh' method to re-read the data from the
file on disk into an existing (loaded) database object.
This is necessary for example if the file is updated
by an external process.
Essentially the database object is closed and then
reopened. By default the data is reloaded in 'readonly'
mode. Set the 'grablock' argument to True to override
any existing lock and force loading of the database in
write mode - see the 'open' method.
- removefile(self, job, ftype, dirs, filename)
- Remove a file from the list of files for a job.
The job is specified by an id number. 'ftype' specifies
whether the file should be associated as an 'input' or
'output' file.
'dirs' is a ccp4i directories object.
Returns True if the operation was successful, False
if there was a problem.
- removeinputfile(self, job, dirs, filename)
- Remove a file from the list of input files for a job.
The job is specified by an id number. 'dirs' is a ccp4i
directories object.
Returns True if the operation was successful, False
if there was a problem.
- removeoutputfile(self, job, dirs, filename)
- Remove a file from the list of output files for a job.
The job is specified by an id number. 'dirs' is a ccp4i
directories object.
Returns True if the operation was successful, False
if there was a problem.
- save(self)
- Save the project database contents to file.
The save method writes the data in the object to persistent
storage (i.e. the database.def file for the project).
On successful completion True is returned.
The save operation will fail if the project is not loaded,
or if the object has lost the lock on the database.def file;
in both cases False is returned.
- selectjobs(self, item, pattern)
- Retrieve a list of jobs based on some selection criterion.
Returns a list of jobs for which the value of the given data
item matches the supplied regular expression pattern.
- set_auto_refresh(self, auto_refresh)
- Set the auto_refresh flag.
'auto_refresh' takes a boolean value. Setting it to 'True'
means that a read attempt on a 'stale' database (i.e. when
the source database file is newer than the data in the
object) will automatically invoke the 'refresh' method to
load the data in read-only mode. Setting it to 'False'
means that the 'refresh' method must be explicitly invoked.
- setdata(self, job, item, value)
- Set the value of a data item for the specified job.
The job is specified by an id number. The operation returns
True if the value was successlly updated and False if the
database is not writeable, or if the data item is valid but
could not be updated.
An IndexError exception will be raised if the data item is
invalid, either because the job doesn't exist or because the
data item doesn't exist for the job, as indicated by a call
to the itemexists method.
Note that the validity of the new value is not checked, except
for the TASKNAME data item, which must not be blank.
- updatetime(self, job)
- Update the time of the job to be the current time.
This wraps the setdata method to automatically set the DATE
attribute of the job specified by the id number, and returns
the result of the setdata operation.
|
class dblockfile(lockfile) |
| |
CCP4i database lock file class.
This class provides methods for creating and manipulating lock
files for CCP4i def file based databases.
The calling subprogram should specify the name of the project
(argument 'project') and the path to the database directory
i.e. the location of the database.def file (argument 'dbdir').
|
| |
Methods defined here:
- __init__(self, project, dbdir)
Methods inherited from lockfile:
- filename(self)
- haslock(self)
- islocked(self)
- lock(self)
- resource(self)
- unlock(self, force=False)
|
class directories |
| |
CCP4i project directories class.
This class provides methods for reading and writing the project
directory information in directories.def for CCP4i-based
applications.
|
| |
Methods defined here:
- __del__(self)
- __init__(self, user='', source='')
- __nonzero__(self)
- __repr__(self)
- adddefdirref(self, def_dir, path)
- addprojectref(self, alias, path, db_dir='')
- array(self)
- ### FIXME
### This function provided for diagnostic purposes
- auto_refresh(self)
- Return the value of the auto_refresh flag.
The 'auto_refresh' feature means that read attempts on a
'stale' database automatically cause the data to be
refreshed from file in read-only mode. See the
set_auto_refresh method for more information.
- create(self)
- defdir(self, def_dir)
- deletedefdirref(self, def_dir)
- deleteprojectref(self, alias)
- filenamecomponents(self, filename)
- fullfilename(self, filen, project)
- getmessage(self)
- haslock(self)
- isdefdir(self, def_dir)
- isdefdirdir(self, dir)
- isloaded(self)
- islocked(self)
- islockedbyccp4i(self)
- isproject(self, project)
- isprojectdir(self, dir)
- isreadable(self)
- isstale(self)
- iswriteable(self)
- listdefdirs(self)
- listprojects(self)
- load(self, force=False, readonly=False)
- logdir(self)
- n_defdirs(self)
- n_projects(self)
- project_menu(self)
- projectdb(self, project)
- projectdir(self, project)
- refresh(self, readonly=True)
- release(self)
- save(self, filen='')
- set_auto_refresh(self, auto_refresh)
- Set the auto_refresh flag.
'auto_refresh' takes a boolean value. Setting it to 'True'
means that a read attempt on a 'stale' database (i.e. when
the source database file is newer than the data in the
object) will automatically invoke the 'refresh' method to
load the data in read-only mode. Setting it to 'False'
means that the 'refresh' method must be explicitly invoked.
- setlogdir(self, log_dir)
- source(self)
- user(self)
|
class extendedhistory(history) |
| |
Extended history class for use with projectDB class.
The extendedhistory class is intended for use with the
projectDB class and adds support for querying the history
for subjobs as well as 'top-level' jobs. The same
methods are available as for the base 'history' class
however they can be used for subjobs by specifying
the subjob id using the x.y notation, e.g.
history.parentsof(9)
will return top-level jobs that are 'parents' of job 9,
while
history.parentsof(9.5)
will return subjobs of job 9 that are 'parents' of job
9.5.
Note that top-level jobs will not be returned as being
linked to subjobs (or vice versa); neither will subjobs
that belong to different top-level jobs.
|
| |
Methods defined here:
- __init__(self, database, directories)
- Override initialisation of base class.
This additionally sets up a dictionary for the histories
of each of the subjob databases.
- arelinked(self, job_id1, job_id2)
- Override the test to see if two jobs are linked.
- childrenof(self, job_id)
- Override the listing of child ids for the current job.
- construct(self)
- Override the construction of the history.
This additionally constructs histories for each of the
subjob databases.
- parentsof(self, job_id)
- Override the listing of parent ids for the current job.
Methods inherited from history:
- addlink(self, parent_job, child_job)
- allchildrenof(self, job_id, result_ids)
- allparentsof(self, job_id, result_ids)
- infiles(self, job_id)
- outfiles(self, job_id)
- update(self)
- updatelinks(self, job_id)
|
class history |
| |
CCP4i database history object.
Supplied with a loaded database object and a loaded directories
object, use the construct method to populate the history object.
Use the parentsof and childrenof methods to return a list of
parent/child job id numbers for each job. Use the update method
to find and add new relationships for an existing history.
Use the arelinked method to find out if two job ids are linked.
|
| |
Methods defined here:
- __init__(self, database, directories)
- addlink(self, parent_job, child_job)
- allchildrenof(self, job_id, result_ids)
- allparentsof(self, job_id, result_ids)
- arelinked(self, job_id1, job_id2)
- childrenof(self, job_id)
- construct(self)
- infiles(self, job_id)
- outfiles(self, job_id)
- parentsof(self, job_id)
- update(self)
- updatelinks(self, job_id)
|
class projectDB(database) |
| |
CCP4i project database class.
This extends the base 'database' class to allow subjobs to
be defined and manipulated.
Subjob databases are opened in a 'lazy' fashion i.e. only
when they are needed.
|
| |
Methods defined here:
- __init__(self, project, directory)
- Initialise the project database.
- addfile(self, jobid, ftype, dirs, filename, alias)
- Add a file to the list of files for a job.
Over-ride the 'addfile' method of the parent class
in order to also deal with subjobs.
- addsubjob(self, job, taskname, title, status='STARTING')
- Add a subjob to a job in the project database.
If this is the first subjob to be added to the job,
then a new subjob database will be created first.
The id number of the subjob will be returned.
- buildjobid(self, job, subjob=None)
- Build a single job id given the job and subjob components.
This is a wrapper for the 'MakeJobid' function.
- close(self)
- Close the project database.
This extends the 'close' method of the base class,
by also closing all the opened subjob databases.
- deletejob(self, jobid)
- Delete a subjob from a job in the project database.
Over-ride the 'deletejob' method of the parent class.
'jobid' can be a job or subjob identifier.
- describejob(self, jobid, itemlist, formatlist)
- Return a formatted string populated with job data.
This over-rides the method in the base class in order
to be able to also deal with subjobs.
- getdata(self, jobid, item)
- Retrieve the value of a data item stored for a job.
This over-rides the 'getdata' method in the parent class.
'job' can be either an id for a job in the main database,
or for a subjob.
- getsubjobdb(self, job)
- Return an existing subjobDB object associated with a job.
This returns a subjobDB object associated with a job
id, or 'False' if a subjob database doesn't already
exist for the specified job.
- has_subjobs(self, job)
- Check if a job currently has subjobs defined.
- hasjob(self, jobid)
- Checks if a job with the specified id exists in the database.
Given a job id, returns True if the database contains a job
with that id and False if not.
Overrides the version in the parent class.
- itemexists(self, jobid, item)
- Check for the existence of a data item for a particular job.
This over-rides the 'itemexists' method in the parent class.
'jobid' can be either an id for a job in the main database,
or for a subjob.
- listjobs(self, jobid=None)
- Return a list of jobs or subjobs.
Over-ride the 'listjob' method of the parent class.
If no jobid is supplied then return the list of top-level
jobs in the project; if a jobid is supplied for a top-level
job then return the list of associated subjobs.
- listjobs_withsubjobs(self)
- Return a list of the jobs which also have subjobs.
This could be quite slow for large projects since it
attempts to open subjob databases for every job, to
test for their existence.
Possibly the projectDB object could keep track of the
subjob data internally to speed this up in future.
- opensubjobdb(self, job, create=True)
- Open a subjob database for the specified job id.
Returns an open subjobDB object, or 'False' if no
subjob could be opened.
If the 'create' argument is True then if the subjob
database does exist in the file system then a new
subjob database will be created (this is the default).
To avoid this set the 'create' argument to False.
- removefile(self, jobid, ftype, dirs, filename)
- Remove a file from the list of files for a job or subjob.
Over-ride the 'addfile' method of the parent class
in order to also deal with subjobs.
- save(self)
- Save the project database to file.
This extends the 'save' method of the base class, by
also performing save operations on all of the opened
subjob databases.
- selectsubjobs(self, job, item, pattern)
- Retrieve a list of subjobs based on some selection criterion.
- setdata(self, jobid, item, value)
- Set the value of a data item for the specified job or subjob.
This over-rides the 'setdata' method in the parent class.
'jobid' can be either an id for a job in the main database,
or for a subjob.
- splitjobid(self, jobid)
- Split a job id into job and subjob components.
This is a wrapper for the 'SplitJobid' function.
Methods inherited from database:
- __del__(self)
- del(project)
Saves data, releases lock and deletes the object.
- __len__(self)
- len(project)
Returns the number of jobs stored in the project, or
zero if the project is not loaded.
- __nonzero__(self)
- Returns True if the project is loaded, False otherwise.
- __repr__(self)
- Returns the project name.
- addinputfile(self, job, dirs, filename, alias)
- Add a file to the list of input files for a job.
This function wraps the addfile method: 'job' is a
job number, 'dirs' is a directories object, 'filename'
is an absolute or relative path, and 'alias' is an
optional project name which is used to resolve
relative paths but is otherwise ignored.
The method returns True on success and False otherwise.
- addoutputfile(self, job, dirs, filename, alias)
- Add a file to the list of output files for a job.
This function wraps the addfile method: 'job' is a
job number, 'dirs' is a directories object, 'filename'
is an absolute or relative path, and 'alias' is an
optional project name which is used to resolve
relative paths but is otherwise ignored.
The method returns True on success and False otherwise.
- auto_refresh(self)
- Return the value of the auto_refresh flag.
The 'auto_refresh' feature means that read attempts on a
'stale' database automatically cause the data to be
refreshed from file in read-only mode. See the
set_auto_refresh method for more information.
- create(self)
- Make a new project.
Creating a new project includes creating the project directory
(if it doesn't already exist), and creating the database
subdirectory and database.def file. The project is then opened
and the project object is returned.
The create operation will fail (returning False) if the project
already exists.
- describe(self, joblist, itemlist, formatlist)
- Return a list of formatted strings populated with job data.
For each job id number in the joblist, the describe method
retrieves the values of each of the data items in the itemlist
and creates a description string by placing those values into
fields with character widths supplied in the formatlist.
It returns a list of these strings, with one list item per job.
Allowed data items can be any stored in the project database
plus JOB_ID (the id number of the job), which is an implicit
data item.
- exists(self)
- Test if the project exists.
Returns True if the project directory and database.def file both
exist, and False otherwise.
- getDbFile(self)
- Return the database file for the project.
- getDbItems(self)
- Return a list of the data items stored for all jobs.
- getDbdir(self)
- Return the database directory of the project.
- getProjectname(self)
- Return the name of the project.
- getmessage(self)
- Retrieve last internal error message.
If an operation fails then the client application can
access the last error message using this method. Invoking
getmessage clears the error message.
- haslock(self)
- Test whether the project is locked by this process.
Returns True if the project database.def file has a lockfile
owned by this process, and False otherwise.
- isloaded(self)
- Test whether the project data has been loaded.
Returns True if the database object has been populated from
the file on disk.
- islocked(self)
- Test whether the project is locked.
Returns True if the project database.def file has a lockfile
from any process, and False otherwise.
- isreadable(self)
- Test if it is possible to get data from the object.
Returns True if the database object has been loaded,
and has data that is at least as recent as the persistent
storage on disk.
- isstale(self)
- Check if the data in the object is older than on disk.
Return True if the modification time of the resource
file is more recent than the last save.
If so then this suggests that the data in this object may
not accurately mirror the data in the persistent
storage.
- iswriteable(self)
- Test if it is possible to modify and save the data.
Returns True if the database object has been loaded, owns
the lock on the persistent storage on disk, and has data
that is at least as recent as the persistent storage.
- listfiles(self, job, ftype, dirs)
- Return a list of files for a job.
The job is specified by an id number. 'ftype' specifies
whether the file list should be the associated 'input' or
'output' files.
'dirs' is a ccp4i directories object.
Returns a list with the full filenames for each of the
files associated with the job, or a blank list if there
are no files.
- listinputfiles(self, job, dirs)
- Return a list of input files for a job.
The job is specified by an id number,'dirs' is a ccp4i
directories object.
Returns a list with the full filenames for each of the
files associated with the job, or a blank list if there
are no files.
- listoutputfiles(self, job, dirs)
- Return a list of output files for a job.
The job is specified by an id number,'dirs' is a ccp4i
directories object.
Returns a list with the full filenames for each of the
files associated with the job, or a blank list if there
are no files.
- newjob(self, taskname='', status='STARTING', title='')
- Create a new job record in the project.
Creates a new entry in the current project with the current date
and returns the id number for the new job. Optionally the
taskname (TASKNAME data item), the status (STATUS item) and/or
title (TITLE item) may also be set when the job is created.
If no taskname is specified then it is set to 'none', if no status
is specified then it is set to 'STARTING'. (Note that blank values
of the taskname can crash CCP4i and cause corruption of the
database.)
The operation will fail and return -1 if project is not loaded,
or if the current object has lost the lock on the database.def
file.
- njobs(self)
- Return the value of the NJOBS data item.
For historical reasons, the NJOBS data item records the current
highest job id number used in the project, and is not generally
the actual number of jobs in the project. To get the actual
number of jobs in an object project, do len(project).
Returns -1 if the project has not been correctly loaded.
- open(self, grablock=False, readonly=False, strict=False)
- Open a project.
Read the data from the project database and return the project
object.
The open operation will fail (return False) if the project is
locked, if the project is already loaded, or if the project
doesn't exist. Set the 'grablock' argument to True to override
any existing lock and force loading of the database.
Specifying 'readonly' as True populates the database object
from the file on disk without attempting to lock the resource.
However it will clear an existing lock, so the process may
still own the lock even if it is in 'read-only' mode.
If the 'strict' flag is true then header information about
the project name and database directory are checked against
the internal values, and warnings issued if there is a
discrepancy.
- refresh(self, grablock=False, readonly=True)
- Reload the data from the file into a database object.
Use the 'refresh' method to re-read the data from the
file on disk into an existing (loaded) database object.
This is necessary for example if the file is updated
by an external process.
Essentially the database object is closed and then
reopened. By default the data is reloaded in 'readonly'
mode. Set the 'grablock' argument to True to override
any existing lock and force loading of the database in
write mode - see the 'open' method.
- removeinputfile(self, job, dirs, filename)
- Remove a file from the list of input files for a job.
The job is specified by an id number. 'dirs' is a ccp4i
directories object.
Returns True if the operation was successful, False
if there was a problem.
- removeoutputfile(self, job, dirs, filename)
- Remove a file from the list of output files for a job.
The job is specified by an id number. 'dirs' is a ccp4i
directories object.
Returns True if the operation was successful, False
if there was a problem.
- selectjobs(self, item, pattern)
- Retrieve a list of jobs based on some selection criterion.
Returns a list of jobs for which the value of the given data
item matches the supplied regular expression pattern.
- set_auto_refresh(self, auto_refresh)
- Set the auto_refresh flag.
'auto_refresh' takes a boolean value. Setting it to 'True'
means that a read attempt on a 'stale' database (i.e. when
the source database file is newer than the data in the
object) will automatically invoke the 'refresh' method to
load the data in read-only mode. Setting it to 'False'
means that the 'refresh' method must be explicitly invoked.
- updatetime(self, job)
- Update the time of the job to be the current time.
This wraps the setdata method to automatically set the DATE
attribute of the job specified by the id number, and returns
the result of the setdata operation.
|
class subjobDB(database) |
| |
CCP4i subjob database class.
This stores the job data for substeps defined for a job in
the main project directory.
The subjob data are stored in a subdirectory of the project
database directory, called '<jobid>_database. The name of
the project database directory must be given explicitly as
the 'dbdir' argument.
Note that the subjob databases differ from the base class
databases, in that when a subjob db is closed it will
automatically delete the persistent storage (i.e. the
subdirectory and database.def file) if there are no subjobs
stored at this point.
|
| |
Methods defined here:
- __init__(self, job, taskname, dbdir)
- Initialise the subjob database.
The calling process should include the taskname and job id
number of the main job to which the subjobs belong.
- close(self)
- Close the subjob database.
This extends the 'close' method of the base class.
If there are no subjobs defined at the point of closure
then the subjob database is removed.
- getId(self)
- Return the identifier for the subjob database.
Methods inherited from database:
- __del__(self)
- del(project)
Saves data, releases lock and deletes the object.
- __len__(self)
- len(project)
Returns the number of jobs stored in the project, or
zero if the project is not loaded.
- __nonzero__(self)
- Returns True if the project is loaded, False otherwise.
- __repr__(self)
- Returns the project name.
- addfile(self, job, ftype, dirs, filename, alias)
- Add a file to the list of files for a job.
The job is specified by an id number. 'ftype' specifies
whether the file should be associated as an 'input' or
'output' file.
'filename' specifies the name of the file to be added
to the can be either a full path or a relative path,
and 'alias' specifies a project alias to be associated
with the file. The alias is only used to resolve
relative paths in the filename, and is ignored if a
full path is supplied for the file being added.
'dirs' is a ccp4i directories object.
Returns True if the operation was successful, False
if there was a problem. Note that if the file name
already appears in the list of files then this is
not an error.
- addinputfile(self, job, dirs, filename, alias)
- Add a file to the list of input files for a job.
This function wraps the addfile method: 'job' is a
job number, 'dirs' is a directories object, 'filename'
is an absolute or relative path, and 'alias' is an
optional project name which is used to resolve
relative paths but is otherwise ignored.
The method returns True on success and False otherwise.
- addoutputfile(self, job, dirs, filename, alias)
- Add a file to the list of output files for a job.
This function wraps the addfile method: 'job' is a
job number, 'dirs' is a directories object, 'filename'
is an absolute or relative path, and 'alias' is an
optional project name which is used to resolve
relative paths but is otherwise ignored.
The method returns True on success and False otherwise.
- auto_refresh(self)
- Return the value of the auto_refresh flag.
The 'auto_refresh' feature means that read attempts on a
'stale' database automatically cause the data to be
refreshed from file in read-only mode. See the
set_auto_refresh method for more information.
- create(self)
- Make a new project.
Creating a new project includes creating the project directory
(if it doesn't already exist), and creating the database
subdirectory and database.def file. The project is then opened
and the project object is returned.
The create operation will fail (returning False) if the project
already exists.
- deletejob(self, job)
- Remove an existing job record from the project.
The job with the specified id number will be removed from the
project, also removing all associated data, returning True on
success. The operation will fail if the database is not loaded,
if the object has lost the lock on the database.def file, or
if a job with the specified id number is not found.
- describe(self, joblist, itemlist, formatlist)
- Return a list of formatted strings populated with job data.
For each job id number in the joblist, the describe method
retrieves the values of each of the data items in the itemlist
and creates a description string by placing those values into
fields with character widths supplied in the formatlist.
It returns a list of these strings, with one list item per job.
Allowed data items can be any stored in the project database
plus JOB_ID (the id number of the job), which is an implicit
data item.
- describejob(self, job, itemlist, formatlist)
- Return a formatted string populated with job data.
For the specified job id number, the describejob method
retrieves the values of each of the data items in the itemlist
and creates a description string by placing those values into
fields with character widths supplied in the formatlist.
Allowed data items can be any stored in the project database
plus JOB_ID (the id number of the job), which is an implicit
data item.
- exists(self)
- Test if the project exists.
Returns True if the project directory and database.def file both
exist, and False otherwise.
- getDbFile(self)
- Return the database file for the project.
- getDbItems(self)
- Return a list of the data items stored for all jobs.
- getDbdir(self)
- Return the database directory of the project.
- getProjectname(self)
- Return the name of the project.
- getdata(self, job, item)
- Retrieve the value of a data item stored for a job.
The job is specified by an id number. If the job doesn't
exist, or if the item is otherwise inaccessible as
indicated by a call to the itemexists method, then the
operation raises an IndexError exception.
Otherwise, the specific value of the data item for the
specified job id is returned (or the 'generic' value,
if a specific value was not found).
If the database is not readable then 'None' is returned.
- getmessage(self)
- Retrieve last internal error message.
If an operation fails then the client application can
access the last error message using this method. Invoking
getmessage clears the error message.
- hasjob(self, job)
- Checks if a job with the specified id exists in the database.
Given a job id, returns True if the database contains a job
with that id and False if not.
- haslock(self)
- Test whether the project is locked by this process.
Returns True if the project database.def file has a lockfile
owned by this process, and False otherwise.
- isloaded(self)
- Test whether the project data has been loaded.
Returns True if the database object has been populated from
the file on disk.
- islocked(self)
- Test whether the project is locked.
Returns True if the project database.def file has a lockfile
from any process, and False otherwise.
- isreadable(self)
- Test if it is possible to get data from the object.
Returns True if the database object has been loaded,
and has data that is at least as recent as the persistent
storage on disk.
- isstale(self)
- Check if the data in the object is older than on disk.
Return True if the modification time of the resource
file is more recent than the last save.
If so then this suggests that the data in this object may
not accurately mirror the data in the persistent
storage.
- iswriteable(self)
- Test if it is possible to modify and save the data.
Returns True if the database object has been loaded, owns
the lock on the persistent storage on disk, and has data
that is at least as recent as the persistent storage.
- itemexists(self, job, item)
- Check for the existence of a data item for a particular job.
Returns True if the data item is found for the specified job
id number. If the item is not found for the specified job id,
but was defined in the template database.def file, then this
method also returns True. Otherwise, itemexists returns False,
indicating that the item is not accessible for the specified
job.
itemexists will also return False in the event that the project
is not loaded.
- listfiles(self, job, ftype, dirs)
- Return a list of files for a job.
The job is specified by an id number. 'ftype' specifies
whether the file list should be the associated 'input' or
'output' files.
'dirs' is a ccp4i directories object.
Returns a list with the full filenames for each of the
files associated with the job, or a blank list if there
are no files.
- listinputfiles(self, job, dirs)
- Return a list of input files for a job.
The job is specified by an id number,'dirs' is a ccp4i
directories object.
Returns a list with the full filenames for each of the
files associated with the job, or a blank list if there
are no files.
- listjobs(self)
- Return the list of all jobs in the project.
The list is an unsorted list of job id numbers, or an empty list
if the project is not currently loaded.
- listoutputfiles(self, job, dirs)
- Return a list of output files for a job.
The job is specified by an id number,'dirs' is a ccp4i
directories object.
Returns a list with the full filenames for each of the
files associated with the job, or a blank list if there
are no files.
- newjob(self, taskname='', status='STARTING', title='')
- Create a new job record in the project.
Creates a new entry in the current project with the current date
and returns the id number for the new job. Optionally the
taskname (TASKNAME data item), the status (STATUS item) and/or
title (TITLE item) may also be set when the job is created.
If no taskname is specified then it is set to 'none', if no status
is specified then it is set to 'STARTING'. (Note that blank values
of the taskname can crash CCP4i and cause corruption of the
database.)
The operation will fail and return -1 if project is not loaded,
or if the current object has lost the lock on the database.def
file.
- njobs(self)
- Return the value of the NJOBS data item.
For historical reasons, the NJOBS data item records the current
highest job id number used in the project, and is not generally
the actual number of jobs in the project. To get the actual
number of jobs in an object project, do len(project).
Returns -1 if the project has not been correctly loaded.
- open(self, grablock=False, readonly=False, strict=False)
- Open a project.
Read the data from the project database and return the project
object.
The open operation will fail (return False) if the project is
locked, if the project is already loaded, or if the project
doesn't exist. Set the 'grablock' argument to True to override
any existing lock and force loading of the database.
Specifying 'readonly' as True populates the database object
from the file on disk without attempting to lock the resource.
However it will clear an existing lock, so the process may
still own the lock even if it is in 'read-only' mode.
If the 'strict' flag is true then header information about
the project name and database directory are checked against
the internal values, and warnings issued if there is a
discrepancy.
- refresh(self, grablock=False, readonly=True)
- Reload the data from the file into a database object.
Use the 'refresh' method to re-read the data from the
file on disk into an existing (loaded) database object.
This is necessary for example if the file is updated
by an external process.
Essentially the database object is closed and then
reopened. By default the data is reloaded in 'readonly'
mode. Set the 'grablock' argument to True to override
any existing lock and force loading of the database in
write mode - see the 'open' method.
- removefile(self, job, ftype, dirs, filename)
- Remove a file from the list of files for a job.
The job is specified by an id number. 'ftype' specifies
whether the file should be associated as an 'input' or
'output' file.
'dirs' is a ccp4i directories object.
Returns True if the operation was successful, False
if there was a problem.
- removeinputfile(self, job, dirs, filename)
- Remove a file from the list of input files for a job.
The job is specified by an id number. 'dirs' is a ccp4i
directories object.
Returns True if the operation was successful, False
if there was a problem.
- removeoutputfile(self, job, dirs, filename)
- Remove a file from the list of output files for a job.
The job is specified by an id number. 'dirs' is a ccp4i
directories object.
Returns True if the operation was successful, False
if there was a problem.
- save(self)
- Save the project database contents to file.
The save method writes the data in the object to persistent
storage (i.e. the database.def file for the project).
On successful completion True is returned.
The save operation will fail if the project is not loaded,
or if the object has lost the lock on the database.def file;
in both cases False is returned.
- selectjobs(self, item, pattern)
- Retrieve a list of jobs based on some selection criterion.
Returns a list of jobs for which the value of the given data
item matches the supplied regular expression pattern.
- set_auto_refresh(self, auto_refresh)
- Set the auto_refresh flag.
'auto_refresh' takes a boolean value. Setting it to 'True'
means that a read attempt on a 'stale' database (i.e. when
the source database file is newer than the data in the
object) will automatically invoke the 'refresh' method to
load the data in read-only mode. Setting it to 'False'
means that the 'refresh' method must be explicitly invoked.
- setdata(self, job, item, value)
- Set the value of a data item for the specified job.
The job is specified by an id number. The operation returns
True if the value was successlly updated and False if the
database is not writeable, or if the data item is valid but
could not be updated.
An IndexError exception will be raised if the data item is
invalid, either because the job doesn't exist or because the
data item doesn't exist for the job, as indicated by a call
to the itemexists method.
Note that the validity of the new value is not checked, except
for the TASKNAME data item, which must not be blank.
- updatetime(self, job)
- Update the time of the job to be the current time.
This wraps the setdata method to automatically set the DATE
attribute of the job specified by the id number, and returns
the result of the setdata operation.
| |