| | |
- array
- database
-
- projectDB
- subjobDB
- directories
- history
-
- extendedhistory
- lockfile
-
- dblockfile
class array |
| |
CCP4i array class.
Read and write CCP4i .def files and allow manipulation of data.
|
| |
Methods defined here:
- __contains__(self, key)
- __delitem__(self, key)
- __getitem__(self, key)
- __init__(self, initfile='')
- __setitem__(self, key, value)
- addparameter(self, parameter, index='', type='', value='')
- delparameter(self, parameter, *indices)
- getequivalent(self, parameter1, parameter2)
- getheaderdata(self, pattern)
- # Methods for extracting data from the header
- getvalue(self, parameter, *indices)
- has_parameter(self, parameter, *indices)
- incr(self, parameter, increment=1)
- isparameter(self, parameter)
- listindexed(self, parameter, include_prototype=False)
- newvalue(self, parameter, *indices)
- parameter_index(self, parameter)
- parameter_isindexed(self, parameter)
- parameter_root(self, parameter)
- parameters(self)
- read(self, filename)
- Reads the contents of a def-format file into the array.
The file format can be either key-value pairs, or
key-type-value triplets. If a value in the file is of the
form '\$name' then 'name' is assumed to be an environment
variable which is substituted via a call to the GetEnvVar
function.
- replacevalue(self, parameter, *indices)
- setvalue(self, parameter, *indices)
- typeof(self, parameter)
- write(self, filename, application='', version='', defname='', user='', project='', include_types=False, include_prototypes=False)
- Write the contents of the array object to a def file.
If include_types is True then the type information will also be
written to the file. If include_prototypes is True then the
parameter prototypes for indexed parameters (i.e. the 0th-indexed
parameters) will also be written.
- writeparameter(self, parameter, f, include_types=False)
- Write an array parameter to a def file.
The calling function must supply the complete parameter name (i.e.
including index, if appropriate) and an initialised file object f.
If include_types is True then the parameter type is also written.
|
class database |
| |
CCP4i def file project database class.
Creates an instance of a CCP4i project database object associated
with a particular project name and a project directory. If the
project doesn't exist then the create method must be used to make
a new project database.def file; use the open method to load
data from an existing database.
|
| |
Methods defined here:
- __del__(self)
- del(project)
Saves data, releases lock and deletes the object.
- __init__(self, project, directory, dbdir='', template='')
- Internal: initialise a new project database object.
The name of the project and the path to the corresponding
project directory must be supplied.
Optionally a database directory can be specified using the
'dbdir' argument - this specifies where the object will look
for or write the persistent storage.
By default the data items in the file are taken from the
'template' def file in $CCP4/ccp4i/etc; this can be over-ridden
by specifying a template def file via the optional 'template'
argument.
No data is loaded until either the open or create methods
are invoked.
- __len__(self)
- len(project)
Returns the number of jobs stored in the project, or
zero if the project is not loaded.
- __nonzero__(self)
- Returns True if the project is loaded, False otherwise.
- __repr__(self)
- Returns the project name.
- addfile(self, job, ftype, dirs, filename, alias)
- Add a file to the list of files for a job.
The job is specified by an id number. 'ftype' specifies
whether the file should be associated as an 'input' or
'output' file.
'filename' specifies the name of the file to be added
to the can be either a full path or a relative path,
and 'alias' specifies a project alias to be associated
with the file. The alias is only used to resolve
relative paths in the filename, and is ignored if a
full path is supplied for the file being added.
'dirs' is a ccp4i directories object.
Returns True if the operation was successful, False
if there was a problem. Note that if the file name
already appears in the list of files then this is
not an error.
- addinputfile(self, job, dirs, filename, alias)
- Add a file to the list of input files for a job.
This function wraps the addfile method: 'job' is a
job number, 'dirs' is a directories object, 'filename'
is an absolute or relative path, and 'alias' is an
optional project name which is used to resolve
relative paths but is otherwise ignored.
The method returns True on success and False otherwise.
- addoutputfile(self, job, dirs, filename, alias)
- Add a file to the list of output files for a job.
This function wraps the addfile method: 'job' is a
job number, 'dirs' is a directories object, 'filename'
is an absolute or relative path, and 'alias' is an
optional project name which is used to resolve
relative paths but is otherwise ignored.
The method returns True on success and False otherwise.
- auto_refresh(self)
- Return the value of the auto_refresh flag.
The 'auto_refresh' feature means that read attempts on a
'stale' database automatically cause the data to be
refreshed from file in read-only mode. See the
set_auto_refresh method for more information.
- close(self)
- Close an open project.
This saves any unsaved data to file, releases the lock on the
database.def file, and clears the data from the project object,
returning True on success.
The project must have first been loaded via an open or create
operation, otherwise False is returned.
- create(self)
- Make a new project.
Creating a new project includes creating the project directory
(if it doesn't already exist), and creating the database
subdirectory and database.def file. The project is then opened
and the project object is returned.
The create operation will fail (returning False) if the project
already exists.
- deletejob(self, job)
- Remove an existing job record from the project.
The job with the specified id number will be removed from the
project, also removing all associated data, returning True on
success. The operation will fail if the database is not loaded,
if the object has lost the lock on the database.def file, or
if a job with the specified id number is not found.
- describe(self, joblist, itemlist, formatlist)
- Return a list of formatted strings populated with job data.
For each job id number in the joblist, the describe method
retrieves the values of each of the data items in the itemlist
and creates a description string by placing those values into
fields with character widths supplied in the formatlist.
It returns a list of these strings, with one list item per job.
Allowed data items can be any stored in the project database
plus JOB_ID (the id number of the job), which is an implicit
data item.
- describejob(self, job, itemlist, formatlist)
- Return a formatted string populated with job data.
For the specified job id number, the describejob method
retrieves the values of each of the data items in the itemlist
and creates a description string by placing those values into
fields with character widths supplied in the formatlist.
Allowed data items can be any stored in the project database
plus JOB_ID (the id number of the job), which is an implicit
data item.
- exists(self)
- Test if the project exists.
Returns True if the project directory and database.def file both
exist, and False otherwise.
- getDbFile(self)
- Return the database file for the project.
- getDbItems(self)
- Return a list of the core data items stored for all jobs.
- getDbdir(self)
- Return the database directory of the project.
- getProjectname(self)
- Return the name of the project.
- getdata(self, job, item)
- Retrieve the value of a data item stored for a job.
The job is specified by an id number. If the item doesn't exist
(established through a call to the itemexists method) then the
operation raise an IndexError exception - otherwise the value of
the data item is returned.
- getmessage(self)
- Retrieve last internal error message.
If an operation fails then the client application can
access the last error message using this method. Invoking
getmessage clears the error message.
- hasjob(self, job)
- Checks if a job with the specified id exists in the database.
Given a job id, returns True if the database contains a job
with that id and False if not.
- haslock(self)
- Test whether the project is locked by this process.
Returns True if the project database.def file has a lockfile
owned by this process, and False otherwise.
- isloaded(self)
- Test whether the project data has been loaded.
Returns True if the database object has been populated from
the file on disk.
- islocked(self)
- Test whether the project is locked.
Returns True if the project database.def file has a lockfile
from any process, and False otherwise.
- isreadable(self)
- Test if it is possible to get data from the object.
Returns True if the database object has been loaded,
and has data that is at least as recent as the persistent
storage on disk.
- isstale(self)
- Check if the data in the object is older than on disk.
Return True if the modification time of the resource
file is more recent than the last save.
If so then this suggests that the data in this object may
not accurately mirror the data in the persistent
storage.
- iswriteable(self)
- Test if it is possible to modify and save the data.
Returns True if the database object has been loaded, owns
the lock on the persistent storage on disk, and has data
that is at least as recent as the persistent storage.
- itemexists(self, job, item)
- Check for the existence of a data item for a particular job.
Returns True if the data item is found for the specified job
id number, and False if not. Also returns False in the event
that the project is not loaded.
- listfiles(self, job, ftype, dirs)
- Return a list of files for a job.
The job is specified by an id number. 'ftype' specifies
whether the file list should be the associated 'input' or
'output' files.
'dirs' is a ccp4i directories object.
Returns a list with the full filenames for each of the
files associated with the job, or a blank list if there
are no files.
- listinputfiles(self, job, dirs)
- Return a list of input files for a job.
The job is specified by an id number,'dirs' is a ccp4i
directories object.
Returns a list with the full filenames for each of the
files associated with the job, or a blank list if there
are no files.
- listjobs(self)
- Return the list of all jobs in the project.
The list is an unsorted list of job id numbers, or an empty list
if the project is not currently loaded.
- listoutputfiles(self, job, dirs)
- Return a list of output files for a job.
The job is specified by an id number,'dirs' is a ccp4i
directories object.
Returns a list with the full filenames for each of the
files associated with the job, or a blank list if there
are no files.
- newjob(self, taskname='', status='STARTING', title='')
- Create a new job record in the project.
Creates a new entry in the current project with the current date
and returns the id number for the new job. Optionally the
taskname (TASKNAME data item), the status (STATUS item) and/or
title (TITLE item) may also be set when the job is created.
If no taskname is specified then it is set to 'none', if no status
is specified then it is set to 'STARTING'. (Note that blank values
of the taskname can crash CCP4i and cause corruption of the
database.)
The operation will fail and return -1 if project is not loaded,
or if the current object has lost the lock on the database.def
file.
- njobs(self)
- Return the value of the NJOBS data item.
For historical reasons, the NJOBS data item records the current
highest job id number used in the project, and is not generally
the actual number of jobs in the project. To get the actual
number of jobs in an object project, do len(project).
Returns -1 if the project has not been correctly loaded.
- open(self, grablock=False, readonly=False)
- Open a project.
Read the data from the project database and return the project
object.
The open operation will fail (return False) if the project is
locked, if the project is already loaded, or if the project
doesn't exist. Set the 'grablock' argument to True to override
any existing lock and force loading of the database.
Specifying 'readonly' as True populates the database object
from the file on disk without attempting to lock the resource.
However it will clear an existing lock, so the process may
still own the lock even if it is in 'read-only' mode.
- refresh(self, grablock=False, readonly=True)
- Reload the data from the file into a database object.
Use the 'refresh' method to re-read the data from the
file on disk into an existing (loaded) database object.
This is necessary for example if the file is updated
by an external process.
Essentially the database object is closed and then
reopened. By default the data is reloaded in 'readonly'
mode. Set the 'grablock' argument to True to override
any existing lock and force loading of the database in
write mode - see the 'open' method.
- removefile(self, job, ftype, dirs, filename)
- Remove a file from the list of files for a job.
The job is specified by an id number. 'ftype' specifies
whether the file should be associated as an 'input' or
'output' file.
'dirs' is a ccp4i directories object.
Returns True if the operation was successful, False
if there was a problem.
- removeinputfile(self, job, dirs, filename)
- Remove a file from the list of input files for a job.
The job is specified by an id number. 'dirs' is a ccp4i
directories object.
Returns True if the operation was successful, False
if there was a problem.
- removeoutputfile(self, job, dirs, filename)
- Remove a file from the list of output files for a job.
The job is specified by an id number. 'dirs' is a ccp4i
directories object.
Returns True if the operation was successful, False
if there was a problem.
- save(self)
- Save the project database contents to file.
The save method writes the data in the object to persistent
storage (i.e. the database.def file for the project).
On successful completion True is returned.
The save operation will fail if the project is not loaded,
or if the object has lost the lock on the database.def file;
in both cases False is returned.
- selectjobs(self, item, pattern)
- Retrieve a list of jobs based on some selection criterion.
Returns a list of jobs for which the value of the given data
item matches the supplied regular expression pattern.
- set_auto_refresh(self, auto_refresh)
- Set the auto_refresh flag.
'auto_refresh' takes a boolean value. Setting it to 'True'
means that a read attempt on a 'stale' database (i.e. when
the source database file is newer than the data in the
object) will automatically invoke the 'refresh' method to
load the data in read-only mode. Setting it to 'False'
means that the 'refresh' method must be explicitly invoked.
- setdata(self, job, item, value)
- Set the value of a data item for the specified job.
The job is specified as an id number, and the operation returns
True on success, and False on failure. The operation will fail
if: the project is not loaded; the data item doesn't exist for
the job; the object has lost the lock on the database.def file.
No attempt is made to check the validity of the new value.
- updatetime(self, job)
- Update the time of the job to be the current time.
This wraps the setdata method to automatically set the DATE
attribute of the job specified by the id number, and returns
the result of the setdata operation.
|
class dblockfile(lockfile) |
| |
CCP4i database lock file class.
This class provides methods for creating and manipulating lock
files for CCP4i def file based databases.
The calling subprogram should specify the name of the project
(argument 'project') and the path to the database directory
i.e. the location of the database.def file (argument 'dbdir').
|
| |
Methods defined here:
- __init__(self, project, dbdir)
Methods inherited from lockfile:
- filename(self)
- haslock(self)
- islocked(self)
- lock(self)
- resource(self)
- unlock(self, force=False)
|
class directories |
| |
CCP4i project directories class.
This class provides methods for reading and writing the project
directory information in directories.def for CCP4i-based
applications.
|
| |
Methods defined here:
- __del__(self)
- __init__(self, user='', source='')
- __nonzero__(self)
- __repr__(self)
- adddefdirref(self, def_dir, path)
- addprojectref(self, alias, path, db_dir='')
- array(self)
- ### FIXME
### This function provided for diagnostic purposes
- auto_refresh(self)
- Return the value of the auto_refresh flag.
The 'auto_refresh' feature means that read attempts on a
'stale' database automatically cause the data to be
refreshed from file in read-only mode. See the
set_auto_refresh method for more information.
- create(self)
- defdir(self, def_dir)
- deletedefdirref(self, def_dir)
- deleteprojectref(self, alias)
- filenamecomponents(self, filename)
- fullfilename(self, filen, project)
- getmessage(self)
- haslock(self)
- isdefdir(self, def_dir)
- isdefdirdir(self, dir)
- isloaded(self)
- islocked(self)
- islockedbyccp4i(self)
- isproject(self, project)
- isprojectdir(self, dir)
- isreadable(self)
- isstale(self)
- iswriteable(self)
- listdefdirs(self)
- listprojects(self)
- load(self, force=False, readonly=False)
- logdir(self)
- n_defdirs(self)
- n_projects(self)
- project_menu(self)
- projectdb(self, project)
- projectdir(self, project)
- refresh(self, readonly=True)
- release(self)
- save(self, filen='')
- set_auto_refresh(self, auto_refresh)
- Set the auto_refresh flag.
'auto_refresh' takes a boolean value. Setting it to 'True'
means that a read attempt on a 'stale' database (i.e. when
the source database file is newer than the data in the
object) will automatically invoke the 'refresh' method to
load the data in read-only mode. Setting it to 'False'
means that the 'refresh' method must be explicitly invoked.
- setlogdir(self, log_dir)
- source(self)
- user(self)
|
class extendedhistory(history) |
| |
Extended history class for use with projectDB class.
The extendedhistory class is intended for use with the
projectDB class and adds support for querying the history
for subjobs as well as 'top-level' jobs. The same
methods are available as for the base 'history' class
however they can be used for subjobs by specifying
the subjob id using the x.y notation, e.g.
history.parentsof(9)
will return top-level jobs that are 'parents' of job 9,
while
history.parentsof(9.5)
will return subjobs of job 9 that are 'parents' of job
9.5.
Note that top-level jobs will not be returned as being
linked to subjobs (or vice versa); neither will subjobs
that belong to different top-level jobs.
|
| |
Methods defined here:
- __init__(self, database, directories)
- Override initialisation of base class.
This additionally sets up a dictionary for the histories
of each of the subjob databases.
- arelinked(self, job_id1, job_id2)
- Override the test to see if two jobs are linked.
- childrenof(self, job_id)
- Override the listing of child ids for the current job.
- construct(self)
- Override the construction of the history.
This additionally constructs histories for each of the
subjob databases.
- parentsof(self, job_id)
- Override the listing of parent ids for the current job.
Methods inherited from history:
- addlink(self, parent_job, child_job)
- allchildrenof(self, job_id, result_ids)
- allparentsof(self, job_id, result_ids)
- infiles(self, job_id)
- outfiles(self, job_id)
- update(self)
- updatelinks(self, job_id)
|
class history |
| |
CCP4i database history object.
Supplied with a loaded database object and a loaded directories
object, use the construct method to populate the history object.
Use the parentsof and childrenof methods to return a list of
parent/child job id numbers for each job. Use the update method
to find and add new relationships for an existing history.
Use the arelinked method to find out if two job ids are linked.
|
| |
Methods defined here:
- __init__(self, database, directories)
- addlink(self, parent_job, child_job)
- allchildrenof(self, job_id, result_ids)
- allparentsof(self, job_id, result_ids)
- arelinked(self, job_id1, job_id2)
- childrenof(self, job_id)
- construct(self)
- infiles(self, job_id)
- outfiles(self, job_id)
- parentsof(self, job_id)
- update(self)
- updatelinks(self, job_id)
|
class projectDB(database) |
| |
CCP4i project database class.
This extends the base 'database' class to allow subjobs to
be defined and manipulated.
Subjob databases are opened in a 'lazy' fashion i.e. only
when they are needed.
|
| |
Methods defined here:
- __init__(self, project, directory)
- Initialise the project database.
- addfile(self, jobid, ftype, dirs, filename, alias)
- Add a file to the list of files for a job.
Over-ride the 'addfile' method of the parent class
in order to also deal with subjobs.
- addsubjob(self, job, taskname, title, status='STARTING')
- Add a subjob to a job in the project database.
If this is the first subjob to be added to the job,
then a new subjob database will be created first.
The id number of the subjob will be returned.
- buildjobid(self, job, subjob=None)
- Build a single job id given the job and subjob components.
This is a wrapper for the 'MakeJobid' function.
- close(self)
- Close the project database.
This extends the 'close' method of the base class,
by also closing all the opened subjob databases.
- deletejob(self, jobid)
- Delete a subjob from a job in the project database.
Over-ride the 'deletejob' method of the parent class.
'jobid' can be a job or subjob identifier.
- describejob(self, jobid, itemlist, formatlist)
- Return a formatted string populated with job data.
This over-rides the method in the base class in order
to be able to also deal with subjobs.
- getdata(self, jobid, item)
- Retrieve the value of a data item stored for a job.
This over-rides the 'getdata' method in the parent class.
'job' can be either an id for a job in the main database,
or for a subjob.
- getsubjobdb(self, job)
- Return an existing subjobDB object associated with a job.
This returns a subjobDB object associated with a job
id, or 'False' if a subjob database doesn't already
exist for the specified job.
- has_subjobs(self, job)
- Check if a job currently has subjobs defined.
- hasjob(self, jobid)
- Checks if a job with the specified id exists in the database.
Given a job id, returns True if the database contains a job
with that id and False if not.
Overrides the version in the parent class.
- itemexists(self, jobid, item)
- Check for the existence of a data item for a particular job.
This over-rides the 'itemexists' method in the parent class.
'jobid' can be either an id for a job in the main database,
or for a subjob.
- listjobs(self, jobid=None)
- Return a list of jobs or subjobs.
Over-ride the 'listjob' method of the parent class.
If no jobid is supplied then return the list of top-level
jobs in the project; if a jobid is supplied for a top-level
job then return the list of associated subjobs.
- listjobs_withsubjobs(self)
- Return a list of the jobs which also have subjobs.
This could be quite slow for large projects since it
attempts to open subjob databases for every job, to
test for their existence.
Possibly the projectDB object could keep track of the
subjob data internally to speed this up in future.
- opensubjobdb(self, job, create=True)
- Open a subjob database for the specified job id.
Returns an open subjobDB object, or 'False' if no
subjob could be opened.
If the 'create' argument is True then if the subjob
database does exist in the file system then a new
subjob database will be created (this is the default).
To avoid this set the 'create' argument to False.
- removefile(self, jobid, ftype, dirs, filename)
- Remove a file from the list of files for a job or subjob.
Over-ride the 'addfile' method of the parent class
in order to also deal with subjobs.
- save(self)
- Save the project database to file.
This extends the 'save' method of the base class, by
also performing save operations on all of the opened
subjob databases.
- selectsubjobs(self, job, item, pattern)
- Retrieve a list of subjobs based on some selection criterion.
- setdata(self, jobid, item, value)
- Set the value of a data item for the specified job or subjob.
This over-rides the 'setdata' method in the parent class.
'jobid' can be either an id for a job in the main database,
or for a subjob.
- splitjobid(self, jobid)
- Split a job id into job and subjob components.
This is a wrapper for the 'SplitJobid' function.
Methods inherited from database:
- __del__(self)
- del(project)
Saves data, releases lock and deletes the object.
- __len__(self)
- len(project)
Returns the number of jobs stored in the project, or
zero if the project is not loaded.
- __nonzero__(self)
- Returns True if the project is loaded, False otherwise.
- __repr__(self)
- Returns the project name.
- addinputfile(self, job, dirs, filename, alias)
- Add a file to the list of input files for a job.
This function wraps the addfile method: 'job' is a
job number, 'dirs' is a directories object, 'filename'
is an absolute or relative path, and 'alias' is an
optional project name which is used to resolve
relative paths but is otherwise ignored.
The method returns True on success and False otherwise.
- addoutputfile(self, job, dirs, filename, alias)
- Add a file to the list of output files for a job.
This function wraps the addfile method: 'job' is a
job number, 'dirs' is a directories object, 'filename'
is an absolute or relative path, and 'alias' is an
optional project name which is used to resolve
relative paths but is otherwise ignored.
The method returns True on success and False otherwise.
- auto_refresh(self)
- Return the value of the auto_refresh flag.
The 'auto_refresh' feature means that read attempts on a
'stale' database automatically cause the data to be
refreshed from file in read-only mode. See the
set_auto_refresh method for more information.
- create(self)
- Make a new project.
Creating a new project includes creating the project directory
(if it doesn't already exist), and creating the database
subdirectory and database.def file. The project is then opened
and the project object is returned.
The create operation will fail (returning False) if the project
already exists.
- describe(self, joblist, itemlist, formatlist)
- Return a list of formatted strings populated with job data.
For each job id number in the joblist, the describe method
retrieves the values of each of the data items in the itemlist
and creates a description string by placing those values into
fields with character widths supplied in the formatlist.
It returns a list of these strings, with one list item per job.
Allowed data items can be any stored in the project database
plus JOB_ID (the id number of the job), which is an implicit
data item.
- exists(self)
- Test if the project exists.
Returns True if the project directory and database.def file both
exist, and False otherwise.
- getDbFile(self)
- Return the database file for the project.
- getDbItems(self)
- Return a list of the core data items stored for all jobs.
- getDbdir(self)
- Return the database directory of the project.
- getProjectname(self)
- Return the name of the project.
- getmessage(self)
- Retrieve last internal error message.
If an operation fails then the client application can
access the last error message using this method. Invoking
getmessage clears the error message.
- haslock(self)
- Test whether the project is locked by this process.
Returns True if the project database.def file has a lockfile
owned by this process, and False otherwise.
- isloaded(self)
- Test whether the project data has been loaded.
Returns True if the database object has been populated from
the file on disk.
- islocked(self)
- Test whether the project is locked.
Returns True if the project database.def file has a lockfile
from any process, and False otherwise.
- isreadable(self)
- Test if it is possible to get data from the object.
Returns True if the database object has been loaded,
and has data that is at least as recent as the persistent
storage on disk.
- isstale(self)
- Check if the data in the object is older than on disk.
Return True if the modification time of the resource
file is more recent than the last save.
If so then this suggests that the data in this object may
not accurately mirror the data in the persistent
storage.
- iswriteable(self)
- Test if it is possible to modify and save the data.
Returns True if the database object has been loaded, owns
the lock on the persistent storage on disk, and has data
that is at least as recent as the persistent storage.
- listfiles(self, job, ftype, dirs)
- Return a list of files for a job.
The job is specified by an id number. 'ftype' specifies
whether the file list should be the associated 'input' or
'output' files.
'dirs' is a ccp4i directories object.
Returns a list with the full filenames for each of the
files associated with the job, or a blank list if there
are no files.
- listinputfiles(self, job, dirs)
- Return a list of input files for a job.
The job is specified by an id number,'dirs' is a ccp4i
directories object.
Returns a list with the full filenames for each of the
files associated with the job, or a blank list if there
are no files.
- listoutputfiles(self, job, dirs)
- Return a list of output files for a job.
The job is specified by an id number,'dirs' is a ccp4i
directories object.
Returns a list with the full filenames for each of the
files associated with the job, or a blank list if there
are no files.
- newjob(self, taskname='', status='STARTING', title='')
- Create a new job record in the project.
Creates a new entry in the current project with the current date
and returns the id number for the new job. Optionally the
taskname (TASKNAME data item), the status (STATUS item) and/or
title (TITLE item) may also be set when the job is created.
If no taskname is specified then it is set to 'none', if no status
is specified then it is set to 'STARTING'. (Note that blank values
of the taskname can crash CCP4i and cause corruption of the
database.)
The operation will fail and return -1 if project is not loaded,
or if the current object has lost the lock on the database.def
file.
- njobs(self)
- Return the value of the NJOBS data item.
For historical reasons, the NJOBS data item records the current
highest job id number used in the project, and is not generally
the actual number of jobs in the project. To get the actual
number of jobs in an object project, do len(project).
Returns -1 if the project has not been correctly loaded.
- open(self, grablock=False, readonly=False)
- Open a project.
Read the data from the project database and return the project
object.
The open operation will fail (return False) if the project is
locked, if the project is already loaded, or if the project
doesn't exist. Set the 'grablock' argument to True to override
any existing lock and force loading of the database.
Specifying 'readonly' as True populates the database object
from the file on disk without attempting to lock the resource.
However it will clear an existing lock, so the process may
still own the lock even if it is in 'read-only' mode.
- refresh(self, grablock=False, readonly=True)
- Reload the data from the file into a database object.
Use the 'refresh' method to re-read the data from the
file on disk into an existing (loaded) database object.
This is necessary for example if the file is updated
by an external process.
Essentially the database object is closed and then
reopened. By default the data is reloaded in 'readonly'
mode. Set the 'grablock' argument to True to override
any existing lock and force loading of the database in
write mode - see the 'open' method.
- removeinputfile(self, job, dirs, filename)
- Remove a file from the list of input files for a job.
The job is specified by an id number. 'dirs' is a ccp4i
directories object.
Returns True if the operation was successful, False
if there was a problem.
- removeoutputfile(self, job, dirs, filename)
- Remove a file from the list of output files for a job.
The job is specified by an id number. 'dirs' is a ccp4i
directories object.
Returns True if the operation was successful, False
if there was a problem.
- selectjobs(self, item, pattern)
- Retrieve a list of jobs based on some selection criterion.
Returns a list of jobs for which the value of the given data
item matches the supplied regular expression pattern.
- set_auto_refresh(self, auto_refresh)
- Set the auto_refresh flag.
'auto_refresh' takes a boolean value. Setting it to 'True'
means that a read attempt on a 'stale' database (i.e. when
the source database file is newer than the data in the
object) will automatically invoke the 'refresh' method to
load the data in read-only mode. Setting it to 'False'
means that the 'refresh' method must be explicitly invoked.
- updatetime(self, job)
- Update the time of the job to be the current time.
This wraps the setdata method to automatically set the DATE
attribute of the job specified by the id number, and returns
the result of the setdata operation.
|
class subjobDB(database) |
| |
CCP4i subjob database class.
This stores the job data for substeps defined for a job in
the main project directory.
The subjob data are stored in a subdirectory of the project
database directory, called '<jobid>_database. The name of
the project database directory must be given explicitly as
the 'dbdir' argument.
Note that the subjob databases differ from the base class
databases, in that when a subjob db is closed it will
automatically delete the persistent storage (i.e. the
subdirectory and database.def file) if there are no subjobs
stored at this point.
|
| |
Methods defined here:
- __init__(self, job, taskname, dbdir)
- Initialise the subjob database.
The calling process should include the taskname and job id
number of the main job to which the subjobs belong.
- close(self)
- Close the subjob database.
This extends the 'close' method of the base class.
If there are no subjobs defined at the point of closure
then the subjob database is removed.
- getId(self)
- Return the identifier for the subjob database.
Methods inherited from database:
- __del__(self)
- del(project)
Saves data, releases lock and deletes the object.
- __len__(self)
- len(project)
Returns the number of jobs stored in the project, or
zero if the project is not loaded.
- __nonzero__(self)
- Returns True if the project is loaded, False otherwise.
- __repr__(self)
- Returns the project name.
- addfile(self, job, ftype, dirs, filename, alias)
- Add a file to the list of files for a job.
The job is specified by an id number. 'ftype' specifies
whether the file should be associated as an 'input' or
'output' file.
'filename' specifies the name of the file to be added
to the can be either a full path or a relative path,
and 'alias' specifies a project alias to be associated
with the file. The alias is only used to resolve
relative paths in the filename, and is ignored if a
full path is supplied for the file being added.
'dirs' is a ccp4i directories object.
Returns True if the operation was successful, False
if there was a problem. Note that if the file name
already appears in the list of files then this is
not an error.
- addinputfile(self, job, dirs, filename, alias)
- Add a file to the list of input files for a job.
This function wraps the addfile method: 'job' is a
job number, 'dirs' is a directories object, 'filename'
is an absolute or relative path, and 'alias' is an
optional project name which is used to resolve
relative paths but is otherwise ignored.
The method returns True on success and False otherwise.
- addoutputfile(self, job, dirs, filename, alias)
- Add a file to the list of output files for a job.
This function wraps the addfile method: 'job' is a
job number, 'dirs' is a directories object, 'filename'
is an absolute or relative path, and 'alias' is an
optional project name which is used to resolve
relative paths but is otherwise ignored.
The method returns True on success and False otherwise.
- auto_refresh(self)
- Return the value of the auto_refresh flag.
The 'auto_refresh' feature means that read attempts on a
'stale' database automatically cause the data to be
refreshed from file in read-only mode. See the
set_auto_refresh method for more information.
- create(self)
- Make a new project.
Creating a new project includes creating the project directory
(if it doesn't already exist), and creating the database
subdirectory and database.def file. The project is then opened
and the project object is returned.
The create operation will fail (returning False) if the project
already exists.
- deletejob(self, job)
- Remove an existing job record from the project.
The job with the specified id number will be removed from the
project, also removing all associated data, returning True on
success. The operation will fail if the database is not loaded,
if the object has lost the lock on the database.def file, or
if a job with the specified id number is not found.
- describe(self, joblist, itemlist, formatlist)
- Return a list of formatted strings populated with job data.
For each job id number in the joblist, the describe method
retrieves the values of each of the data items in the itemlist
and creates a description string by placing those values into
fields with character widths supplied in the formatlist.
It returns a list of these strings, with one list item per job.
Allowed data items can be any stored in the project database
plus JOB_ID (the id number of the job), which is an implicit
data item.
- describejob(self, job, itemlist, formatlist)
- Return a formatted string populated with job data.
For the specified job id number, the describejob method
retrieves the values of each of the data items in the itemlist
and creates a description string by placing those values into
fields with character widths supplied in the formatlist.
Allowed data items can be any stored in the project database
plus JOB_ID (the id number of the job), which is an implicit
data item.
- exists(self)
- Test if the project exists.
Returns True if the project directory and database.def file both
exist, and False otherwise.
- getDbFile(self)
- Return the database file for the project.
- getDbItems(self)
- Return a list of the core data items stored for all jobs.
- getDbdir(self)
- Return the database directory of the project.
- getProjectname(self)
- Return the name of the project.
- getdata(self, job, item)
- Retrieve the value of a data item stored for a job.
The job is specified by an id number. If the item doesn't exist
(established through a call to the itemexists method) then the
operation raise an IndexError exception - otherwise the value of
the data item is returned.
- getmessage(self)
- Retrieve last internal error message.
If an operation fails then the client application can
access the last error message using this method. Invoking
getmessage clears the error message.
- hasjob(self, job)
- Checks if a job with the specified id exists in the database.
Given a job id, returns True if the database contains a job
with that id and False if not.
- haslock(self)
- Test whether the project is locked by this process.
Returns True if the project database.def file has a lockfile
owned by this process, and False otherwise.
- isloaded(self)
- Test whether the project data has been loaded.
Returns True if the database object has been populated from
the file on disk.
- islocked(self)
- Test whether the project is locked.
Returns True if the project database.def file has a lockfile
from any process, and False otherwise.
- isreadable(self)
- Test if it is possible to get data from the object.
Returns True if the database object has been loaded,
and has data that is at least as recent as the persistent
storage on disk.
- isstale(self)
- Check if the data in the object is older than on disk.
Return True if the modification time of the resource
file is more recent than the last save.
If so then this suggests that the data in this object may
not accurately mirror the data in the persistent
storage.
- iswriteable(self)
- Test if it is possible to modify and save the data.
Returns True if the database object has been loaded, owns
the lock on the persistent storage on disk, and has data
that is at least as recent as the persistent storage.
- itemexists(self, job, item)
- Check for the existence of a data item for a particular job.
Returns True if the data item is found for the specified job
id number, and False if not. Also returns False in the event
that the project is not loaded.
- listfiles(self, job, ftype, dirs)
- Return a list of files for a job.
The job is specified by an id number. 'ftype' specifies
whether the file list should be the associated 'input' or
'output' files.
'dirs' is a ccp4i directories object.
Returns a list with the full filenames for each of the
files associated with the job, or a blank list if there
are no files.
- listinputfiles(self, job, dirs)
- Return a list of input files for a job.
The job is specified by an id number,'dirs' is a ccp4i
directories object.
Returns a list with the full filenames for each of the
files associated with the job, or a blank list if there
are no files.
- listjobs(self)
- Return the list of all jobs in the project.
The list is an unsorted list of job id numbers, or an empty list
if the project is not currently loaded.
- listoutputfiles(self, job, dirs)
- Return a list of output files for a job.
The job is specified by an id number,'dirs' is a ccp4i
directories object.
Returns a list with the full filenames for each of the
files associated with the job, or a blank list if there
are no files.
- newjob(self, taskname='', status='STARTING', title='')
- Create a new job record in the project.
Creates a new entry in the current project with the current date
and returns the id number for the new job. Optionally the
taskname (TASKNAME data item), the status (STATUS item) and/or
title (TITLE item) may also be set when the job is created.
If no taskname is specified then it is set to 'none', if no status
is specified then it is set to 'STARTING'. (Note that blank values
of the taskname can crash CCP4i and cause corruption of the
database.)
The operation will fail and return -1 if project is not loaded,
or if the current object has lost the lock on the database.def
file.
- njobs(self)
- Return the value of the NJOBS data item.
For historical reasons, the NJOBS data item records the current
highest job id number used in the project, and is not generally
the actual number of jobs in the project. To get the actual
number of jobs in an object project, do len(project).
Returns -1 if the project has not been correctly loaded.
- open(self, grablock=False, readonly=False)
- Open a project.
Read the data from the project database and return the project
object.
The open operation will fail (return False) if the project is
locked, if the project is already loaded, or if the project
doesn't exist. Set the 'grablock' argument to True to override
any existing lock and force loading of the database.
Specifying 'readonly' as True populates the database object
from the file on disk without attempting to lock the resource.
However it will clear an existing lock, so the process may
still own the lock even if it is in 'read-only' mode.
- refresh(self, grablock=False, readonly=True)
- Reload the data from the file into a database object.
Use the 'refresh' method to re-read the data from the
file on disk into an existing (loaded) database object.
This is necessary for example if the file is updated
by an external process.
Essentially the database object is closed and then
reopened. By default the data is reloaded in 'readonly'
mode. Set the 'grablock' argument to True to override
any existing lock and force loading of the database in
write mode - see the 'open' method.
- removefile(self, job, ftype, dirs, filename)
- Remove a file from the list of files for a job.
The job is specified by an id number. 'ftype' specifies
whether the file should be associated as an 'input' or
'output' file.
'dirs' is a ccp4i directories object.
Returns True if the operation was successful, False
if there was a problem.
- removeinputfile(self, job, dirs, filename)
- Remove a file from the list of input files for a job.
The job is specified by an id number. 'dirs' is a ccp4i
directories object.
Returns True if the operation was successful, False
if there was a problem.
- removeoutputfile(self, job, dirs, filename)
- Remove a file from the list of output files for a job.
The job is specified by an id number. 'dirs' is a ccp4i
directories object.
Returns True if the operation was successful, False
if there was a problem.
- save(self)
- Save the project database contents to file.
The save method writes the data in the object to persistent
storage (i.e. the database.def file for the project).
On successful completion True is returned.
The save operation will fail if the project is not loaded,
or if the object has lost the lock on the database.def file;
in both cases False is returned.
- selectjobs(self, item, pattern)
- Retrieve a list of jobs based on some selection criterion.
Returns a list of jobs for which the value of the given data
item matches the supplied regular expression pattern.
- set_auto_refresh(self, auto_refresh)
- Set the auto_refresh flag.
'auto_refresh' takes a boolean value. Setting it to 'True'
means that a read attempt on a 'stale' database (i.e. when
the source database file is newer than the data in the
object) will automatically invoke the 'refresh' method to
load the data in read-only mode. Setting it to 'False'
means that the 'refresh' method must be explicitly invoked.
- setdata(self, job, item, value)
- Set the value of a data item for the specified job.
The job is specified as an id number, and the operation returns
True on success, and False on failure. The operation will fail
if: the project is not loaded; the data item doesn't exist for
the job; the object has lost the lock on the database.def file.
No attempt is made to check the validity of the new value.
- updatetime(self, job)
- Update the time of the job to be the current time.
This wraps the setdata method to automatically set the DATE
attribute of the job specified by the id number, and returns
the result of the setdata operation.
| |