I have identified 6 stages:
"Development" is what we do for most of the time (ideally i.e. for 8 months of the year) and should include major changes to the suite (new libraries, major revisions of guis, porting to new platforms) by DL staff. The end of the this development stage should coincide with the deadline for changes from external sources.
"Consolidation" is where the externally contributed programs etc need to be brought into the suite, and loose ends for local developments need to be tied up. At this point no major developments should be started by DL staff.
this implies that we should already have agreed at an early stage what is to be included in the release, both locally and with our major contributors. These agreements will help to prevent "feature creep".
"Testing" starts when we have a reasonable alpha version of the suite which can be released to limited test sites. At this point we have a code freeze for developments going into this release.
there may be several test releases, the test sites must understand that subsequent beta versions override earlier ones. The test sites need to understand what is being asked of them.
perhaps use a centralised bug tracking method? e.g. record bugs as they are reported, queue them up to be dealt with one at a time.
perhaps split off a "developmental" CVS branch? so that developmental contributions can be continuously added into CVS without compromising the release
"Major release" occurs after testing is completed.
"Bug fixes" from users after the major release should be included as part of the release process. The first patch release completes the release project.
Past experience (4.2):
announced deadline for changes as 30th November 2001, and release date as end of February 2002 (when? end of Sept 2001?). The reasoning was to try and avoid a clash with Christmas and Study Weekend.
In actuality, we were still developing many components into the new year, and accepting major changes from external collaborators.
The first alpha release was made 8th March 2002; beta release 5th April; major release 30 April 2002 - so "testing" phase lasted ~2 months.
A number of key components were still unsatisfactorily tested at the release time. This was due to external collaborators either still making major changes until the last weeks, or not applying the same standards for testing as we do here (thinks: perhaps we need to demand test examples as a condition for accepting new applications?).
I think the reason this release stretched out so much was that development was not finished before the release project began. Resources were overcommitted and developments took longer to finish than anticipated. Although we had a plan this might have been more useful if we had started formulating it a lot earlier. We were probably also too optimistic about what we would be able to achieve in the timescale.
Once we had embarked on the release proper, and particularly towards the end, a big problem stemmed from the fact that external contributors didn't appreciate/respect our timescales, and contributions were submitted which often required substantial testing and patching. Whilst communication of the release status was good within the DL group, it was generally much poorer with other developers and this may have been part of the reason for the problem.
Another point is that as the release grew longer, development was stifled and people started to become impatient. Minimising the release time should be beneficial in cutting down on the lock-up of effort.
We should aim to minimise the amount of time from end of development to the point of the major release. Perhaps this should be a project objective? i.e. make release within x months.