An evaluation of software development practice and assessment-based process improvement in small software development firms

摘要

[Abstract]: As software becomes increasingly important to all aspects of industry, there is a need to encourage practitioners to adopt best practice so as to improve the quality of the processes in use, and therefore achieve targets relating to time, budget and quality. The software development industry in Australia is dominated by a myriad of small firms. This presents a challenge in terms of determining the current practices of industry participants, and in devising improvement initiatives which are feasible for small organisations. Currently, the level of adoption of best practice among local software developers is unknown. To help improve the software industry, it is necessary to determine the current status of use of practices and techniques. Furthermore, the effectiveness of assessment-based software process improvement for small organisations needs to be evaluated. The objective of this research is to understand the extent of software development practices currently in use, and to evaluate the effectiveness of assessment-based software process improvement initiatives for small firms.ududTo achieve this objective, an extensive mail survey of the Queensland software industry was conducted to identify and compare best practice in software development with current practice. The survey was based on the software best practice questionnaire used by the European Software Institute. Following on from this, a detailed evaluation of a process improvement program in 22 small firms was carried out. The program used the Rapid Assessments for Process Improvement for software Development (RAPID) model and method. RAPID is based on ISO/IEC 15504 (SPICE) and includes eight processes: requirements elicitation, software development, configuration management, quality assurance, project management, problem resolution, risk management, and process establishment. The evaluation analysed the process capability of the firms as reported from one-day software process assessments and also the extent of improvement as recorded at follow-up meetings held 7 to 16 months after the assessment. Both quantitative and qualitative techniques were used to analyse the assessment reports. ududThe study confirmed that there is wide variation in the extent of adoption of software development best practice in terms of the individual practices, as well as the organisations. While project management planning and customer involvement practices are widely adopted, the use of metrics for estimating and testing are barely used by the organisations that responded to the survey. Overall, practices of a technical nature are more widely adopted compared to techniques related to support and management. Organisations involved in developing commercial off-the-shelf software have higher adoption than firms which do not develop such systems, and adoption of best practice is associated with the size of the development group. The leaders in adoption have significantly better practices when compared to the laggards for 40 of the 44 practices included in the survey. Furthermore, organisations from the finance, insurance and utilities sectors exhibited higher adoption of best practice compared to organisations from other sectors. The overall adoption of 48 percent implies that the organisations which responded have adopted, on average, almost half of the best practices in the questionnaire. While this overall adoption rate places the Queensland software industry in a competitive position compared to adoption of firms in European countries, there is scope for improvement. ududThe process improvement assessments of 22 firms also confirmed that the capability of technical processes is higher than that of management processes; and suggested that higher capability is associated with the proportion of experienced staff and the proportion of staff with post graduate qualifications. Higher process capability is also associated with firms undertaking projects of lengthy durations. Most of the processes were rated at the lowest levels. Almost one third of all the processes were rated as incomplete (level 0) and 46 percent were rated as performed (level 1).ududThe evaluation of the process improvement program was conducted by analysing the 22 assessment reports, and the 20 final reports from the follow-up meetings. The extent of improvement is associated with the proportion of technical staff and the proportion of formally qualified staff. The evaluation revealed that assessment-based process improvement programs are effective for small firms, regardless of the maturity of the processes at the time of the assessment. ududAs well as detailing the process capability of 22 small software firms, this study provides an interesting insight into the actions, reasons for inaction, and reactions of the firms as far as implementing the recommendations from the assessments. Analysis of the reactions of the participants of this program suggests that for small firms, mentoring, training and organisation stability are important factors, while senior management support may not be an issue of concern.ududThe study indicates that small firms can benefit from a low cost process improvement program with a restricted scope, a short time frame to evaluation, and mentoring from external assessors/consultants. It is also crucial that the firm is not disrupted by internal or external events during the course of the software process improvement program. Furthermore, this study provides a contribution to assessment methods by validating the RAPID model and method, and providing recommendations to improve the RAPID method. The outcomes from this research have the potential to better equip practitioners and consultants to undertake software process improvement, hence increasing the success of small software development firms in domestic and global markets.ud