Software Development Reengineering – An Experience Report
Lexical Integration (M) Sdn Bhd
Abstract. Decisions regarding the disposition of an existing multilingual environments can have an enormous effect on the operations, even survival, of commercial and government organizations. Many organizations are grappling with trade-offs of older development tools and moving into more modern and efficient programming languages as advertised, while trying to leverage production capabilities and to reduce cost. In some cases, product reliability is not an important factor in decision making! Lexical is facing these difficult decisions and the clock is ticking. In an effort of searching for a programming language that fits into our development requirement, the company successfully addressed many of the hard issues facing commercial executives and managers today. This paper provides valuable insights and model for actions, through Lexical’s experience, which can help making better decisions and formulate strategies in transition management. Many factors contribute to the success and survival of the company during transition, one of them is to assess learning curves of many different individuals.
1 Introduction and Overview
In the modern world, businesses and government organizations are constantly subject to drastic changes in daily operation and management. All occupations, CEOs, managers, financial planners, administrative personnel, executives, technical experts, and support staff are being constantly peppered with new technologies and different methods of running organization. Regardless of your occupation or type and size of business, you have one thing in common with everyone else: competition!
Lexical Integration Sdn Bhd is a young and dynamic software development company in Malaysia. The company is engaged in a wide variety of projects, including research and development of high-performance, application-specific software systems. The company also has a track record for delivering complex application software solutions to a broad range of businesses and governments. Lexical is no exception from the cruelty of this modern trend. As modern software systems become increasingly more complex and critical, clients are faced with ever more choices and developers are confronted with even more challenges. The company must find ways to maximize profit to maintain the endless development cycles and at the same time, to assert product quality to continue its survival. Regardless of your role in a commercial business or a government organization, the insights contained in this paper can be of benefit, and help you achieve and keep a competitive advantage.
Although the project involved the reengineering of specific software for furniture industry, the experience is very likely to be applicable across many commercial and government domains. The critical necessity for migrating from older software technologies to modern software architectures crosses boundaries of virtually every application domain in the internationally competitive software marketplace.
1.1 Reengineering Multilingual Systems
Lexical was faced with maintaining a complex software system, developed in a multilingual environment, which provided important capabilities to its users. While the clients were satisfied with the system, Lexical was awarded the contract to upgrade the existing software with its increasingly expensive and difficult to maintain software written for a proprietary, “non-open” hardware platform to support a cross-platform environment. As maintainability of the code decreased, support and maintenance costs were rocketing sky-high.
Initially, the development team was directed to re-host the application, and move it to an open systems architecture. Basically, this effort involved translating the existing code from Paradox, Clipper, COBOL, FORTRAN, and Pascal into C. Although some of the resulting code functioned properly, and the application was moved to an open system architecture, the code itself was even less maintainable than the original software. As the team continued working on other modules of the application, they encountered portability issues. The re-host was immediately halted.
The development team was then directed to conduct a study on existing software systems and the next generation software systems they were going to develop and to find a solution to contain the exorbitant maintenance and support cost. Otherwise, they would lose the client. The study was to include an analysis, evaluation, and recommendations to the company as to future directions for all software development. The recommendations for the future were to ensure the continuing satisfaction of the company’s clients and their user community, lower maintenance and support costs, and quality products with shorter time-to-market. The study showed the software system needed to be reengineered to provide future user needs, and the team recommended a blend of technologies and methods, including a layered, object-oriented software architecture, implemented in the Ada programming language. The resulting reengineered system produced a threefold improvement in maintainability of software. Additionally, the reengineered system has enabled new functionality to be added with minimum effort and time required at lower cost.
1.2 Maintainability Index and Metrics
The decision to reengineer the software system and the company software development environment is difficult to justify without appropriate metrics. To justify the decision, specific metrics were collected and analyzed to provide indicators of system maintainability and complexity. These metrics with independently developed polynomial equations have been used by some organizations to calculate a “maintainability index” which provides an indication of the maintainability of a software system. This maintainability index has shown the ability to help in providing sound economic justification for reengineering the software system as well as engineering of new software projects.
1.3 The Role of Software Architecture
The original version of the software system had been developed using functional decomposition methodology and typically made extensive use of proprietary platform features, operating system calls, and specific language constructs which severely hampered maintenance, reuse, and portability. These differences in the underlying software architecture had contributed significantly to the high costs of development, maintenance, and support. There would be lack of benefits to be derived from the reengineering efforts if the old software architecture was to remain.
The development team used an object-oriented layered software architecture to overcome the increasing difficulties in development and maintenance throughout the software life cycle. By using layers and object-oriented design for the application, interface, graphics, operating system, and hardware, the development team was able to deliver superb quality products and benefits to the client. The resulting new software system has not only significantly increased the return of investment for the client, but also productivity, competitiveness, and profitability of the company.
2 Project Background
fMRP is an integrated Manufacturing Resource Planning (MRP) for furniture industry in Malaysia. The software supports specific and complex requirements of some of the many furniture manufacturers in the country. Basically, the software helps the users in managing, planning, executing, and monitoring their daily operation. The software is also a data-mining tool to help making management decisions.
The original version of fMRP, referred to as V1, consists of 20 modules which were developed using various programming languages on DOS/Windows platforms. The software utilized ODBC to interface with various databases including flat-files, Paradox, .dbf, and InterBase. V1 reached its maturity in 1993 and version 2.0 or V2 was released. Like its predecessor, V2 was developed in aforesaid programming languages and ran only on Windows and NT platforms with Sybase and InterBase as the back-end database. In 1999, a completely Ada-reengineered version 3.0 or V3 was implemented and hosted on Linux. It was originally developed to target Sun Solaris but the 1997 global economy crisis had forced the client to reduce their budget. It was the layered object-oriented software architecture that enabled the development team to quickly re-target the software development to Linux with the underlying standard X11 implementation. InterBase 4 or IB4 was selected as the back-end database server for V3.
The benefits that V3 provides to its users including data-mining capabilities, real-time production monitoring, and accurate forecast are significantly critical to enable decisive actions and plans to be formulated. V3 also provides absolute inter-operability among departmental clusters that can share common data in near real-time. V1 was originally developed as stand-alone modules which were integrated along its development and, as V2, were mostly text-based. V3 takes advantage of a color graphics interface to enhance the fundamental capabilities of fMRP, with corresponding benefits to its users.
2.1 Project Evolution
The original V1 capability was developed in 1988. The first module was Store Management written in Clipper to maintain stock records and to manage stock requisition and stock flow. It was originally hosted on DOS. The software was designed using a top-down functional decomposition method, with a high degree of language specific dependency. However, it was a success and the software established a track record of successful usage. The client awarded the company to develop other modules and the development began with new programmers joining the team. The development team was divided into many small teams and each team was free to use whichever programming languages or tools they deemed fit to their projects. As examples, FORTRAN was used to develop MRM (Material Requirement Management) to perform calculations of material required, in this case, the volume of rubber wood in tonnage as well as to perform conversion from one unit of measurement to another. The calculation must be precise. Otherwise it would result in insufficiency or wastage of material. V1’s text-based SOM (Sales Order Management) was developed in Clipper and was reengineered to V2 using Paradox which provided a nice GUI on Windows. The original text-based PFMM (Production Flow Monitoring and Management) was developed in C and was then ported to Windows in V2. V1’s text-based LGM (Logistic Management) which was developed in COBOL was then ported to Delphi (a Pascal-based RAD tool) to provide a nice GUI. Other applications were POM (Purchase Order Management), SFM (Shop Floor Management), etc.
In terms of current open systems architecture, the original V1 was definitely a legendary “closed” system and had no inter-operability among modules. Communications and components integration were established by dumping data to shared flat files. Using tools and OS largely based on proprietary protocols and functionality typically thwarted such vertical integration.
The old underlying software architecture of V1 and V2 which was extensively dependent on proprietary platform features, operating systems calls, and language/tool specific constructs. The development team had no access to internal codes that provided certain operations, for instance, the database connectivity and the graphical user interface (GUI) in Paradox for Windows. Paradox was designed to provide rapid application development with limited freedom. Utilizing operating system specific functions, for instance, Win32 binding, limited the portability of the application to other platforms. As the client migrated toward Windows and NT when more powerful desktops and servers became available, it became clear that the heterogeneous and proprietary V2 architecture needed to be modified. As soon as V2 was released in 1993, the development team was tasked with modifying and re-hosting V2 to enable it to operate in a Windows environment and to rewrite all codes in a single programming language, C. The original software architecture was retained to support product specific functions. The effort was terminated soon after the development team was confronted by portability issues when the client signaled their interest in UNIX. UNIX and Linux allow complete integration of capabilities and methodologies (based on universal, “open” technical standards and protocols) between and amongst hardware and software modules. The other problems encountered were reliability, readability, maintainability, and standardization issues. Figure 1 illustrates the historical overview of the software evolution.
Although the various software implementation were quite successful from the users’ perspective, the different language and platform implementations suffered from configuration management (CM) problems. In addition to the CM challenges, the software was becoming more difficult to maintain and modify. In 1994, Lexical realized the existing implementation of V2 would not be adequate to support user needs into the future. At the same time, the clients likewise were aware that the cost of maintaining and modifying the software was becoming unacceptable, and enhancements were exacerbating the complexity of the software system. As a result of these concerns, and in anticipation of expected user requirements in the future, the clients tasked Lexical to provide a total solution, or otherwise trade-off with new software systems. The development team was instantly required to perform a research study for a single programming language that could help increase maintainability of the software and would be able to support, if not all, but most of the development requirements to ascertain the future direction of V2 and the company.
3 Research Study – The Future
The research study, code-named “Holy Grail”, included four primary objectives:
- Determine the objectives and future goals.
- Analyze the current V2 implementations from a software engineering perspective.
- Identify current software standards which could apply to V2 and future software projects.
- Offer recommendations to management on how to achieve the V2 objectives.
Among the objectives and future goals identified for fMRP was the need to provide software which would be more maintainable and modifiable than the existing multilingual code. More easily maintainable code would allow the company to minimize support costs without sacrificing functionality and reliability. More easily modifiable code would enable the client to keep abreast with new user needs and their constantly evolving business rules and quickly adopt emerging technologies into the software systems. The ability to migrate the software to more powerful computer platforms, which enable the company to respond to variety of requirements and budgets, was also a sound objective.
Contemporary industry trends and standards which could apply to fMRP and future projects included a wide spectrum of technologies and tools. Undoubtedly, the need for an open systems architecture was essential and an ongoing requirement for the future. In addition, the use of modern software development methods, such as object-oriented techniques, offered substantial promise for long-term fMRP usage and support.
As the study progressed, it became clear that there were two major factors which had a direct impact on the design and structure of fMRP: the software architecture and the programming language used for implementation. The deficiencies of the original software architecture, with its reliance on hardware-specific features and operating system calls, were perpetuated in the translated C version of the code (V2). The research team concluded that a continuation of the original software architecture would effectively preclude any major improvement in the quality and maintainability of the software.
Since the study was conducted from a software engineering perspective, the role of programming language selection in support of a well-engineered software implementation was included in the evaluation of fMRP. The lack of software engineering principles in the original software prevented improvement in the software. Software engineering was seen as the application of sound engineering principles to the development of systems that were modifiable, efficient, reliable, and understandable. These principles led us to these goals, including abstraction, information hiding, encapsulation, modularity, localization, uniformity, completeness, and confirmability. It became clear that the Ada programming language offered superior support for a reengineered version of fMRP and future software projects. Ada also offered benefits in the application of an object-oriented design for fMRP. As part of their evaluation of object-oriented technologies and techniques, the research team noted the importance of applying object-oriented methods in a disciplined software engineering context, as opposed to focusing on overrated and highly abused object-oriented programming features, such as inheritance and polymorphism. From a design perspective, as well as a software engineering perspective, Ada was a superior choice for the future of fMRP application.
The reengineering effort was an investment in the future, with an expectation of leveraging off that investment to accommodate user needs in a more cost-effective and reliable manner. The research team’s bottom line recommendations were as follows:
- Reengineer and redesign the system
- Use object-oriented technologies
- Use the Ada programming language
3.1 Programming Language Selection and Reengineering fMRP
The research team produced a Software Development Plan which set forth and documented the process by which fMRP would be reengineered. The Plan emphasized the use of sound software standards, such as Ada, as well as the disciplined application of software engineering principles. To support the team’s recommendation of Ada as the primary programming language, they referred to Albrecht [ALB83] and Jones [JON92] which provided rough estimates of the average number of lines of code required to build one function point in various programming languages as illustrated in Figure 2. To substantiate their recommendation of Ada, they referred to Booch [BOO94a] and concluded object-oriented techniques map well to Ada. In this regard, Ada may serve as both implementation and design language:
“Within any given computer language, sufficient tools must be provided to allow us to express problem solution. Ideally, we would like to use a language that allow us directly reflect our view of problem space….. In the case where programming from the problem space into a language directly reflects our structure of the problem space, we are presented with an implementation that is understandable and therefore helpful in managing the complexity of the larger systems. Such a language must provide tools for expressing primitive objects and operations and must, in addition, be extensible, so that we can build our own abstract objects and operations. In the best case, we also need our language to enforce these abstractions…. Ada is such a language. Not only Ada is suitable as an implementation language, but it is expressive enough to serve as a vehicle for capturing our design decisions….”
The development team followed an iterative lifecycle approach, to ensure flexibility and fast response to changing user requirements.
|Programming Language||LOC / FP (Average)|
|Fourth-generation languages (4GL)||20|
Figure 2:Rough estimates of the average number of LOC required to build one function point in various programming languages.[JON92]
Figure 3: New Reusable Layered Architecture (Click to view large)Because the team focused on a well-engineered layered software architecture, they were able to apply a wide variety of powerful technologies and methods in a disciplined and cost-effective order. Due to the successful reengineering of fMRP using these techniques, methods, and technologies as shown in Figure 4, the company directed the team to reengineer other software systems. These other software systems included SCM (Supply Chain Management), GTM (Government Tender Management), FAM (Fixed Assets Management), etc. All new projects would be engineered in the same manner.
|Information Hiding||Exceptions||Library Unit (Package)||Process Improvement|
Figure 4: Techniques and Methods
“Ada has language defined concurrency and concurrent activities can be identified explicitly in the code, thus it leads to more readable and maintainable programs. There are many different types of operating system, defining the concurrency in the language makes the program more portable. An embedded computer may not have any resident operating system available…”
4 Risk and Transition Management
Migration is change, and change is stressful if not managed well. Deft navigation of available options is essential, and limited viewpoints can only result in limited results. Given the availability of a wide range of programming languages, today’s IT leaders must look past personally limiting viewpoints, eschew “language-loyalty” mindsets, and ensure that technical merit prevails in decision-making. If programmers or software engineers are called upon to learn new programming languages and techniques as part of the migration, this should not be problematic if the people are of good quality. Even so, training will go a long way to smoothing the transition.
The Transition Plan was designed after considering Basili and Zelkowitz’s [BAS78] five important factors that influence software productivity:
- People factors: The size and expertise of the development organization.
- Problem factors: The complexity of the problem to be solved and the number of changes in design constraints or requirements.
- Process factors: Analysis and design techniques that are used, languages and CASE tools available, and review techniques.
- Product factors: Reliability and performance of the computer-based system.
- Resource factors: Availability of CASE tools, hardware and software resources.
5 Measures of Success
5.1 The Maintainability Index
|Software Baseline||MI||Software Baseline||MI|
|Clipper SOM V1||8.30||COBOL LGM V1||43.21|
|Paradox SOM V2||13.77||Pascal LGM V2||39.55|
|Ada SOM V3||73.93||Ada LGM V3||77.28|
|‘C’ PFMM V1||38.69||FORTRAN MRM||31.85|
|‘C’ PFMM V2||33.44||Ada MRM||77.64|
|Ada PFMM V3||78.71||Paradox POM||15.39|
|Ada PFMM V4||82.13||Ada POM||75.59|
|Clipper SFM||7.96||Ada FAM 2.0||74.53|
|Ada SFM||78.02||‘C’ GTM 1.0||35.11|
|‘C’ SCM 1.0||34.52||‘C’ GTM 2.0||31.75|
|Ada SCM 1.0||74.45||Ada GTM 1.0||71.65|
|Ada SCM 2.0||75.37||Ada GTM 2.0||72.74|
|Pascal FAM 4.0||40.13||Ada FAM 1.0||73.28|
Figure 5: Maintainability Index (MI) of various software modules of fMRP and other software systems.
Maintainability Index = 171 - 3.42 x ln(aveE) - 0.23 x aveV(g') - 16.2 x ln (aveLOC)
where aveE is average Halstead [HAL77] effort/module, aveV(g’) is average extended McCabe [MCC76] [MCC89] cyclomatic complexity/module, and aveLOC is average LOC/module.
A static analysis of various implementations in Ada and other languages was conducted using PC-Metric, a source code analysis program. The team calculated and used two widely known software metrics, which were Halstead’s effort/module; and McCabe’s cyclomatic complexity/module, to assess the maintainability of the fMRP family of software. Using the values from the Halstead and McCabe metrics, the team applied a set of field proven polynomial metrics to calculate the maintainability indices for each software baseline. The polynomial metrics were developed at the University of Idaho and have been field-validated by Hewlett-Packard [COL92]. Based on Hewlett-Packard’s experience with software, a software package with maintainability index less than 65 is considered to be “difficult to maintain”. The research team applied the maintainability index to the fMRP family of software to provide comparison and perspective and to verify that Ada together with modern software technologies had helped the company in reengineering its software development and business.
5.2 Software Complexity
A post-project study had documented increased complexity in many software baselines. The trends were reflected in the maintainability index which showed a nonlinear decrement for the `C’ version of PFMM and GTM from version 1 to 2. Both Ada versions of PFMM and GTM had indicated that maintainability had increased. The maintainability of PFMM had increased significantly due to reengineering of the software concurrency as the development teams were getting more familiar with Ada tasking.
The size of the Ada modules was much smaller than the modules written in COBOL, FORTRAN and C. The FORTRAN MRM baseline was comprised of 86,000 lines of code, with typical use of global data access and FORTRAN subroutines that is common with functionally decomposed software design. By comparison, the Ada implementation of MRM was comprised of 62,000 lines of code. In contrast, the size of the Ada modules was much bigger than the modules written in Paradox (PAL), Clipper and Pascal (Delphi). The Delphi FAM was comprised of 23,000 lines of code. By comparison, the Ada implementation of FAM was comprised of 74,000 lines of code. The reason for this difference was the Rapid Application Development (RAD) environment in which the system support modules like GUI and data access were completely transparent to developers. As in Delphi’s RAD environment, an input screen can be developed without the developer writing a single line of code.
5.3 Software Reuse
The team realized a “reuse rate” of 62%, which means that 62% of Ada code modules were reused in one or more applications. This reusable Ada code was the reason for reduced software complexity and lines of code in the overall system and led to more maintainable software.
Smaller, easier to understand modules enabled the use and exploitation of the reusable layered software architecture, with explicit support for the object-oriented design employed by the development team. Reusable software modules had lowered software complexity and increased the responsiveness and quality of support, as the code was more readable and understandable.
5.4 Speed of Development
The speed of development was not taken into account in this reengineering project. This was due to the development team having to learn the Ada programming language and concurrently develop Ada codes. Both the research and development team agreed that the timeline was very dependent on different learning curves of the staff and on the availability of certain technical information in order to develop bindings to hardware peripherals and operating system software.
6 The Benefits of Ada
The benefits of Ada in terms of its explicit support for software engineering discipline and object-oriented design have been cited as major factors in the success of this reengineering project. The Ada language features provided excellent support for the implementation of layered software architecture, as well as the attainment of significant levels of code reuse. Furthermore, Ada’s strong typing, parameter checking in subroutine calls, and array constraints contributed to the benefits achieved in this reengineering effort.
In addition to these powerful attributes and benefits, the clarity of Ada source code combined with the structured nature of its package specification and more descriptive names of variables, has a profound effect on software reuse and documentation. Ada software implementations are virtually “self documenting” due to the explicit representation of the functions and interfaces of the respective modules. This additional clarity of the Ada source code pays substantial rewards in the area of maintenance and modification of the software.
Having tasking construction as an intrinsic part of the language, Ada provides greater reliability. Concurrent activities can be identified explicitly in the code, resulting in additional clarity of Ada source code which is more understandable and maintainable. In general, every operating system is different and does not provide the control and timing needed by many applications. Ada tasking solves the problems by providing control and timing needed. Concurrent applications written in Ada are more portable and modifiable.
Ada is obviously an excellent programming and design language and the development team undeniably concludes that Ada has been the success factor in this reengineering effort and has helped the company to survive the brutal software lifecycles and the competitive business. The company will continue the use of Ada in all future software projects and hope that this experience report exemplifies for the interests of organizations or individuals who wish to transition to Ada.
|[ALB83]||Albrecht, A.J., and J.E. Gaffney, “Software Function, Source Lines of Code and Development Effort Prediction: A Software Science Violation”, IEEE Trans. Software Engineering, Vol. SE-9, no. 6, Nov 1983.|
|[JON92]||Jones, C., “Measuring Software Productivity and Quality”, 1992.|
|[BOO94a]||Booch, G., and Bryan, D., “Software Engineering with Ada”, 3rd Edition, Addison Wesley, 1994.|
|[BOO94b]||Booch, G., “Object-Oriented Analysis and Design With Applications”, 2nd Edition, Benjamin/Cummings, 1994.|
|[COA97]||Coad, P., North, D., and Mayfield, M., “Object Models – Strategies, Patterns, and Applications”, 2nd Edition, Prentice Hall, 1997.|
|[RUM91]||Rumbaugh, J., Blaha, M., Premerlani, W., Eddy, F., and Lorensen, W., “Object-Oriented Modeling and Design”, Prentice Hall, 1991.|
|[BUR95]||Burns, A., and Wellings, A., “Concurrency in Ada”, Cambridge University Press, 1995.|
|[BAS78]||Basili, V., and M. Zelkowitz, “Analyzing Medium Scale Software Development”, Proc. 3rd International Conference of Software Engineering, IEEE, 1978.|
|[OMA92]||Oman, P. “Construction and Validation of Polynomials for Predicting Software Maintainability”, University of Idaho, 1992.|
|[HAL77]||Halstead, M., “Elements of Software Science”, North Holland, 1977.|
|[MCC76]||McCabe, T.J., “A Software Complexity Measure”, IEEE Trans. Software Engineering, vol. 2, no. 6, December 1976.|
|[MCC89]||McCabe, T.J., and Butler, C.W., “Design Complexity Measurement and Testing”, CACM, vol. 32, no. 12, December 1989.|
|[COL92]||Coleman, D. “1992 Software Engineering Productivity Conference Proceedings”, Hewlett-Packard Company, 1992.|
This paper is published in the conference proceeding of 7th International Conference on Reliable Software Technologies, Ada-Europe 2002, Vienna, Austria.
Copyright © 2002 Springer-Verlag