This article will help software testers to gain an understanding of the specific practices that are commonly practiced by financial service providers in financial software testing of banking and financial services applications. Banking applications include core banking, retail, private, corporate, investment, credit card applications and financial services applications like payment gateways, mutual funds applications.
Author: Anindita Chandra
The novice testers will gain an understanding of the practices and the experienced software testers will find a correlation with the practices practiced in their financial software testing projects. In this article, I explore some of the specific financial software testing practices practiced commonly by the financial testing service providers.
Understanding / knowledge of system(s) from business viewpoint
Software testers that have domain knowledge of financial system are the most suitable candidates to test finance systems. Testers of related domain expertise could also be considered in case, if there is not enough specific domain expertise. Retail banking personnel for Rates STP (straight through processing) project is an example of related domain expertise. Mandatory domain training sessions are arranged for those testers.
On the project start, a knowledge transition plan can be prepared with inputs from the client and old vendor based on an “As Is” assessment. After the team is formed, knowledge transition is provided either/both by the client or/and the old vendor about the business functionalities of the IT systems and business workflows between different IT systems that are part of the test scope. This way, testers gain the understanding and knowledge of not only how the system works but also about the clients’ business system landscape.
For maintenance projects, the testers can evolve and gain more in-depth knowledge interacting with the business analysts and development team. The knowledge base system that stores business requirement documents, technical design documents, regression test suites are also used for detailed understanding of the system. Regression test suites cover the entire core system end to end business scenarios and this complies with how the business users use the integrated IT system real time in production.
End to end testing between downstream and upstream system interfaces (e.g., front to back office testing)
End to end testing involves integration and regression testing of complex financial applications that emerges from the interdependencies between the different systems. To illustrate how the end to end testing works, let us consider the example of “Rates STP”, it uses 3 different systems for the front (FO), middle (MO) & back office (MO), each being handled by a different vendor. System integration and regression testing in front to office testing means that the transaction flows from the first system to the third system via the second system. Communication between the systems happens through XML files referred to as messages.
Pre-requisites to integration testing between systems include signoff of the 3 component systems, release build deployment complete of all systems, recent data dump migrated, connection established and check between the 3 systems and sanity test passed. FO should do trade booking in batch during Integration testing; the test pack should be updated with trade details so that once, the trade reaches third system via second system, the BO vendor can validate it at their end. Complexity of integration testing lies in getting issues resolved that are found during integration testing. Resolving integration issues requires coordination between multiple vendors and identifying what is the root cause of issues and at which stage of the whole flow did the issue occur. Example of issues can be either the trade is not booked with correct details and hence require rebooking by first vendor, trade booked did not reach the third system, trade booked reach the BO, but does not display appropriate trade details though the trade is booked correctly at the FO. Based on the work processes set the issues are logged and directed to the right team for resolution
Multiple vendor coordination
When multiple vendors outsourcing is practiced, the workload to finish different parts of the project is outsourced to more than one vendor. As opposed to prime contracting where vendor takes sole responsibility for identifying what might be subcontracted, here, the client will outline the list of tasks to vendors thus they need to have full picture of the project. Since the vendors are not responsible for other vendors’ performances, ensuring smooth cooperation from such vendors on integration between systems may become a major issue. To ensure smoothness of cooperation between the vendors and thus achieve success in integration between processes, it is essential to have:
- Demarcation of responsibility for vendors where task dependencies exist. The contractual agreement should clearly mention the support agreement required from each of the vendors to achieve success in integration testing. The client can take the responsibility of integration or assign it to one supplier.
- Many points of contacts rather than single point of contact. Integration team is led by the representative of teams of business analysts, development and testing of the systems handled by different vendors. To ensure that the same information is sent to the information team, a distribution list of the representatives can be created for communication purposes. Daily meetings can be scheduled between the different test team vendors and client to check upon each other’s progress and discuss existing issues.
- Setting up the common standard and platform is necessary. Having structured work processes and procedures must take place from the very start i.e., for planning the entire project and not at integration stages. This include process sequence for artifact maintenance by different vendors, process coordination for integration activities – log integration issues, tag them and assign to appropriate vendor representative following integration process and procedure.
- Standard report template defined for the project must be used by all the vendors to update their component, integration and system testing status. Each vendor can update their status by checking out the most recent previous status report last updated from the configuration management tool without overriding each other’s status and thus giving a complete picture of the project status and communicated to the integration team.
- Customer plays an active role in monitoring and controlling intra-vendor issues, thus client is involved in operational activities rather than just playing the strategic role.
Regression testing priority over new enhancements and risk based regression testing
Regression testing is performed with every new release during maintenance phase in which new enhancements are added to the existing business functionality that exists in production to ensure the application runs smoothly and reliably in the absence of which, critical business functions can be disrupted causing loss to the business. Delay in bug detection can lead to fix them at cost that is 10 times higher. New enhancements incorporated with the new release do not already exist in production and thus are low priority over regression functionalities.
When there is not enough time and resources to test all new enhancements and regression suite, then high priority enhancements and regression suite becomes priority.
Regression testing consumes large amount of time and accounts for almost half of the software maintenance costs. The regression effort can be minimized without sacrificing regression effectiveness by selective test regression: 1) This includes identification of affected parts – this involves identification of unmodified parts of the program that are affected by modifications. 2) Identification of subset of tests which can effectively test unmodified parts of the program that has potential to detect errors created by the changes.
Reprioritization of regression suite is essential in context of every release based on the new changes introduced. Optimization of the regression tests can be achieved by avoiding coverage of same requirement in multiple tests and minimizing the testing efforts by testing related requirements together.
Test Automation can further minimize regression testing effort and help accommodate the needs for increased test coverage without impacting timelines, creating more time for resources to focus on other important tasks.
Risk based regression testing prioritizes testing of business functionalities based on risks. The risk exposure of each scenario can be weighed by summing up the weights assigned to each of the risk analysis factors. The business impact factor of each scenario can be weighed by business criticality, frequency of usage, application stability and likely hood of failure can be weighed by defect history and environmental factor. For example, business criticality weight can be 1 for business loss up to 5K USD, 2 for USD 5K to 10K,…..,5 for business loss of USD>50K etc The sum of risk analysis factors must equal 100%.
Table 1. Risk based regression testing prioritization
Selective regression technique along with risk based regression test selection will be the most effective regression strategy. Keep the regression test suites always up to date by identifying and adding the new enhancement core business test cases to the regression test suites. For defects/issues found in production, test cases must be added to the regression suite to ensure testing in future release cycles.
Verification of financial calculations
Testing of financial applications always include verification of financial calculations. Manual testing of huge financial calculations is a tedious and consuming process. Developing a macro to automate the test speeds up and completes execution in few seconds can save lots of resource time. The developed macro can be re-used to regression test for every release to verify all the financial calculations. Take as an example an application that has 8 business entities and for each entity, we need to verify the financial calculations for 8 line numbers and every line number is derived from combination of 5/6 different inputs. Macro can be developed using excel where in based on entity selection, navigates to the particular entity screen; update input values in the placeholders for the inputs. The output placeholder for each line number gets calculated and updated with the output values derived from the respective formulae’s using input values. If some of the functionality changes involve modification to any line number formulas, the macros need to be updated accordingly for it to be re-usable.
Scarce resources
The projects require delivering short releases that have both tight deadline and limited budget. Due to limited budget, software test resources are limited and thus every tester handles the testing of one or more application functionalities. Testers are cross-trained on applications that are handled by other resource(s). Once cross-trained, they become primary resource for their one or more application(s) and secondary resource for other application(s). A primary resource is more knowledgeable and has more expertise over a secondary resource.
In case of resource absence/attrition, the applications which were handled by those resources are reassigned to other primary/secondary resources and their primary task gets shared between them and other resources which have been cross trained. In some projects, there is limited number of accesses to test system and thus testers might be required to work on different shifts.
Summary
This article summarizes on some of the specific financial software testing practices and discusses the different tactics and techniques like end to end testing, multiple vendor coordination involving different systems, selective/ risk based regression testing technique, automation of regression suites to reduce execution time, usage of macros for financial calculations, cross training program etc that are used to manage software testing activities with scarce resources and within tight deadlines of short releases without sacrificing on software testing efficiency.
About the author
Anindita Chandra has 14 years of IT experience with 10 years in independent software testing. She has leaded the execution of numerous financial application development / maintenance testing projects primarily in financial services domain.