This is a new methodology in testing. This new methodology is being used for testing the Motorola CDMA 1x Radio Access Network and is driven by the need to select test cases more effectively and more efficiently and execute them to identify maximum defects as early as possible. This methodology is also focused in aligning defects finding pattern to the type of defects customers would face.

Test Efficiency: We were missing how to identify effective test cases to cover new code changes to do regression testing for a system with huge code base. We came up with the idea – Test Efficiency Tool. Test Efficiency Tool is a historical repository of all files that were modified due to a defect linked to a test case. When the file linked to a test case is modified, the test case is prioritized to be executed earlier in the test cycle.Test Efficiency Tool is developed by generating a database for historical source code file changes for each test. This mapping is then used to identify tests for the modified files in the code for the new load.

Test Effectiveness: We also needed a better way to identify test cases and rank them as some test cases are effective at finding defects and the others are not as effective. We defined a ratio that impacted the high severity defects finding rate the most. We called this the “Test Effectiveness Ratio” (TER). TER is defined as the number of high severity defects (P1 and P2) found per test case, and some other factors such as Defect to Execution Ratio, etc. Using TER, test cases can be ranked according to their ability of finding defects. It was found that the TER would have resulted in finding more high severity defects earlier after analyzing the data for past few releases. Test Effectiveness Tool (TET) generates a Test Effectiveness Ratio for each and every test case in the database. Using Klocwork File Risk Analysis (KFRA), again code files are ranked.

With KFRA and TER, we prioritize the test cases selected by Test Efficiency Tool. More effective and efficient test cases can be targeted.

Amalendu, Sr. Staff Engineer, is responsible for test system engineering and test lead role for Motorola CDMA Infrastructure Software release validation and has worked on Telecom Software testing for more than 10 Years with different Telecom Vendors..He is working on improving testing methodologies and test tools development.He is a B.Tech from University of Calcutta and has worked with Alcatel on Long Distance Career Software Testing and with Nortel Networks on Intelligent Networks Testing.