Pursuit Software performed an initial assessment which revealed the following:

Percentage of manual test cases automated: Approximately 35% of test cases for the web interface had been automated using standard web automation techniques, while no automation had been created for the desktop interface. Moreover, selection of automation candidates was done by focusing on certain specific areas of functionality, instead of critical, high visibility areas from an end user perspective.

Visibility to test progression and results: While the client was using an off-the-shelf test management tool for manual testing, there was no clear mechanism for sharing the testing statistics. This also provided a partial picture since the test suites had a mix of manual and automated tests, which were being executed using a de-facto test execution environment. Since no clear mechanism existed for providing a snapshot of the test results, test failures were not being addressed over long periods of time, thereby impacting the overall quality of the claims being administered.

Based on the findings, Pursuit implemented the following steps:

Created an automation road map clearly outlining the prioritized areas to be automated – covering both the desktop and web interfaces – over specific time periods.

Created a preliminary dashboard to display the status of manual and automated test progress, that would be accessible to all stakeholders. The dashboard incorporated specific levels of abstraction to cater to the target audience (senior management, mid-level management, engineering).