User acceptance testing for AMI implementation: A perspective

An approach to user acceptance testing for an AMI is reviewed by Suhas M Kharad, delivery head at L&T Technology Services (LTTS), the company responsible for delivering one of India’s first large-scale smart metering projects.

In India, the state power departments and distribution companies (discoms) selling power directly to consumers recorded a year-on-year growth of 5.49%, from Rs371,959 crore ($44.7 billion) in 2014-15 to Rs392,398 crore ($47.1 billion) in 2015-16.

The total energy sold by these utilities was 753,436MWh in 2014-15 and 789,512MWh in 2015-16, a 4.49% year-on-year growth. However, the aggregate technical and commercial (AT&C) losses in those years were 25.72% and 23.98%, respectively.

In the latest report on the performance of power utilities in 2020-21, the AT&C losses of the discoms were recorded as 20.73% in 2019-20 and 22.32% in 2020-21.

To address this scenario, the government of India launched the AMI programme to replace 250 million conventional meters with smart meters across all households by March 2025 and to revamp the whole power supply infrastructure to reduce the AT&C losses to 15%. The revamp included the two way-communication network to control centre equipment and all the applications that enable the gathering and transfer of energy usage information in near real-time.

Have you read?
Smart metering – a powerful tool to enrich the consumer experience
Report identifies top performing states in India’s electricity transition

AMI makes two-way communications with customers possible and is the backbone of a truly smart grid. It seeks to remotely receive meter reading data, streamline network problem identification, drive load profiling, enable energy audits and partial load curtailment in place of load shedding.

System reliability, energy costs and electricity theft are the other three governing factors in driving the implementation of the AMI programme.

Smart metering for Uttar Pradesh, Haryana and Delhi

As part of this AMI programme L&T SmartWorld & Communication, now named L&T Technology Services (LTTS), was awarded one of the first and largest AMI projects in India on March 2018.

The scope was to install 5 million smart meters and to implement both post-paid and pre-paid functionalities, as well as to develop a consumer portal, mobile app, helpdesk and analytics across three states, Uttar Pradesh, Haryana and Delhi (NDMC).

As a system integrator, we felt fortunate and proud to be part of the country’s first and successful massive digitalisation programme.

Indicative of the complexity, the system/components involved in an AMI include smart meters, meter data acquisition system (MDAS), meter data management system (MDMS), consumer indexing and meter installation system (CI & MI), warehouse management system, web portal and mobile app, analytics, integration with utility billing applications and other discom applications such as a GIS system, outage management system, etc.

User acceptance testing in AMI

The AMI backbone relies on intensive testing to ensure a streamlined rollout and adoption.

Before the handover of the application to the customer, there is a clear need to undertake complete module level testing, integration testing and system testing. This would be followed by user acceptance testing (UAT).

We suggest the UAT preparation can start in parallel when the requirement specification is finalised.

The UAT strategy is a crucial document and should include the scope of testing, test coverage, assumptions, constraints, dependencies, approach, entry-exit criteria, suspension criteria/resumption requirements, defect logging and tracking mechanisms, requests from customers, acceptance criteria, governance, roles and responsibilities, environmental test setup, risk and mitigation plan, UAT committee, SMEs, RFP compliance, finalisation of status report template, final report template, etc.

Once all the relevant steps have been completed, we can then move to the actual UAT. Based on RFP clauses the testing may vary.

Live meter installation testing

This testing is a live testing session where all the committee members would be on a video call and the systems integrator is asked to start the process right from consumer indexing to meter installation, warehouse management receipt, fixing of meter, making live in the meter data acquisition system, then to the MDM, generate the billing in case of prepaid, meter fault, allocation of workforce management, etc.

The tables shows indicative high-level functionalities for testing and the steps involved in live testing with our recommendations.

System/module and end-to-end integration testing

This involves creating a plan based on pre-UAT with certain assumptions such as availability of full-time committee members, number of holidays, time allocated for testing, number of UAT cases prepared per system/component, etc.

We recommend that the systems integration team create a separate UAT environment rather than use a QA environment that may hamper the development and support activities.

The test case results obtained can be segregated into five categories, such as ‘passed’, ‘passed with observations’, ‘failed’, ‘functionality not available’ and ‘functionality not demonstrated’.

RFP compliance

This RFP compliance is optional, but customers do ask for it to ensure that the systems integrator has completed its development portion as per the RFP.

To our understanding, this mainly happens when a lot of confrontation occurs and/or there is a lack of trust. However, we suggest that the systems integrator should be prepared for the worst-case scenario to avoid last minute delay in UAT signoff and revenue recognition.

Conclusion

In conclusion, we would say that systems integration teams looking to drive UAT scenarios in the AMI ecosystem should focus and each respective SME must play its role diligently.

The test manager/test director should act as a binding force and work towards the end objective.

Streamlined teamwork, combined with deep technological expertise, will help drive success in the emerging power distribution landscape.

About the author
Suhas M. Kharad works as delivery head in L&T Technology Services - Sustainable Smart Work BU. He holds MSc Tech. and MPhil degrees in Geology. He has 30+ years of IT experience and has worked on multiple domains and technologies. As test director, he carried out the user acceptance testing as a first of its kind for an AMI programme in India.