Published on the 17/07/2024 | Written by Heather Wright

Open source key to attracting and retaining talent…

A 10-year modernisation program has brought Transpower New Zealand some significant financial wins – but it’s also helped the organisation on the crucial talent battlefront.

The extensive modernisation work to a key system using open source saw New Zealand’s national electricity grid owner and operator avoid costs of around NZ$50 million to rewrite the system, and provided a two-year payback.

“It’s not a niche technology that no one wants to work on.”

But while those benefits are significant, Arosha Aluwihare, Transpower New Zealand head of application services, information services and technology, says one of the biggest benefits has been the ability to attract and retain talent, through using open source, rather than the niche, proprietary software, which came with limited numbers of experts to work with the complex architectures.

“It is easier to find skill sets and talent around open source technology, versus proprietary,” Aluwihare told iStart. “It’s not a niche technology that no one wants to work on and that’s a huge benefit.

“That was really important as it is a critical application and we want to retain the knowledge.”

The modernisation work revolved around the Market System, a cornerstone for the organisation and used to ensure a secure power system and the real-time dispatching of electricity prices.

“It was a monolithic system with some complex business logic, which was written in PL/SQL. There was accumulated technical debt, we had software systems that were reaching end-of-life, which was the biggest driver, and our system processes at the time were mostly manual,” she says.

While replacing the Market System completely was considered, it still provided the functionality needed – it was simply the underlying technology that needed to be optimised. Off-the-shelf packages aren’t an option, requiring the system to be rewritten.

“As a result of that, the cost was significantly more than if we modernised it.”

With an eye on the five to 10-year total cost of ownership, Transpower found a ‘significant’ price differential between commercially supported open source software versus proprietary offerings.

“The first thing we modernised was the enterprise service bus, based on Red Hat Fuse, used to talk to other systems, rather than point to point integration. It was the first thing that went on to the open source stack,” Aluwihare says.

That porting of the corporate ESB technology to Fuse achieved a two-year payback period for the organisation.

The work was completed in phases. Following the ESB work came a transformation of PL/SWL to Java to put it on Red Hat OpenShift.

The dispatch system, which pricing every 30 minutes, was also put on OpenShift.

Using automation ensured Transpower could build and rebuild environments from source code.

Lessons learned during from the dispatch project allowed Transpower’s teams to confidently begin the modernisation of the architecture of the rest of the Market System, also built on OpenShift.

“Doing it in discrete pieces of work, rather than a big bang approach which is hugely painful to undertake and a challenge to be successful, was another reason for our success,” Aluwihare says.

“Don’t get me wrong, it wasn’t a walk in the park, all projects are challenging to deliver. However, when you phase it and have discrete pieces of work, your ability to focus is better, your ability to contain something is better and therefore your chances of success are greater.”

Alongside the cost and talent value propositions was another aspect, one Aluwihare admits they didn’t put into the selection, but realised afterwards: Getting access to source code.

“It is really helpful for us when we have to troubleshoot. With a proprietary system it is a black box and most of the time you’re having to go to a vendor who may or may not respond in a timely manner.

“With open source, we can get into the root cause quickly.”

Aluwihare says the story, however, isn’t just one of technology change for Transpower.

“Our system processes went from manual to more automation, we adopted more agile ways of working, we built capability on the back of this transformation in house.”

The company had previously outsourced most of the work – something which proved problematic when it came to retaining talent. Today, most of the talent is in-house, with the Market Systems development team growing from three to more than 20 in a move that reduces organisational risk and enables Transpower to scale delivery and take on large industry changes such as real-time pricing.

“We have good retention of talent because it is interesting work, good technology.”

There were benefits in terms of the learning curve too, Aluwihare says “to be honest, technology change is technology change – the learning curve is quite generic if you change a technology in your stack’.

“The difference is that with open source technology you have more people who can learn it because it is not a black box. There are more people interested in learning it and more people you can recruit and there is a bigger community and forums which support faster adoption.”

As new power generation types and new technologies connect to the grid, Transpower says a stable, flexible platform will provide it with the ability to adapt and take on complex challenges.

In 2022 it launched real-time pricing, allowing generators, retailers and consumers to make decisions in real time about their energy consumption or generation. The integration of tools like modernised dispatch applications with enhanced capability in the Market System software has played a key role in making it easier to provide accurate and timely prices essential to emerging battery storage and smart appliance technologies.

“Our story is one of transformation on the people side, the process side, the tools and technology and the architecture, and bringing it all together in a little over a decade.”