How TotalEnergies PGE improves its data mining

The Total group was renamed TotalEnergies in May 2021, marking the evolution of its businesses, moving exclusively from gas and oil to all forms of energy. It now has four branches: exploration-production, refining-chemicals, gas-renewables-electricity (including the subsidiary TotalEnergies Electricity and Gaz France, formerly Direct Energie) and marketing-services. The gas-renewable-electricity (GRE) branch serves 5.5 million customers and is targeting 7.8 million customer sites and 78.6 TWh delivered by 2025. As in many companies, data was underused with concerns about processing performance, cost, volume growth and production lead times. An overhaul was needed. Head of IT solutions data and exchanges at TotalEnergies Power gaz Europe France, Ridha Nabli testified at the invitation of Snowflake.

The GRE branch wanted to have a real-time platform, which required redesigning the data model to have satisfactory performance, in particular by reducing the number of tables. Several technical solutions were then considered to house the TotalEnergies GRE data: Amazon Redshift, Google Big Table, Snowflake, etc. Based on Keyrus’ expertise, TotalEnergies GRE created a demonstrator based on Snowflake which demonstrated its relevance. technological. Snowflake is indeed in SaaS mode, scalable on the fly [scalable], clonable and efficient. With this solution, TotalEnergies GRE can have 90 days of online data and, above all, can, thanks to the data sharing, share data between users without moving or duplicating it. With other solutions, the cost of hosting data could reach $135/TB (124 euros) against $23/TB (21 euros) with Snowflake. Data integration as events happen using Talend’s solution, including extraction from SAP by simulating an interaction with the BW module.

Ridha Nabli, IT data and exchange solutions manager at TotalEnergies Power gaz Europe France, presented the general architecture of the project.

In the end, TotalEnergies GRE now has a single database with a datalake under Snowflake whereas, previously, certain uses involved replications (such as decision-making under Qlik). The data is directly usable there, without an intermediary datamart in most cases, even if Cassandra is used for some NoSQL uses, in particular for mobile applications. The solution has many uses, such as managing charging stations, developing predictive non-renewal models (churn), forecasting consumption and coaching customers to optimize their consumption.

Framed in the fourth quarter of 2020, the redesign project was carried out over two years in batches. Each batch lasted a maximum of three quarters. When a batch was delivered, the old tool corresponding to the same use was immediately decommissioned. Among the elements of the project’s success, Ridha Nabli insisted on driving change so that everyone takes ownership of the solution, with user training having started in the third quarter of 2021.