Our client, a Fortune 500 subsidiary, collected sensor data from garbage trucks but wasn’t using it for insights. They needed a solution to improve fleet performance, reduce fuel use, and enable preventive maintenance. Using Power BI, NeenOpal built interactive dashboards to reveal performance metrics, patterns, and machine learning results like anomaly detection and clustering.
Organizing and processing the sensor data was challenging due to its complexity and inconsistencies, requiring significant effort to prepare it for analysis. Here are the key challenges:
Each garbage truck had hundreds of sensors collecting data on wheel speed, positions, pressures, temperatures, and switch states. However, no insights were being gained because the data wasn’t synced and lacked a consistent framework. While all the sensor data was stored in a common database, the data had issues like inconsistencies, missing values, and required changes. To make the data usable for analysis, it needed to be synced, columns had to be combined, and other transformations were necessary.
As each sensor was capturing data at different timestamps, data points across trucks or across different timeframes were difficult to compare. With such misalignment, meaningful insights and comparisons were not possible to attain.
To address the challenges with sensor data, a comprehensive approach was implemented, focusing on synchronization, analysis, and visualization. Here are the key steps taken to transform and extract valuable insights from the data:
NeenOpal came up with a data structure to synchronize the same timestamps among the sensor data. This would allow cross-truck comparison. We stored and processed data using Snowflake through data transformation using both Python and Snowflake SQL.
We applied correlation analysis through Python to observe the interrelation between pairs of sensors. This helped the company determine how different components are performing across different trucks under different conditions.
We developed a series of interactive Power BI reports to help visualize the processed sensor data. These reports allowed users to compare sensor interactions between trucks, monitor individual sensor values and real-time data volumes, and track sensor frequencies and detect out-of-range values. They also provided insights into sensor-value distributions across trucks using kernel-density estimation and quantiles, along with timeline trends and comparative sensor values across various time aggregates such as seconds, minutes, and days.
Machine learning algorithms like isolation forest, k-means clustering have been deployed to perform anomaly detection to identify out-of-range timestamps in the data as potential early signs of fault indicators to prevent failure.
We developed custom algorithms that would calculate average fuel consumption per truck errand. This way, correlating different sensor data points, we could estimate fuel consumption during different errands, compare trucks, and benchmark against competitors.
As a Microsoft Solutions Partner with expertise in Snowflake and Power BI, NeenOpal helps businesses overcome data challenges by syncing data, creating custom reports, and developing tailored metrics. Our goal is to help businesses harness the full potential of their data, driving success and achieving their objectives.
A Business Intelligence (BI) roadmap is a plan that guides how to set up and use data analytics tools and processes to gain valuable insights and make informed decisions. It helps organizations build a clear path for integrating data solutions that support long-term growth and success. By aligning technology with business goals, it fosters efficient data management and drives continuous improvement. Learn more.
By using NeenOpal’s analytical solution, the client transformed raw sensor data into actionable insights, enhancing fleet performance, reducing costs, and streamlining operations. The solution delivered the following key improvements:
By using Snowflake for data storage, processing, and analysis, and Power BI for reporting, the client gained real-time visibility into their fleet operations. Through remote sensing data synchronization, correlation analysis, and dashboard development, our client was able to track and optimize truck performance, fuel consumption, and maintenance schedules. Custom anomaly detection and fuel consumption metrics played a crucial role in enhancing operational efficiency.
Senior Data Scientist
Content Writer
