the software side of warehousing series graphic

By Elvin Johnson, Data Scientist, Lucas Systems

AI data on laptop

Artificial intelligence is transforming how businesses solve problems. With the help of AI, organizations are finding new ways to make faster decisions, boost efficiency, and uncover insights within their operational data. But behind every successful AI solution is something much less glamorous: data. AI models don’t simply know how a business operates; they learn patterns from data made available to them. In a real-world scenario, that data is rarely perfect. Data can be incomplete, contain unusual observations, or may have been collected for an entirely different purpose than the application it’s being leveraged for. In this blog, we are going to talk about some real-world data issues.

At Lucas Systems, we work at the intersection of technology, data, and warehouse operations, where our customers generate a large amount of operational data every day. Recently, we collaborated with graduate students at Carnegie Mellon University on a data science project exploring how historical warehouse data could support areas of warehouse execution.

Consider this scenario: a warehouse has hundreds of activities happening every shift, with several factors influencing how efficiently its operations run. With so many activities happening throughout a warehouse every day, making the right operational decisions can become quite challenging. Warehouse supervisors have to consider multiple factors when planning and managing their operations, and having the right information available can make a big difference. This is where we saw an opportunity to use data and explore how advanced algorithmic planning could help support better decision-making. However, solving this problem involved much more than simply training a machine learning algorithm. There was a significant amount of data to analyze and understand before we could even begin building models.

How do we make sure we have relevant data for our application?

As mentioned before, when leveraging AI to solve a business problem, the data that’s available is not often collected for the application you’re trying to solve. Excessive, unwanted data could end up hurting the performance of the model being developed. The first step, therefore, is to do exploratory data analysis (EDA) to understand the data at our disposal better. Exploratory data analysis involves doing analysis on the data with the context of having domain knowledge for the field the data originated from. Data analysis many times exposes outliers/anomalies and bad data. It’s important to separate out outlier data from bad data. Outlier data is data that is expected, while bad data is unwanted data that made it’s way into the data due to various reasons. Bad data can end up harming downstream AI models, while outlier data is expected in real life and must not be discarded. This illustrates an important principle for data quality and AI: cleaning data isn’t about making a dataset look perfect. It is about understanding which information represents the behavior you want the model to learn, which needs domain expertise.

What do we do with historical data that contains signals that won’t be available in production?

Another important thing to consider when developing an AI model is that all the datapoint/features we have collected historically may not always be the same data we have available when the model is actually being used. This is something we ran into during our work. Some of the features we had within historical data were very useful for prediction, but would not necessarily be available when making a prediction in production.

Instead of completely ignoring these features, we can still use them to understand the problem better. We can first use all the information available historically to create a benchmark for how well the problem can potentially be solved and understand which features are most important. From there, we can remove the features that won’t be available in production and see how much the model performance changes. This gives us a much more realistic understanding of what we can expect when the model is actually deployed. It’s easy to build a model that performs well on historical data, but that performance doesn’t mean much if the model depends on information we won’t have when we need to make the prediction. So, the question shouldn’t only be: does this feature improve the model? We also need to ask; will we actually have this information when we need it, and if not, how could we use this information to better understand our use case?

What do we do to make the most of the data we have?

Picking up from the previous section, once we remove information that won’t be available in production, we may find ourselves with fewer useful features than we would like. This is where feature engineering can be really useful to help boost model performance. Feature engineering is essentially taking the data we already have and using it to create new information that can better represent the problem we’re trying to solve.

One example from our work involved identifying useful information that wasn’t directly available as a feature in the original data. While we had a large amount of historical operational data, it didn’t necessarily contain every piece of information that could help improve our predictions. What we did have, however, were patterns in the data that we could analyze to better understand the different factors influencing warehouse operations. We explored ways of using these patterns to create additional features that could help improve model performance. This is why feature engineering can be so important when working with real-world data. Sometimes we don’t have the exact data we want, but we can use the data we already have to create a good representation of it.

Analyzing the story the data is telling with the business outcome in mind

It’s also super important that the metrics we use to evaluate our models actually align with the business problem we’re trying to solve. When working on machine learning models, we typically use different metrics to understand how well a model is performing. While these metrics can tell us how accurate our predictions are, they don’t necessarily tell us how useful those predictions would be for the business. This is why it’s important to understand what we’re actually trying to achieve before deciding how we evaluate the model.

This was something we had to consider during our work as well. We looked at different ways of evaluating our models and how those results related to the overall problem we were trying to solve. Something we realized was that looking at individual predictions alone doesn’t always give us the complete picture. We explored alternative ways of evaluating the predictions by looking at how they contribute to the larger business objective, rather than just measuring how accurate each prediction was individually. Interestingly, we found that even when individual predictions weren’t always as accurate as we’d like, the model could still provide useful and reliable results when evaluated in the context of the broader problem. This allowed us to get more value out of the same model without necessarily having to improve the accuracy of every individual prediction. At the end of the day, building a model that performs well according to a technical metric is one thing, but making sure that model can actually help solve a business problem is what matters.

Ultimately, AI is only as valuable as the data and context behind it. Real-world operational data is messy, incomplete, and often collected for purposes other than model development, but that does not make it unusable. With the right combination of exploratory analysis, domain expertise, thoughtful feature engineering, and business-aligned evaluation, organizations can turn imperfect data into practical insights. The goal is not to build a model that looks perfect in a lab; it is to build one that helps people make better decisions in the real world.

Elvin Johnson headshot 2026

Elvin Johnson is a Data Scientist at Lucas Systems, where he applies advanced machine learning and AI to transform warehouse and supply chain operations. He earned his Master’s degree in Electrical and Computer Engineering from Carnegie Mellon University, specializing in Machine Learning and Data Science.

Elvin has completed multiple engagements across industry and academia, contributing to data science and deep learning projects spanning end-to-end pipelines, from data collection and model training to deployment. His research background includes co-authoring three machine learning publications, as well as hands-on experience with a wide range of ML and data science applications across diverse domains.

Passionate about leveraging technology for the greater good, Elvin continues to explore innovative ways to apply data-driven solutions to complex real-world challenges.

Share This Story, Choose Your Platform!

Continue Reading

Ready to create an agile, optimized DC operation?

Let’s turn your distribution operation into a competitive advantage, together.