The process: AI edition

Executive Summary

In this article, I discuss the challenges and intricacies of deploying AI technologies in business environments. I highlight the unpredictable nature of AI projects, especially those involving advanced techniques like machine learning, emphasizing the need for business leaders to adjust their expectations and timelines. The piece advocates for patience and strategic foresight in navigating the complexities of AI integration.

Let’s break it down

Previously, I discussed the process of breaking down an emerging technology to help you understand it and maximize your use of it in a way that makes sense for your business. Let’s talk about AI—not just GenAI but ML, computer vision, and GenAI.

All of these versions of “AI” (we can discuss the incorrect use of that term some other time) have a common trait that makes them incredibly difficult for business leaders to understand: lack of predictability. This trait spans all phases of learning about and implementing these technologies.

Let’s back up for a second and talk about other types of projects and how they typically go.

“SIMPSON! Get in here!”

“Yes, boss?”

“Simpson, we need a new section of our app that shows our customers how great we are!”

“How great we are, sir?”

“YES!”

“You mean… something like showing them how much money they have saved by shopping with us?”

“PERFECT!”

“On top of it, sir!”

Simpson would then go off and talk to all of the normal cross-functional teams. Engineering (both back end and front end), UI/UX, Product, Marketing, Legal, etc. Once Simpson collects all of the estimates on how long things will take, they can add them up and have a reasonable pass at how much it will cost and how long it will take.

Now, let’s go through a “small” computer vision project.

“SIMPSON! Get in here!”

“Yes, boss?”

“Simpson, I read about this new thing called computer vision, and we need to use it!”

“Computer vision, sir?”

“YES! We have a storage room for pallets. I want a dashboard that shows the number of pallets that are in the storage room. Use a camera and this computer vision thing to give me my dashboard. I NEED my dashboard as soon as possible..”

“Right away, sir! I will go and talk to the team.”

Simpson would then go off and find a data scientist to talk to about this computer vision project.

“Boss wants a dashboard that uses computer vision to count the pallets in the storage room.”

“OK, what else is in the storage room?”

“Does that matter?”

“Yes, it matters”

“Hmmm….lots of things. Boxes, shelves to hold the pallets, a couple of those anniversary signs that we don’t know what to do with…oh and the pallet jack.”

“Ok, we must train a custom computer vision model to do this.”

“Why?”

“Because I first have to break down the scene into parts that have any pallets and those that don’t. Once I do that, I send the parts that do through a different model that tries to recognize smaller parts that represent one pallet regardless of skew and orientation. Then I can count those parts or potentially something else entirely.”

“Hmm, sounds complicated…”

“It is.”

“Ok, how long will it take?”

“I don’t know".”

“What do you mean you don’t know?”

“Look, I am going to have to source training images that I can use to train the model. Once I have that I will figure out how to use those training images to train the model to find clumps of pallets. I will then train the model over and over until I can get the model to a high enough degree of accuracy”

“Ok! How long does that take?”

“I don’t know…”

The data scientist in this example isn’t being smug or difficult. It is the nature of machine learning. To the trained eye, there are multiple tasks with fully unknown timelines. One of the less obvious ones is “source training images”. Training a model falls squarely under the old adage “garbage in, garbage out”. Sometimes, you won’t even know the training data is bad until you try to train with it. Training data isn’t JUST data. The “training” part is the catch. In order to use training data, you have to know what result you want for every input. So, in the above example, you will want to know how many pallets are in each image so you can decide if the model is getting better or worse.

This brings us to the training portion of the project. Training models are not an exact science. In some cases, it feels more like an art. You might be running along, watching your training charts. The accuracy chart is moving up and to the right. 35%…..45%…..47%….60%….61%….61%……61%….GAH!

The first analysis is deciding if 61% accuracy is good enough. If it is not, it is time to go back and try to figure out whether you need more or different training data, a different base model, or a modification to how you are training. The process is to use educated intuition to create another training run and try again, which might yield better or worse results.

To be clear, computer vision, machine learning, and GenAI are incredibly powerful. I have repeatedly seen my reports and colleagues do absolutely astonishing things. The key as a leader is understanding that you will need a different kind of patience than you are used to. Another key skill is understanding the lack of predictability and factoring that into your “juice is worth the squeeze” analysis.

Okay, Mark, but what about GenAI… That is what I really want to know! GenAI is up next…and the parallels to our pallet counting project are clear.

BONUS LEADERSHIP TIP (BLT, I LOVE a good BLT!): Your reports will really WANT to give you a date on when a project like the pallet counter will be done. You can ask when it will be done, but the only answers you should consider 100% accurate are “Today!” and “Not today!”. If you have someone really push that they are sure of some future date…they are most likely bleary-eyed, and their brain is hallucinating a date so that the madness stops.