The setting is decidedly modern and functional. The salesman proposes a system to solve all your problems. The photographs in the catalog are artfully arranged, accompanied by adjective-laden descriptions about ease of use and simplicity. That GROLVIK looks beautiful next to the HETSQ. And the FYIPS and optional KLONF in the background tie it all together. All you need to add is a PEBMU and OCRD to connect it to your ERPSTER and CRMPO.
Next, you are told to just go to the warehouse and pull the relevant parts and you’ll be able to assemble it in no time.
The reality: Many days/months/years of frustration later, you give up on the assembly and call for help. Maybe an expert can decipher those unlabeled diagrams in the instruction manual. If you had only known how much trouble and cost was involved in assembly, you would’ve just purchased a finished product.
Another typical experience at the blue and yellow furniture store featuring lingonberries in the lobby? No, we’re talking about artificial intelligence (AI) tools. Every week there is an announcement about a new AI tool, which you are welcome to purchase, install, interconnect with your other systems, train, and validate. It’s a lot like buying ready-to-assemble furniture: The pictures often do not match the results.
Some assembly required
Unlike furniture, training AI tools yourself is a task that requires far more than a long weekend. There are only so many ways you can put together a bookshelf when you have only three different types of screws and 15 holes.
Compare that to legal contracts and the myriad ways that those can be written. Any given business term could be described in countless ways, each of which requires training work for your system.
Imagine that you’re trying to train the system to find the effective date of an agreement. Do all of your agreements use the magic words “effective date”? Not likely. Instead, you need to train the system to also look for the initiation date, the start date, the beginning date, the kickoff date, the launch date, the rollout date, and many more terms.
And this is just the beginning of the complexity. What happens when you start delving into other legal concepts, such as expiration dates, and you need to train the system to look for the third anniversary of the second month after the Maturity Date? You will wish you were working on the bookshelf assembly instead…
Simply put, contracts and other legal documents are much more complex to decipher than bookshelf and kitchen cabinet assembly instructions. And the people who can engage in the deciphering of contracts and training of AI are much more expensive because they need a specialized skill: deep experience in contract law.
It’s unlikely that you can pick a name out of TaskRabbit or Craigslist to train your AI tools to find mutual indemnifications and Most Favored Nation clauses. The type of human “assembly” experts who can help you train AI tools are scarce and costly.
What does this mean for your budget when purchasing AI tools that you need to train yourself? Don’t be fooled by the speed of the YouTube videos or the beautiful demo in the showroom.
Due to the high cost of attorneys and the great heterogeneity in the way that legal documents can be worded, expect to pay at least 300 percent more on training than you spend on the AI tools themselves. This can lead to a lot of friction with your CFO later because you promised great productivity gains from buying the AI tools, but end up instead needing to ask for a headcount increase to train and quality check the AI tool’s output.
Typically finance teams evaluate investments on a total cost of ownership (TCO) basis. And on a TCO basis, a train-it-yourself AI system is often not the winner.
The missing parts problem
In the more sophisticated ready to assemble furniture kits, instruction start with a pictorial inventory of all the parts and tools that you will need. But how can you tell the 3/4 inch screw apart from the 5/8 inch screw? And is the black-and-white picture of the dowel intended to show the plastic or wood piece?
For vendors selling AI tools that you are supposed to configure and run yourself, similar problems arise. You might have three different versions of a particular subprogram, and only later find out that none of them are compatible with the new operating system on your computer. Or perhaps there is a missing plug-in you’re supposed to download in order to make the system work with your current browser.
If we call the furniture store about missing screws, it’s likely that they will have replacements on hand or at least be able to tell you the dimensions of what you need to pick up your local hardware store to complete the project. Unfortunately, when you discover incompatible components of your AI tools, the matter is rarely remedied so easily.
You may have long delays in your project while the vendor attempts to develop, patch, test, and deploy new code to fill the gaps between the tool’s promised functionality compared to what you actually received. And, unlike missing screws that might only prevent you from attaching a particular shelf, it is typically the case that missing a piece of software can render your entire system inoperable.
Ready-to-assemble furniture is often self-contained. By contrast, AI tools are typically interdependent with your hardware, operating system, and other IT infrastructure. Thus, while you hector the vendor to fix compatibility issues in their AI tools, there is a significant risk that the vendor places the blame on your operating system, browser, or third-party software — which means energy is spent pointing fingers rather than solving problems.
While business schools often teach the phrase “The Customer is King,” software tools vendors often have a very different view of that topic. Go look at your recent software license agreement that you signed and flip to the warranty section.
Typically, what you will see from software tools vendors is a disclaimer about every aspect of their product, even its most basic functionality. This is why it’s essential to write your contract with software and SaaS vendors to specify the result that you’re buying, as opposed to a listing of components that might — unbeknownst to you — be missing critical pieces.
But it looked so charming in the picture!
At last, you tracked down and purchased all the relevant furniture components, including the accessory shelves, bases, legs, drawers, and casters. After chasing the vendor for several more weeks about missing parts and pieces, you have the parts to complete assembly.
Next, you bought a few more tools to correct for predrilled holes in the wrong places. After watching 23 self-help videos in slow motion and debating assembly approaches on Quora and Reddit, you finally cobble together your complete furniture set.
It’s finally time for the big unveiling! You turn up the lights in the room and discover that, while the beautiful combination desk-bed-dresser-chair-bookshelf-treadmill system indeed does resemble the photo in the catalog, it is so small that it might only accommodate your toddler.
Unfortunately, similar surprises occur when buying train-it-yourself AI tools. After you train the tool on five hundred similar invoices, it might be able to parse out certain text (e.g., dates and amounts) for you automatically.
However, you may be in for a big surprise when you try to load a different format of invoice or contract into the system. Suddenly, you’re getting random answers again and feel like you’re back at square one.
This is because off-the-shelf AI tools are often designed for a small scope — just like the furniture that looks great in the picture but has limited functionality. How could you spot this problem in advance?
Make sure to demo AI tools using a wide range of content and output to see how they perform. And be careful to not overgeneralize from a tool’s ability to solve only a small or limited problem.
Escape the showroom labyrinth
There’s always something new in the showroom to lure you into the ready-to-assemble or ready-to-train store. Nary a month goes by without some press release about the latest furniture kits or used AI tools.
In the AI tools space, they might mention TensorFlow or AutoML or some super-duper analytics plugin for your eDiscovery system. Each vendor touts that “This time it’s different!”
Supposedly the latest touted AI tool will train itself, and mow your lawn at the same time. Better yet, perhaps it is already “pre-trained” and all you need to do is run your own quality control on it.
You need to remember what happens when you leave that showroom and it’s time for your own assembly and training work. Dig into the total cost of ownership (TCO) of AI tools — including training and validation — in order to identify the real financial and time commitment required.
Will the person selling the AI tools make a promise that the tools will provide complete and accurate responses out of the box? If not, will the vendor agree to cap the number of hours you need to spend training the system? If not, take a lingonberry break and then shop for a full-service solution that offers a better TCO.
