The Death of Data Silos: How AI Primitives Are Uniting Every Business Process
A little horror story for you all about EDI, human integrators, and iPaaS
Better and better software abstractions for the last 30 years have democratized the access to business software. That’s great news. But with great power, comes great responsibility, and software has failed mission-critical supply chains in one key aspect: isolating disparate business processes that are essential for companies to communicate with each other.
What does that mean in practical terms? That each company in every vertical on Earth is running on software (more or less sophisticated), but none of these systems talk to each other, so anything that is movement of goods or funds between them, is left to either middlemen or email attachments. Having built these systems before, I can tell you that even integration middlemen fail at capturing the most chaotic data — sometimes the most essential.
A sad story
Let me paint you a picture. When I was building Silo, a couple of years ago I flew to Miami to onboard a large importer of fruits and vegetables from Central America. Chances are, if you’ve eaten a plantain chip — and if you haven’t, you’re missing out — in the last 5 years, that chip has been through that warehouse. My goal for the trip was clear: onboarding them to the platform, that is, learning everything about their business and processes and designing how to adapt Silo to their needs (or the other way around 😬).
As a member of their team tours me around their warehouse and offices, I get a sense that they’ve been trying to work around a combination of Quickbooks and Google Sheets to run a business producing $100M in revenue per year. That’s not the surprising part. We had seen worse at that point, trust me.
Out of the corner of my eye, I spot one of their office workers in a corner cubicle with 2 monitors, side by side, each with its keyboard and mouse. The person doing the tour just glossed over her quickly, so I just asked to get closer. I wanted to get a full sense of their business. I start a conversation with their team member and I see that every time a bell rings in one computer, she opens a UI in the other and starts transcribing everything manually from one computer to the other.
The Human API
“Sorry, if you don’t mind, I’d love to know what you’re doing now”.
She looks at me, bored out of her mind, and starts explaining it in detail. Every time Walmart and Costco (each running in a separate application on one of the Windows computers) send an order via EDI, she opens Quickbooks on the other computer and transcribes manually every item, every order. 8 hours a day. Every day of the week.
Behold, the Human Integrator. Now, you may think this is an isolated case, and you may think this is related to a lack of technology access/knowledge. Well that’s stupid, right? Why isn’t this company using Orderful or Stedi to receive their orders? And why would they receive EDI through 2 different softwares, each with their expensive license (around six figures yearly, in total).
Because simply put, deterministic integrators haven’t figured out this problem either. There’s only so much you can do when translating EDI (or anything else) into JSON, considering a few aspects: that every major retailer has a different flavor/variation of EDI, and that frequently, the “translation” of data requires context and analysis, something that a mapping JSON algorithm won’t do.
How do we solve this? By *not* replacing humans
With AI. With caveats and nuance, something the industry has been lacking lately.
Because here’s what a lot of AI companies are getting wrong: AI is a new layer of abstraction to achieve efficiency gains. That’s it. Every 10-20 years, we’ve gotten a new generation of abstractions, all the way from the Babbage machine to the Prisma ORM. AI is not for doing something humans can’t. AI won’t effectively do something a human doesn’t know what to do, at least not at enterprise scale in a real-life scenario.
AI can do something faster, more efficient, and safer than a human, provided it’s built in a way the human can safely interact with it, be evaluated, monitored, tested, and ran as an abstraction. As many abstractions have been before.
Through this enablement between humans and AI, AI can learn, acquire context, and function as a safe state machine for business process efficiency.
That’s our bet and that’s our philosophy. AI as business infrastructure.
As far as our previously mentioned human API, my personal goal is to enable her to do better things faster, not replace her, and until I’ve done that, I won’t rest.