Big Data Boom from 5G, IoT Has WANdisco Dancing

(Fit-Ztudio/Shutterstock)

When interest in Hadoop was building years ago, WANDisco prepared itself for an expected surge of business with its big data movement technology. However, that business never fully materialized, and when the Hadoop bubble finally burst, the company was left searching for a new market. Now it’s found one in the big data boom occurring in the cloud thanks to industrial IoT data collected via 5G networks.

Like many in the industry, WANdisco CEO David Richards has scars from the Hadoop years. But unlike some, he managed to successfully navigate his company through the Hadoop storm and into more agreeable waters.

“It did us a favor,” Richards says of the whole Hadoop experience. “We built a product to solve massive scale, and we looked like a bunch of idiots because nobody had that massive scale.”

As migrations into Hadoop tailed off, WANDisco needed a new market for its core technology, which helps users move massive amounts of data over the network in a timely manner. The company’s claim to fame is the capability to enable analysts or data scientists to begin working with data the minute that data begins arriving at its destination, instead of waiting days, weeks, or months for the entire data set to be moved.

“We don’t have to scan through the data, so you can start running machine learning within one second of turning the product on,” Richards tells Datanami. “We use consensus to ensure that the data is accurate…but we also are able to multi-plex it across whatever bandwidth is available and saturate the whole thing.”

While that Hadoop thing fizzled, cloud data platforms have risen in its place, and the data flows into object storage in the cloud are truly massive. WANdisco recently was used to move 25PB of data for large telecom company into the cloud, where it can be analyzed using one of the popular cloud data warehouses. It’s involved in another deal to move 1 exabyte (EB) of data per year into the cloud for a large automotive company.

“I guess we’re a 10-year overnight success, I keep saying to our investors,” Richards quips. “The technology thesis has never changed, which is that moving petabyte scale data is pretty tough over the Internet and making sure it arrives in the right order and timely is a problem. It turns out we’re the only ones who can actually solve this.”

Business is booming for WANdisco, but it’s coming from an unexpected place: 5G networks.

5G technology was originally marketed largely as a consumer technology. The wireless technology would enable us to download full-length feature films in a matter of seconds, and it would open up a new era of interactive games. 5G would become the new mesh enabling us to engage in augmented reality and bask in the metaverse.

Turns out, that’s not what 5G is being used for as all. “The use case is not videos or your phone,” Richards says. “[Telecoms] are pivoting to be IoT backbone companies with complete solutions for sensor-edge-cloud.”

Data collected from automobiles is particularly big and somewhat timely. Manufacturers are gathering reliability and performance data from components on electric cars, while insurance companies are interested in the driving data to help price premiums.

“Monetization happens, of course, when it goes to cloud. It has to go to cloud because that’s where CPU farms are to run machine learning,” Richards says. “We’ve done a whole raft of deals with primarily telecos, but I think we’ve got most of the automotive industry and I think we’ll get most of the top IoT backbone teleco companies as well. The use case is edge to cloud. It’s a pure infrastructure play, with data sets of multi-petabytes.”

You don’t have to take Richards word for it, as WANdisco is now a public company, traded in UK. In 2022, revenues skyrocketed 967%, from about $12 million to $127 million. The company reports deferred revenue of about $110 million, up from less than $10 million last year.

David Richards, CEO and co-founder of WANDisco

Richards, who calls the numbers “ridiculous,” is flabbergasted at the amount of business WANDisco is suddenly generating.

“It is bonkers what is going on right now with our company. I’ve never seen anything like it,” he says. “It was always sort of, this was what we said was going to happen. When it actually happens, you have to have the numbers to back it up.”

Richards is bullish that the big data spigot is only going to open up wider. As companies realize that the value of their data is vastly greater than the value of their legacy business, it will seed demand for even bigger data movements to fuel even bigger machine learning workloads.

“You can see the Gartner Hype Cycle, the Trough of Disillusionment and all that,” he says. “But to see it really taking place is really cool. What’s going to happen in automotive and communication and rail and airlines and manfuacuring–this is going to be the fundamental source of competitive advantage across all those industries.”

Today, WANdisco, which is publicly listed in the UK but has American headquarters in San Ramon, California, unveiled Data Activation Platform 2.0, which the company claims will lead to faster movement of bigger datasets into the cloud. The new version brings the better support for the simultaneous use of multiple data transfer agents; automated cleanup of data source environments; enhanced verification of data fidelity; more controls over the prioritization of data transfers; and user interface enhancements.

After suffering through the Hadoop years, which were filled with more hype than bookable revenue, Richards sounds a bit incredulous that he now finds himself in a bonafide boom at the edge of a 5G network, with a dearth of competitors.

“IoT is actually, really happening,” he says. “I thought I would never see this in my business lifetime. I genuinely never thought there would be use cases that require that amount of data. It’s happening.”

How to Address Scale and Performance Needs in IoT

WANdisco Opens Up the Dance Floor Beyond Just Hadoop