And then a funny thing happened. Some of those vendors realized they didn't really need the agencies.

Many venture-backed ad-tech companies that originated as agency vendors are increasingly going directly to marketers and offering them cheaper ways to implement their technology. A combination of factors -- ad-tech companies wanting their own client rosters and marketers wanting to control their own data and lower costs, among them -- has spurred the movement. And several big marketers have bought in. Procter & Gamble and Kellogg, for example, have set up trading operations, in part to exert more control over their own data.

"Everyone wants to cut out the middle man and see if they can lock down dollars for themselves," said Christine Peterson, U.S. group media director for Publicis Groupe agency MRY.

The heightened competition between ad-tech firms and agencies is generating real market changes. It's pressuring individual agencies to reposition their strengths and reconsider the costs of their ad-tech offerings. And it's pushing ad-tech firms to build up agency-like capabilities to handle accounts.

Inherent biases
The trend compelled Quentin George and Brendan Moorcroft to leave the holding-company media world, where they were integral in the launch of Interpublic's audience-buying platform Cadreon, and co-found Unbound, a consultancy for brands and publishers that want their own tech stacks.

"An agency might own a [relationship] with a DSP and get a better deal, but by and large I think the tech ought to sit with both publishers and brand marketers," said Mr. George, referring to demand-side platforms, the technology driving automated buying.

"Agencies and ad-tech companies ... are inherently biased," said the former chief innovation officer at Interpublic's Mediabrands. Agencies are incentivized to persuade clients to put money through their trading desks, while ad-tech companies are incentivized to convince advertisers that their proposition is as simple as, "Give us a check and we'll give you inventory."

Unbound is working with three brands that want customized systems to automate the media-buying and -selling process. One brand has been outsourcing programmatic spending to ad networks and ad-tech companies, such as Rocket Fuel, but wants to build its own system; it doesn't have an agency and doesn't want one. Of the other two, both of which work with agency trading desks, one wants to "liberate" more of its organization's data but doesn't feel comfortable giving that data to its agency's trading desk, and the other brand is unhappy with the amount of disclosure it gets regarding how its agency's trading desk makes money.

"What we thought was valuable and proprietary [in the agency] -- the algorithms and bidder -- is the most commoditized part [of the business]," said Mr. George. "The application of data and understanding audiences is where real value is."

Aggressive tactics
Ad-tech companies go around agency partners to get in tight with the clients "all the time" in an attempt to solidify and expand their businesses, said Jack Southerland, senior VP-director of client services at agency trading desk Varick Media Management. He wouldn't name names, but said it tends to be those that "have aggressive business goals, are VC-backed, trying to go public."

Over the last few years, investor-funded ad-tech companies have swung from a feast of merger-and-acquisition possibilities to famine and possibly back again. The ad-tech M&A landscape exploded in 2011, with 60 deals valued at a total of $4.5 billion, according to investment bank Coady Diemar Partners, and shrunk to 41 deals totaling $396 million the following year. But through the first half of 2013, the bank tabulated 34 ad-tech deals raking in a combined $1.25 billion. Between December 2010 and December 2012, 74 new display-ad-technology companies have popped up to replace the 53 that were acquired, according to investment bank Luma Partners.

This new wave of ad-tech companies is "not interested in profits, just in gaining ground," said Drew Kurth, CEO of digital-marketing platforms at Publicis Groupe agency Razorfish. "We see them coming in and saying, "We'll do this for free and it'll be low-cost and low-impact on your side.' They might do something for free to get a big brand name on a client list to merchandise."

Clients can cheaply implement and oversee many ad-tech functions themselves, endangering the fees they paid to full-time agency employees for overseeing those functions. Razorfish has cut fees it charges clients for ad-tech services such as dashboard software and cloud-based hosting services 10% to 20%, said Mr. Kurth.

Agencies like Razorfish say they are making up that lost margin with more modern data services that require fewer but more expensive full-time employees and global data services through partnerships with companies like Adobe. The pitch goes something like this: While it might be cheaper for clients to deal directly with ad-tech vendors, their savings will be eaten up by the time and resources needed to cobble together a global ad-tech ecosystem on their own.

Agencies are positioning themselves as having the experts who know how to integrate data and technologies and can build the platforms that do so.

Scott Hagedorn, CEO of Annalect, the data arm of Omnicom's media-agency network, said he's most focused on building a system that can integrate different data sets from separate tech companies, such Facebook and Google.

Without the right technology, "you can't actively make them work together," he said.

Long term vs. short term
"There is no doubt that DSPs are very aggressively lowering their pricing," said Brian Lesser, CEO of Xaxis, the central data and media-trading operation for WPP's GroupM. "We are reevaluating our pricing constantly as the market evolves."

One "large" advertising client went around Xaxis to work directly with a DSP in an attempt to create "efficiencies," said Mr. Lesser. But after achieving a short-term boost, the client realized that it couldn't milk much more out of the arrangement with the DSP. It returned to Xaxis for audience buying, he said, but kept the DSP relationship intact for additional support.

"If you have a performance-oriented product ... you will see a short-term lift if your goal is cost-per-acquisition improvement," he said. "But going straight to a DSP, you won't see an increase in the amount of insight you're getting from a campaign."

Like Razorfish, SapientNitro has lowered the price of certain ad-tech services as they've been commoditized, shifting fees to higher-end services, such as helping clients understand the impact of their spend and analyzing their customer segments, said Serge Del Grosso, North American media director. "It's about making sure we're competitively priced on the basic tools and positioning the attribution component of it as a true value-add and business driver. One is a machine and one is a strategic overlay."

Mutually beneficial
Sometimes deals benefit both ad-tech vendors and agencies. Agencies often lay out rules of engagement for ad-tech partners and map out how the two sides can team up to win new clients. One former agency trading-desk executive said his firm and others would offer quid pro quos to DSPs. If a DSP wanted to be the trading desk's primary automated ad-buying tool or wanted the agency to guarantee to funnel $1 million a month in ad spending through its system, the trading desks would ask the DSP for introductions to new clients or to give the agency first-look at any inbound client inquiries.

Publicis Groupe's ViVaKi explores new-business opportunities with its ad-tech partners but doesn't formalize such deals, according to ViVaKi president Kurt Unkel. "To be really transparent, it's not structured or anything like that," he said, describing it as a way to bring more clients into the fold to benefit both sides.

Blame game
For all their handwringing, agencies are partly to blame for the competitive environment. When automated ad-buying companies started popping up and promoting their services to agencies around the middle of the last decade, holding companies typically picked one or two to effectively serve as a house tool. "The focus was on becoming the one [DSP] to rule them all," said Steve Ustaris, senior VP-marketing at ad-targeting company OwnerIQ.

That didn't happen. Agencies, under pressure to reach ever-larger audiences for clients, began working with multiple DSPs. Those DSPs, faced with a shrinking share of advertisers' online ad budgets, angled for ways to get closer to the advertisers in order to avoid being cut out.

A tipping point came a few years ago when travel-booking site Kayak opted to bypass media agencies entirely and work exclusively with an ad-tech company, MediaMath, for its online-media buying. "That was one of the first times that happened, and it changed how everyone viewed the agency-DSP relationship," Mr. Ustaris said.

Then last year MediaMath spun out an agency, Kepler Group, from its core automated ad-buying business.

Creating customers
MediaMath has "gone out and tried to sell every agency they can at this point," said a former agency trading-desk executive. "They need more agencies out there that are smart about [automated ad buying] and can't find them." By spinning off Kepler, "it's almost like creating customers for themselves."

MediaMath rationalized the move as a way to ease any conflict concerns over its role as a technology provider and the agency-like services it had been providing some clients, like Kayak. "Kepler is a trading specialist that supports Kayak and several other clients. We view them in the same way we see our relationships with Merkle, Epsilon, Hill Holiday and Razorfish: They are partners using our technology platform to drive efficiency and greater value for their clients," MediaMath CEO Joe Zawadzki said in an email, adding that his company licenses its technology to more than 350 agencies and 700 brands.

Back to the beginning
Ad-tech execs say those agency-like arms are necessary until existing agencies get up to speed on this computer-based method of buying media."When something's new in the industry, traditional companies like agencies don't have that services layer in place; they don't have all that expertise. So you get companies that wind up providing both technology and services to them," said Philip Smolin, senior VP-market solutions at automated ad buyer Turn, MediaMath's foremost independent rival.

To an extent, we may see a return to the early days when an agency worked with a more select group of ad-tech vendors -- but maybe not of their choosing. As marketers have become more technically literate about the space, they are more comfortable making their own decisions about which ad-tech companies power their online media-buying operation and are tapping their agencies to oversee it.

Unilever and Kimberly-Clark Corp. in the past year have moved to create their own standalone data-management platforms and trading desks, working with WPP's Mindshare but staying separate from WPP's centralized Xaxis trading desk.

"Ad networks have gone to clients saying, "We can do all your buying' ... [but] realize they're not set up to account manage a client," said Tom Potts, head of media at independent digital shop Profero. That's where the agency comes in, serving as the layer between tech provider and marketer.

"If you want to just add more and more partners to the table as a client, at some point someone starts saying, "There's a lot of complexity here; who's in charge?'" said ViVaKi's Mr. Unkel.