While the official confirmations have yet to land (and my colleague Dan Primack is following up on Groupon CEO Andrew Mason’s hopefully tongue-in-cheek offer to discuss the finer points of his affection for miniature dollhouses), it’s looking like talks between Google and Groupon have fallen apart. The situation is eerily reminiscent of Google’s (GOOG) talks with Yelp, just about a year ago. Google, thought to be after Yelp’s ground army of local sales teams, now stretching across much of the U.S. and Canada and some European countries, either got cold feet, or got a cold shoulder from CEO Jeremy Stoppelman, and returned him the favor.

Maybe it’s something about the holiday season that leaves Eric Schmidt wishing the company had a killer local business offering up its sleeve, but whatever the case, that marks two failed acquisitions of social or local-oriented startups. (His successful 2005 acquisition of Dennis Crowley’s Foursquare precursor, Dodgeball, came at a different time in tech, and anyway, Google ended up shutting it down and alienating Crowley.)

It’s worth noting that Yelp was, according to reports, in talks to be bought for three quarters of a billion dollars. The bidding for Groupon, a site by all accounts offering far less sophisticated technology, was said to have reached $6 billion, including various retention incentives for Mason and other top executives. Social and local are not getting any less valuable.

Groupon is almost certainly set to clear far more cash in a year than Yelp seems to be, thanks to the fact that Groupon hit upon a real revenue stream and incentive for both users and businesses to pay for its services, it is now being said to have a projected $2 billion revenue run rate for 2011. But Yelp’s database and user community would seem to have even  greater implicit value to Google, even if that value is currently locked up by the site’s emphasis on reviews rather than transactions. (That’s fixable, by the way: Isn’t Yelp the perfect place for group coupons?)

While the courting continues — or doesn’t — it’s hard not to wonder how Google got itself into this position. Here’s the dominant company of the Internet age, the ultimate traffic generator that should cause any of the companies it is talking to to see massive growth and to make the founders wildly rich. But, it seems, the issue now is not what Google’s willing to spend, it’s about what kind of company it has become — and whether this  founders believe this is the caretaker that will help their babies grow.

While Google Ads revolutionized the way companies of all sizes connect to customers, when you think about the really big companies that rely on Google for traffic, where is the profit? YouTube, which Google acquired in 2006, will finally be profitable on $700 million in revenue this year. (Maybe.) Media organizations, most prominently the New York Times, that rely on Google’s search results for their traffic are in the same boat as YouTube: They have experienced huge search-based traffic growth, but poor profitability, especially, as Felix Salmon notes in his dissection of Gawker’s new business model, with CPM rates looking like a “race to the bottom.”

Now think about the companies that have emerged on the back of Facebook‘s social discovery platform: Zynga and Groupon, to name the two biggest, are wildly profitable. While those companies were set up to monetize traffic in a way that YouTube and the New York Times (NYSE:NYT) clearly were not, they benefitted from the power of Facebook’s social graph and, as Fortune‘s Kevin Kelleher recently wrote, its evolution of search advertising into targeted and highly effective social advertising. [See: “How Facebook fixed the social advertising problem.”]

So why would Groupon choose to align with Google’s search-oriented ethos instead of the social one that helped it become a power?

A war is brewing between the open and closed webs, and the poles around which companies are aligning seem to be Google on one side, and Facebook on the other. What few analysts predicted, especially during Facebook’s near-disastrous privacy spats over the last year, is how comfortable the web-surfing public would be in the closed web, as long as that web provides everything the average person needs or wants from it. Increasingly, Facebook and its partners are doing just that.

And besides Facebook’s guarded principles of data access which Google is happy to complain about, the trick Facebook has pulled is that it doesn’t feel like a walled-off world on the web. It just feels like the web: a place where users go to see what their friends are up to, take note of any games they’re playing, Groupons they’re buying, restaurants they’re reviewing on Yelp or checking into on Foursquare. Users are dipping their toes in and out of the stream, and they understand and are tolerant of the fact that the stream is generated only because they’ve shared so much data with Facebook to begin with. It feels like a huge added value.

Google is increasingly feeling like less of a value add. In its quest to be open, it’s stopped feeling like a smart filter that brings the most relevant parts of the web to users of its search, and more like the actual wild, woolly, untameable raw web itself. Most twentysomethings and all teenagers today don’t remember the days of trying to bend HotBot, Yahoo (NYT) or AltaVista to your whims, and what a massive improvement Google was on that. Instead, Google is their AltaVista, their baseline Internet from which things can (and must) only get better.

It’s nothing more than the relentless pace of technology that’s made that the case; Google hasn’t done much wrong in its decade long run at the top of the tech heap. And their business isn’t going away; it has added far too much value to the ecosystem of the Internet for that to happen. But until it proves to the current generation of Andrew Masons that it understand social and the closed web — not just as a nice bag on the side of their search engine, but as core to the future of the Internet — it’s going to have a hard time getting superstar startups to take their money, at any price.

Though only 30, Mason can surely still remember the bad old days of useless search engines and therefore witnessed how quickly dominant players can be left behind. Why — given his company’s profitability and his personal wealth — would he subject Groupon to the company that killed Dodgeball, bungled Yelp, and screwed up almost all of its own social efforts, from Buzz to Wave?

Schmidt has been hinting to the press that he knows Google is facing its first serious re-orienting as a company, and like all transformations, it’s probably very painful for insiders to consider. Core search will have to be marginalized to make social and local the heart of the operation. (Its mobile and OS efforts essentially amount to parallel operations in the company; neither one is likely to become Google’s new core, but more to layer on what that core is.) The entire tech world, from Mark Zuckerberg to Andrew Mason, seems to be watching and waiting to see if Google can figure it out.