Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Facebook Overestimated Key Video Metric for Two Years (wsj.com)
283 points by tshtf on Sept 22, 2016 | hide | past | favorite | 150 comments


>the tech giant vastly overestimated average viewing time for video ads on its platform for two years, according to people familiar with the situation

They didn't "overestimate". They plainly gave false numbers to drive ad sales.

Coldtea's law: Never attribute to incompetence what can be explained by profit.


"It is difficult to get a man to understand something, when his salary depends upon his not understanding it!"

-Upton Sinclair


That is basically how Wall Street works. If a trader or salesperson is printing money, management doesn't want to know how it's done. Plausible deniability will at worst get you a failure to supervise penalty, whereas if you understand how these folks are making money you could become complicit in any illicit acts and thus exposure yourself to a much larger penalty and/or jail time.

Of course, the government has proven themselves time and time again unwilling to punish management so management might as well figure out how the illicit gains are being made so they can replicate such processes when the big producing trader or salesperson is invariably poached by a competing shop.


I'd qualify that.

Things that appear to be generating a ton of money don't get looked at that closely, because everything's going great. Things creating losses get looked at much faster.

So it's not pure deceptiveness. Probably nobody made the call "let's defraud everyone". However, a bug that undercounted the viewing time would have been uncovered much sooner.


Seriously, is anyone surprised by this? This was obviously the #1 reason why autoplaying videos were introduced in the first place. Amazing that it took two years for the advertisers to figure it out and start getting angry.


I'd call it SV's Law, but you can get the merits of it :)


Same goes for Facebook's user tracking "bugs".


+ there's a reason videos auto-play....


There's another dumb law that says the opposite.

Has something to do with Gillette.


Actually it doesn't say the opposite at all.

Except if you think that incompetent Facebook-level engineers are more possible than greedy Facebook-level execs.

I think it's very clear that "stupid Facebook engineers couldn't manage to see such a basic metric in two years" fails the Occam razor test much more than "Facebook as a business likes ad money".


"How Facebook is Stealing Billions of Views" - Kurzgesagt – In a Nutshell exposed this with detailed analysis and evidence. Watch it here => https://www.youtube.com/watch?v=t7tA3NNKF0Q

This is neither a bug nor a "mistake". It's straight up fraud. Fraud because Facebook uses these false #s to prove to their advertizing clients that they are getting a HUGE ROI, when they really aren't. Also the investors and analysts bump up their ratings and the stock goes higher and higher on false ad view #s.

By counting a view as a legitmate view after the video plays only for 3 seconds, their "algorithm" counted billions of views even when the user has not even seen it, because Auto-play is enabled by default and you have to opt-out / disable it and no-one does it. It takes a person roughly 4 seconds to scroll off their feed as they quickly "scan" their friends posts.


Your video is about how FB is viewed as lying to content creators because it counts a view as having the video play for 3 seconds, which content creators believe is too small a threshold.

The article here is about how the average view length is only calculated from exactly those impressions which lasted more than 3 seconds, where the outrage is that it should have been computed for all impressions. In other words, that the threshold was too high.


ROI from FB ads can only be calculated from tracking conversions on the advertiser's site or in their app. View count alone can never be attributed a ROI as there is no measurable return.

If you pay solely for video views on Facebook or Youtube without any tracking, you're not being a smart advertiser and it doesn't really matter if view counts are more or less correct - you're blindly throwing your money away.

If you do however track your ROI correctly it doesn't really matter how many true view counts you get.


The main problem here is attribution - and with many ad platforms pushing the 'viewthrough conversion' metric, it becomes very messy when hundreds of millions of users start getting tagged as having 'viewed' a video too easily. Future purchases can then be tied back to that 'view', even if it was insignificant in the buying process.

This is where as an advertiser I would be very interested in the distribution of view lengths, to better understand the user's engagement and likelihood that the video made any impact on their behaviours at all.

Although being totally honest, you can't rule out the impact of even a passive 3 second peripheral glimpse of brand video content. Most traditional advertising - TV and outdoor, for example - is not actively viewed, but rather passively 'seen'.

Ultimately it comes back to being pragmatic in the way you assess advertising effectiveness - the promise of digital is perfect ROI analysis and measurement, but the reality is far trickier.


Do you have a citation for the "4 seconds" figure? If it is 4 seconds, and it logs a view after 3 seconds, that's pretty damning.


It's not publicly discussed for obvious reasons, but I worked closely with Mudd for a couple of months last year. Here's a related interview where he touches upon it.

http://digiday.com/platforms/advertisingweek2015-facebook-fi...

Facebook spend an awful lot of resources tracking all sorts of user behaviour inside it's "walled garden".

I got really sick of it after a while and decided to stop working for them (even as a contractor)


That it logs invisible videos at all would be a problem.


On my programmer hand, I can see myself making a similar technical choice. We have auto-playing videos, we shouldn't count the views less than 3 seconds because that wasn't really an intentional view. But that means we should also exclude them from the overall view count. I don't know if FB did that as well.

On my shareholder hand, this seems slightly like fraud.

On my advertiser foot, this seems slightly like fraud.


See just get rid of the auto-playing videos problem solved, and make the world a better place while you're at it.


I bet Facebook gets a decent enough amount from people who scroll, look away for a second, video starts playing, then user pauses video, but not before the 3 second mark.

What really should be done is getting rid of auto playing videos at the HTML spec level. While there are legitimate uses for autoplaying videos (YouTube for example), it has too much potential for abuse by ad networks.


Advertisers will find a workaround and it will inevitably be much worse for the user. In the following talk, someone from the Google Chrome team describes how, when they tried to not autoplay videos on Android, video advertisers worked around the restriction by compiling video codecs to JavaScript and then rendering their video-like ads to an HTML canvas.

https://youtu.be/wQGa_6CRc9I?t=23m52s


That's where NoScript comes in


Or blocking 3rd party scrips with uBlock.


Don't ad networks charge for ad size as well? (or they should start charging for this - would cut on abuse)

If your ad is > 50k then you're paying a fee per display, even if you're "pay per click"


> What really should be done is getting rid of auto playing videos at the HTML spec level.

Or, let autoplay be a an explicit flag in the spec that is advisory as to publisher's intent, and establish a norm of user agents providing at least "big switch" configurability (Autoplay video on/off) and, ideally, optional per-site override of default on/off configuration for autoplay.


Wait... get rid of auto playing videos at the spec level? I'd rather have the autoplay flag made explicit, rather than some hacked up soup of code that makes sure the video autoplays.

When intent is captured (e.g. specific "autoplay" flag) my browser client can interpret the flags according to my choosing. Capturing intent > not capturing intent.


You mean an attribute on the video tag? That is a thing. It looks like this:

    <video autoplay>...</video>


> hat really should be done is getting rid of auto playing videos at the HTML spec level. While there are legitimate uses for autoplaying videos (YouTube for example), it has too much potential for abuse by ad networks.

Most of Facebook's video traffic is in their mobile app, getting rid of it in the HTML spec level wouldn't make a difference for FB.


I disagree. I see small auto playing videos all the time in reviews or on feature pages, explaining little concepts or details that are much easier to grasp in a few seconds of (silent) video than in a paragraph or two.

The alternative is going back to large, low quality and bandwidth-heavy gifs, which isn't the better option in my opinion.


That's a legitimate use of video, but not of autoplaying video.


I'm not sure, I actually enjoy the autoplaying videos on FB. I probably wouldn't click 99% of videos I see but silently autoplaying it I get to see a few seconds without having to interact and then decide if I want to keep scrolling or click on it to get sound.


It's not a problem but a feature that increases Facebook's revenue by not insignificant multiples.


That is an important question that wasn't clear in the article. If they counted those sub 3 sec views in the count but not in the duration then it would definitely be fraudulent. If they didn't include sub 3 sec in the count or in the average then it is a valid technical decision. I am guessing they were included in the count but not in the duration.

They should have a histogram of durations.


It looks like they didn't count it in either metric. Peoples' complaint is that they excluded it from all averages and all reports, which inflates their average duration of video watched because videos less than 3 seconds aren't included at all.


That distinction makes all the difference to me. IMO one answer is fraud and the other is borderline valid / no big deal.


> If they counted those sub 3 sec views in the count but not in the duration

It sounds like this was exactly what was happening.

From the article: "Video views of under three seconds were not factored in, thereby inflating the average. Video views of under three seconds were not factored in, thereby inflating the average. Facebook’s new metric, “Average Watch Time”, will reflect video views of any duration."


One wonders whether this was an intentional filter to avoid scans of the video that shouldn't count as watching, or just a bug – some deeply nested, vestigial filter that everybody forgot about until some random engineer stumbled across it...


On my programmer hand, if I made the cutoff 3 seconds for the metrics, I would make it 3 seconds to count a view.

If the metrics and the view counts match, I don't see a problem.

If they calculate the average for > 3 seconds, but count EVERY view, I agree--fraudish.


This sort of Windsorizing will inflate the average anyway.

Making decisions based on "average" view time is still silly.


Why would it be silly? If I have an ad that has double the average view time than other wouldn't it make sense to invest more in it and less in the second one?


The distribution is what they really need to include. The average view time doesn't mean much when the distribution is super skewed, which it's likely to be for these ads.

If 0.1% watch all 30 seconds, 0.9% watch to 28 seconds and quit at the credits, and the rest all watch 0.2 seconds as they scroll through, that tells you something that "Average watch time = 0.5 seconds" doesn't tell you. Especially if last week, no one watched the whole thing, but a bunch of people watched for 2 seconds and then quit, and your average was 0.9 seconds.


This seems slightly overblown. I suspect this was a metric useful for engineering that found its way to external usage. That's because in many recommendation systems in practice you want to filter out the spurious views, and a simple way to do that is with dwell time.

It's not uncommon to require at least 50% of pixels in view for 1 second before you have an impression for static images [0, 1]. AOL defines an impression requiring 2 seconds for images [1]. Facebook likely did some analyses to find that 3 seconds was a good cutoff for their site.

There are more sophisticated ways to estimate dwell time but they seem uncommon in practice; perhaps due to their difficulty communicating to advertisers what the impression metric actually means.

For sites with many bots or inbound marketing you often find users bounce quickly which drives some of this timing. I'm a bit surprised it needs to be that high for Facebook without many bots or users bouncing quickly. Perhaps this is for mobile users scrolling quickly.

[0] http://advertising.aol.com/specs/terms/aol-viewability-terms [1] http://mediaratingcouncil.org/063014%20Viewable%20Ad%20Impre...


It's not overblown if you're lying to advertisers about the quality of impressions their ads received, and lying to investors about it too...especially when your stock has rocketed to making Facebook easily in the top 10 largest corporations in the world.

Not only that but Facebook actively brags about video impressions in their conference calls. This is fraud. I think this is something that should be investigated by the SEC to be perfectly honest. Doesn't mean they should be charged, just investigated; because if they are misreporting this, what other metrics are they misreporting to shareholders?


> quality of impressions their ads received

My understanding is that they only counted it as an impression if it was a view for at least 3 seconds. If anything, including non-impression views is lying about the quality of impressions their ads received, because they're not impressions.

I don't know if that's how video ad impression data is normally reported, but it sure sounds more intuitive to me.


> quality of impressions their ads received

I don't disagree, but FWIW the ads still had better ROI than Facebook's competitors, so the oversight (while embarrassing) may not have actually changed the willingness of ad buyers to pay for ads.


Better ROI than competitors. Did facebook tell you that?


Any modern advertiser knows their ROI from different channels. As a big FB spender, this doesn't effect us at all.


FB has more information about other channels than you do, I.e. for all you know they have algorithms favoring users to see FB ads that are on path that usually leads to your site based on FB pixel tracking. Then all the multi-attribution ROI of FB is nonsenses but you have no idea.

I'm not saying they are doing that, I am saying be especially careful with anyone in advertising who is misleading and who doesn't try to protect their reputation by being exact in what they say.


Regardless, dollar for dollar, if people are seeing bigger conversions or returns it doesn't matter whether they came from display advertising or 'organic content'.

If they buy something, who cares?


If someone is intentionally adding correlations using predictive information then they are stealing money you did not have to spend on traffic that was inevitable. It is like standing inside your waiting room to hand out your buisiness cards then claiming the sales commission.


If you have independent metrics of your own spend, then no, facebook didn't tell you. But you'd have had to have had similar spends in similar ways with all their competitors to know that.


Speculation on alternatives won't have a bearing on an investigation into the legitimacy of their claims.


Comments like these should have disclaimers about the author's position on the stock.


What specifically is the lie here? That they report average view time without trivial views? That seems like a more useful metric.

You also can't really directly compare these simple metrics across sites as they don't get at customer lifetime value. With these simple metrics you run into problems where the average video view may be higher on site A, but the average spend for users is higher on site B, and simply following the metric will lead you to an ineffective campaign. Whose responsibility is it to effectively manage an advertising campaign?

While I agree that more oversight and transparency is welcome here, I think jumping out and calling it fraud is exactly the kind of reaction that makes me feel this is overblown.


Brick and mortal local retailer here. I spend over $100,000 annually on advertising in my local market. That is all digital at this point. Facebook video is a significant chunk.

The lie is about the effectiveness of video on Facebook.

I directed a fair amount of additional ad spend to Facebook precisely because Facebook users on average seemed to be giving a lot of attention to videos on the platform, even if they were promoted. It actually seemed like there was finally something that could work to replace the kind of lift we would see from TV advertising 5-10 years ago.

This calls that into question (which may be putting it mildly). It also calls into question all of the other metrics Facebook provides advertisers.


Does the additional ad spend convert to an acceptable lift in sales? If yes, why do you care about video engagement? If no, why maintain such spend even if the engagement numbers are genuinely high?


How could anyone possibly know? You maintain spend because Facebook claims it's providing value. There's no way to actually know this with any certainty, and Facebook manipulating their figures doesn't add clarity in the least. That's why this whole issue is important — these people are spending millions on ads because of the numbers Facebook reports. Then people are spending millions more on stocks because of the profit generated by aforementioned hand waving.


Statistical measurement of lift in sales.

Most small advertisers wouldn't be able to do this, and attribution is hairy as hell, but the options for measuring this are there.


This is a bias of online thinking. Brick and mortar retail doesn't really know the conversion rate on ad spend. The old adage is that you are wasting half of your advertising spend, but the problem is that you never know which half.

So, we make decisions based on imperfect information. When it was TV or Radio ads, there were 3rd parties that measured the market and reach of the station. With Facebook, Youtube, etc we have to take their word for it.


You don't measure the conversion in case of a brick and mortar vendor. But you can still measure proxy values for effectiveness: an uptick in sales volume or number of clients after after starting a campaign, for example.


But we aren't just running one form of advertising at a time, so what to we attribute the uptick?

Also, our marketing doesn't exist in a vacuum. Weather, competitors, a traffic jam down the street, variations in employee performance, etc all impact the performance as well.

It's not like we can just A/B test Facebook video ads vs Pandora Ads and pick the winner.

So, we shift our budget based on the available information.

And the average video view length is part of the information that we used to make decisions.


I'm confused. you seem very confident that facebooks video marketing is a lie, even while you admit being unable to attribute sales.

I know multi-touch is hard, but doesn't that mean you should be less confident in your statements because the data is fuzzy?


He said that he used the avg view length on FB as a reason to choose that channel. If he had know that they didn't factor in <3 sec views he would perhaps not used that channel.


If the ads boosts sales, use them. The speciffic number seconds they are shown is irrelevant. You will have to do your own analytics anyway.


> Duh, just solve the fundamental problem that has plagued advertising since the birth of the industry!

That's kind of why Facebook/Google was ever supposed to be valuable: they empower the analysis. How do you do good analysis with bad inputs?


You don't really.

But assuming the FB/Google reported numbers are accurate, would you pay for them and just convince yourself they are actually increasing your sales?

You should be able to at least plot sales numbers against ad spending per week over a year.


Facebook also "should" be able to report numbers in a way that isn't misleading to their customers.


Inputs are not good or bad, they are better or worse.

In the 1300s, feedback to the advertiser took years.

In the 1900s, months.

In the 1960s, you could now measure the effect of blanketing a MMR with ads, and see how sales there compared to nationally.

This century, you have still-imperfect, yet much better than before, attribution.

Good analysis in the 1930s was different than now. Good marketing was the same, but with cruder tools.


Ah, classic courtroom fraud defense. "Feel fortunate you even caught me; that would've been impossible a century ago!"

Having wiggle room on numbers isn't a bad thing. Lying about the amount of wiggle room on said numbers is.


You can do the analysis that you want, there's no proof that advertising increase something. They are bad and should be banned from our live.


See my reply to the comment above. With local retail, you can't just measure the conversion rate.

It is not about the specific number of seconds. It is about the implied engagement and focus of the audience. It is about a reported difference in the behavior of Facebook users vs other online properties.


You never can. But are you just going to blindly spend money on ads and just believe they work?


Yeah, that is basically the entire advertising industry. Precisely what "brand exposure" is.


What did they define the metric as prior to this 3 second revelation?


The one thing that would make it misleading is if they reported the average time excluding views under 3 seconds, but reported impressions including them. From the article it is not clear that this is the case


I think your comment is the key one. I actually don't see a problem at all with what they've reported IF they also separately excluded views under 3 seconds from their total count of views (i.e. the denominator). If they included them in view count, I would think a lawsuit would be warranted.


This is the piece I've been digging for info on.

The dimensions to break out reporting columns for various view duration percentages has always existed. Any advertiser worth their salt is looking at the breakdown for stats on meaningful views because out outside of cases where your message is delivered in a couple seconds you typically only care about longer views and would just look at those numbers.

So the question is how deep this polluted other metrics.


As far as I understand it, what happens is that views under 3 seconds aren't counted towards the average. So if you have 999 999 views of <1 second each and 1 view of 3 seconds, the reported average is 3 seconds / 1 view = average view time of 3 seconds, while the true average is something like 500 000 seconds / 1 000 000 views.


So they would report that hypothetical as "999 999 views; 3s average"?


I don't think this is overblown. Considering that issues of money and trust are on the line, (and the inverse of trust which is fraud), and we are talking about a company that deals with over $17 billion a year in ad revenues, I'm surprised there isn't more scrutiny on this kind of thing in general.


When an impression isnt made (say 50% in view for 1 second) then no one is getting billed. This is the standard practice for internet advertising.

As far as I know Facebook does the same thing. I haven't read the Facebook advertising documentation in a long time but know that they outline many of their metrics in detail. People haven't been getting overcharged and it hasn't effected billing. From the article: "... it did not impact billing, and we have notified our partners both through our product dashboards and via sales and publisher outreach. We also renamed the metric to make it clearer what we measure. This metric is one of many our partners use to assess their video campaigns."

What are the implications of this? "Due to the miscalculated data, marketers may have misjudged the performance of video advertising they have purchased from Facebook over the past two years."

The other part about making it difficult to compare against competitors is true regardless of having the same metric due to site and user differences. Maybe facebook users spend more on average than twitter users after clicking on your ad for an ecommerce site? A/B tests towards the goal you're interested in is much more fruitful from a business perspective then say comparing click thru rates.


I currently spend $X,000 per day in advertising on facebook. Their reported clicks barely match up within an order of magnitude.

I've spent 4 days investigating this, so it's worthwhile to share the data.

For every 100 clicks I purchase, at a strong CPC ($0.10 - $2.00 CPC), I have found: - GA tells me i got between 40-60 sessions - piwik tells me ~70 - server logs show that page was requested ~400x (not indexed, a FB-only lander). I have IP and UA recorded, and both are rotated. My guess is that FB preloads the HTML LP when the impressions shows from mobile, so the browser is primed to quickly load assets (w/o sig resources to d/l images/etc -- just request the page that initially loads all that)

My point is attribution for conversions and link clicks is hard.

Any marketer worth their salt avoids the thought of "I paid for 10 clicks but GA only shows 6".

A better marketer thinks, "I paid $660 to acquire 2 customers. Is that acceptable?"


>A better marketer thinks, "I paid $660 to acquire 2 customers. Is that acceptable?" //

You're right, but that doesn't have a bearing on whether the advertising you bought was fraudulently presented though. Yes you should be wise, but we should still imprison people for fraudulent acquisition of other people's money, even if you still thought you ultimately got a good deal.


This. Experienced marketers know analytics sources will never fully match. That's why you look at things directionally and focus on measuring bottom line impact.

Source: someone who has had to deal with clients who insist that all our various js-based tracking methods should 100% match backend data and each other.


"I currently spend $X,000 per day in advertising on Facebook" you are wasting $X,000 per day period.


uh---what if he is making $X,000 * 2 in sales? Is it still a waste?


Literally every form of fraud ever is detectable and preventable if the victim is willing to dedicate her life to dissecting every metric and cross-referencing with a billion other metrics. How is this an argument? They lied, and it induced people to give them money.


I thought that this metric is supposed to measure the quality of the views that the advertiser is paying for. In that case maybe advertisers shouldn't pay for views under 3 seconds, but if they do it should count towards the average.


Based on the stat in the article about it resulting in 60% to 80% difference in total viewing time, I think they were probably reporting total views including views under 3 seconds and reporting avg view length discounting views under 3 seconds. This results in a pretty misleading viewing time. Still quite possibly a mistake, but I can see why advertisers would be pissed.


Isn't it the advertisers' job to measure the quality of the channel? Or does this kind of advertising have an intangible outcome like brand awareness?


How do you measure video watched on their platform? You can't since they host the video.


You measure the change in your sales while the campaign is active. That's the only relevant metric anyways.


Which is affected by about a billion other metrics, like if it's sunny out. Attribution is the fundamental problem of advertising, and getting closer to solving it is Google and Facebook's value proposition. This indicates that they are not nearly as helpful in solving it as they have been claiming since they've been providing bad inputs for their customers to do analysis.


In order for this to be a real issue, customers must be incapable of attributing sales to advertising campaigns but somehow capable of attributing them to seconds of video displayed. Doesn't make sense.


Yes it does, because getting accurate numbers from Facebook is a key part to calculating either of those things. You definitely can't do it with certainty, but you would hope the entity you're paying to record the data necessary to get as close to possible would in fact be an accurate source of said data.


If you are incapable of measuring the overall ROI of, say, a YouTube campaign vs. a Facebook campaign, then you are also certainly going to be incapable of determining the ROI of an X second average viewing time on Facebook vs. a Y second average viewing time on YouTube.

Therefore both the true and reported values of X are completely irrelevant because you are incapable of using them.

Please explain how the above reasoning is incorrect.


Detecting differences between several Facebook campaigns?


They would both be misrepresented the same way in that case and would remain comparable.


That works for impulse buys online I expect; doesn't seem useful beyond that. If you're running a one day festival you can have fantastic ad response - but if it rains the visitor figures are rubbish.

That's when you rely on metrics like bounce/view period to decide if your advertising worked.


It's perfectly fine to use the metric in question if its definition is clear. However, the definition was apparently not clear and Facebook therefore misled advertisers. Metrics are at the core of Facebook's business. If they don't get this right, that's fairly embarrassing, and raises a lot of questions about their other metrics. Perhaps this was not outright fraud but even if it was mere incompetence that's bad enough.


2 seconds is also Google/DoubleClick's cut-off, they refer to it as IAB standard: "For in-stream video ads, 50% of the ad’s pixels must be visible in the browser window for a continuous 2 seconds."

https://support.google.com/dfp_premium/answer/4574077?hl=en


Yeah. The main purpose of the metric is to compare the relative effectiveness of ads. If each number was off by the same amount, that doesn't affect the delta


... if there's only one advertiser you can buy placements from!!

Or indeed if all advertisers are using the same misleading metric.


"likely overestimated average time spent watching videos by between 60% and 80%"

Soooooooo... fraud? They say it didn't impact billings/revenue, but I have to imagine they will now be in a position to give discounts, if not refunds.


Yes. straight up fraud. "How Facebook is Stealing Billions of Views" - Kurzgesagt – In a Nutshell exposed this with detailed analysis and evidence. Watch it here => https://www.youtube.com/watch?v=t7tA3NNKF0Q


That video is discussing a slightly different problem than the recently discovered issue the article's talking about.


I don't think this is overblown at all.

I expect to see a class action lawsuit because of this. If you are a big advertiser and you can somehow show damages because of this error there may be a case.

There are some adtech companies recently who focus purely on reporting the metrics of your ad purchases. Kind of like acting as a third party. I forgot exactly what they are called.


This assumes most advertisers can prove the value of a video view of a certain length. I assure you as someone who deals with attribution on a daily basis that most cannot and the ones that claim to would have a hard time proving it in court. Attribution is hard.


I don't believe for a second that this was unintentional. The amount of press and mind share that Facebook has received for their "rocketing video efforts that rival YouTube" was totally worth pissing off some ad exec. Very few people will see this news compared to "Facebook dominates video" headlines that have been everywhere. This was a smart play on their part.


"Never attribute to malice that which can be adequately explained by stupidity."

I can see exactly how things like this could happen in large corps. You'd be surprised.


This might not be totally faor towards Facebook, but:

When it comes to Facebook I guess there are slightly different rules:

A good rule of thumb for Facebook might be: expect malice.

We have seen it with the way the made everything public.

We have seen how they decided to sell out its own user a year after proclaiming their love and how this wouldn't happen.


While your point is very fair, a long track record of behavior which pushes the boundaries of ethics becomes indistinguishable from malice. In these matters, people always judge you more harshly if you have taken advantage of their trust (many many times in FB's case) in the past.

Facebook wants to have the cake (the hype surrounding its success to be a self-perpetuating thing) and simultaneously eat it too (always expecting the benefit of doubt around its integrity around its self-reported success metrics).

While I welcome people with means doing philanthropy, I can't help but wonder if the recent announcement of philanthropy from Mark Z is an attempt to generate good press. If Mark Z is so keen on doing charity, perhaps he should divert all that money to the top 10 privacy watchdogs in the world. In today's world, that is as much a just cause as any other, particularly because no one really wants to talk about it.

Note: I am not saying the other tech giants don't indulge in these practices. Two wrongs can't make a right.


Admittedly, I'm biased. But along with Hanlon's law, there's also Hanlon's dodge.(http://www.ribbonfarm.com/2011/10/14/the-gervais-principle-v...) . Won't claim I know for sure, but this kind of thing wouldn't surprise me at all.


Coldtea's law: Never attribute to incompetence what can be explained by profit.


My data-weenie hat tells me that this is a silly metric anyway. As an advertiser, you really care about the distribution of viewing times, and in particular what fraction of viewers watch the video all or most of the way through. Arithmetic mean is virtually useless when the data has a power-law or other non-linear distribution, and it's highly likely video viewing times exhibits this.


As someone who helps make decisions around ad spend... not really. I would take a distribution if I was given it but it wouldn't substantially drive my advertising strategy. Views (over 3 sec views is fine), clicks, viewthrough conversion, clickthrough rate, and conversion rate are orders of magnitude higher signal measures than the specifics of how far they got through the video.

I'd imagine that view time distributions are way more important to new media companies that rely heavily on video media like Buzzfeed or anyone putting out content on Youtube.


What's the clickthru rate on a movie trailer? What is that going to tell you? Do I click on ads on TV or in the newspaper?

Most video ads I see are more informational than a call to action. Who really wants to click on an ad while in facebook on a phone?


The advertising arm of movie companies are definitely looking at the interaction rates of their trailer ads to help make decisions. The reason they do so is because it is a valuable signal.

> "Who really wants to click on an ad while in facebook on a phone?"

Advertising for apps (read: games) is a massive, massive market.


> > "Who really wants to click on an ad while in facebook on a phone?"

99% of the time I click on an ad, it's on accident.


Doesnt this vastly affect the distribution though? It excludes all views less than 3 seconds.


Then again, if a "video view" is defined as "watched over 3 seconds (50% of the video visible in the screen, IIRC)", then it sort of makes sense that "Average Duration of Video Viewed" doesn't include non-views.

For sure Facebook should attempt to name their metrics as descriptively as possible, but also advertisers should make sure they understand how different conversions are measured, what's included and what's not. Another example would be "Clicks" metric, which included all engagement (i.e. likes, shares etc), instead of just "Link clicks".


I thought about this as well, and concluded that FB must be reporting 'video views' as any time a video starts autoplaying (or something similar). In this way I could see the total estimated viewing time getting significantly overestimated.


It doesn't seem all that unreasonable, yeah. If a user scrolls right past a video, should that really count towards the average?


A user below summarized the issue the way I understand it:

> what happens is that views under 3 seconds aren't counted towards the average. So if you have 999 999 views of <1 second each and 1 view of 3 seconds, the reported average is 3 seconds / 1 view = average view time of 3 seconds, while the true average is something like 500 000 seconds / 1 000 000 views.

This is massive fraud, especially if you're shouting from the rooftops "Look at how much more attention people are paying to your videos in our platform!".


I didn't fully understand what the bug was, but I found on a German news page[1] with a slightly more detailed description:

Aufgezeichnet wurde zwar die Gesamtsehdauer aller User, geteilt wurde dieser Wert allerdings nur durch diejenigen User, die das Video länger als drei Sekunden ansahen

So the average was not calculated correctly: They accumulated the duration of all video views (including of those shorter than 3 seconds), but divided it by the number of 'legitmate' views - i.e. only those longer than 3 seconds. So you get a pretty big offset if you have many views under 3 seconds (which they probably do, thanks to autoplay).

[1] https://www.heise.de/newsticker/meldung/Facebook-Unklare-Vid...


We are using Facebook Audience Network in random order with other ad network for mobile ads. FB ads earnings are about 3-5% of other networks, with exactly the same traffic. Other networks disclose the percentage the revenue split between them and the publishers, but FB does not. What do they overestimate in this case, I wonder?


I don't think average watch time is a particularly useful metric, because no matter how they count views, there's some kind of arbitrary cutoff. If they count a view as soon as the player finishes loading, they'll include a whole bunch of people who didn't intend to watch the video, and in fact only "watched" a few milliseconds of it before bouncing. So that will drag the metric down artificially. I bet if they count views as soon as the video starts playing, they could drag down average watch time by improving player load time, because they'd count more unintentional "views" of very short duration.

I hope/assume this is part of a suite of metrics that they provide to advertisers, so you can understand it in the context of other things like video completion rate or counts of views that reach X% through the video.


It's like watching a wasp land on a nettle. You know somebody's going to get stung, but you just don't care who.


I don't get it, it makes sense?

Less than 3 seconds is not a view any more than seeing the picture of the video is a view. It doesn't count I'd say?

Or do advertisers have to still pay for a 2 second view?


While this is concerning, smart advertisers were looking at the distribution of view length percentages anyway as you can easily pull those columns in and their derived metrics.

Likewise, smart advertisers look at lift in conversion metrics when possible, in which case this stat is irrelevant.

That said, FB has not exactly helped things by making metric definitions a little obfuscated in general.

Personally, I see a bigger concern is them giving 100% view through conversion credit with a 1 day window by default as part of any website conversion action tracking. There are very few cases (like some retargeting situations) where you'd ever want to give full weighting for a VT, and while there is likely value in VT's, I probably wouldn't give 100% credit to them by default given the rate at which people scroll on mobile. But advertisers like to see big numbers, agencies like to show big numbers, and so you have platforms like FB aggressively try to push metrics like this and some of their rather loose definitions of "engagement" without great explanation of the nuances or pros and cons. These are largely left up to the advertiser to determine since, to be fair, they are very subjective.

Savvy buyers know this and configure their reporting and tracking settings accordingly because FB and PMDs give you those options. They are sometimes just buried.

Ultimately, IMHO FB and Google's greatest defense towards any of these sorts of claims is better and (more importantly) transparent attribution data and tools. If they can prove their value on the bottom line, other things often don't matter to many advertisers. Attribution is a tough nut to crack, but for advertisers spending large sums, it is critical to be successful in these channels.


It is unreasonable, even for a developer, to count 3 seconds as a play. Anyone working on a video product would know they were inflating the numbers.


The top-voted comment on this thread, by emcq, has a link to AOL's viewability guidelines which states

"For purposes of Video inventory, an impression is considered “In-View” to a user when at least fifty percent (50%) of the pixels in the ad are in the viewable browser window for a minimum of two (2) consecutive seconds."

Google here https://support.google.com/dfp_premium/answer/4574077?hl=en says "For in-stream video ads, 50% of the ad’s pixels must be visible in the browser window for a continuous 2 seconds."


That's the IAB's standard viewable impression metric: https://www.iab.com/wp-content/uploads/2015/06/MRC-Viewable-...


Ok, what am I not understanding here?

If they are only counting a view as over 3 seconds. And then only averaging the those views what is the problem? They tell me 1000 people viewed it (over 3 seconds), and then say the average view time was 10 seconds for those 1000 users. How is this an issue? Why do I care that 9,000 people scrolled past it and weren't counted in the viewer numbers? Now if they were reporting this as 10,000 viewers and saying 10 second average, I see the problem. Is that what was happening? Or what am I missing?


My suspicion (based on nothing) is that they averaged viewing over viewings with length >3 seconds, but _counted_ all views.

I.e. suppose your ad was displayed 5 times for 0.1 seconds and 5 times for 5 seconds. They would report this as "10 views, with average viewing time of 5 seconds".


I wish your comment was at the highest level. Everyone seems to be attempting to make the same point in the other subthreads. The example here makes it crystal clear.


My guess was almost correct. The issue was indeed that they use "views with >3 secs" in some metrics and use "all views" in other metrics. Do one of the other, don't mix them up.

http://www.adweek.com/news/technology/facebook-gave-ad-agenc...

> UPDATE: Facebook authored a blog post today to futher explain the metric-based miscue. David Fischer, vp of advertising and global operations, wrote: "About a month ago, we found an error in the way we calculate one of the video metrics on our dashboard – average duration of video viewed. The metric should have reflected the total time spent watching a video divided by the total number of people who played the video. But it didn't – it reflected the total time spent watching a video divided by only the number of 'views' of a video (that is, when the video was watched for three or more seconds). And so the miscalculation overstated this metric. While this is only one of the many metrics marketers look at, we take any mistake seriously."


In 2012 there were reports that 80% of traffic from Facebook Ads were automatic bots. So this is not the first time Facebook use tricks to get more money from clients.


From what I read [1] it's only one startup that claimed this, and I failed to see any confirmation from any other company. For me, no replication from companies that spend thousands every month on advertising = the startup messed up with its stat tools or wanted to create a buzz around its name.

[1]: https://techcrunch.com/2012/07/30/startup-claims-80-of-its-f...


Running campaigns on Facebook's advertising platform does seem to attract bots. It's not 80 % though and it's obviously not Facebook doing it. Why would they sabotage their own business?


This looks like a flat out lie. Startups have to sometimes do that to woo potential investors. Although I like Facebook overall, I do not agree with many of their business practices.


What was the actual bug? Does this mean that no video had an average view time less than 3 seconds?


They dont count views for less than 3 seconds in their view count.

They didnt include views of less than 3 seconds in their "average view time" metric, which is very misleading. Their metric was "average view time of video's in our view count".

Which is not the same thing and the latter vastly inflates the effectiveness of facebook video


This isn't anything new. Several YouTubers have "exposed" how Facebook tracks views. It's not accurate, but it's the system that has been in place for a while and they've defended it as such.


I am disappointed by FB. Next we'll learn that they do tax evasion.


If face book only count a view as over 3 seconds, and they measure the average view as only those videos over 3 seconds, what is the problem? Surely now its less representative?


When will Facebook 'content creators'(users) get compensated for the value they create?


Will there be a class action on this matter?


It might be inaccurate but does not matter much in advertiser perspective. Given the conversions etc. eCPMs conversion to the real expected value.


how much of their video income will they be refunding to advertisers?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: