Well done Jordan, this is a good example of how insincere company culture, which endorses email spam and dark design patterns, WILL backfire.
In a company with great culture, this kind of project would have been caught by the various stakeholders of this project and it would not have been released.
However, at Linkedin, the engineers and product managers having learned over the years that these kind of things are "fine" and can be looked over as long as the feature brings more engagement, more traffic --> more money, made this dodgy product slip through the cracks of Linkedin.
I don't know how many times Linkedin has been on hackernews for email spam, when will they finally get it?
I talked to the main growth hacker of Linkedin a while ago who does all the email marketing and asked him if it's not a bit dodgy what they are doing. He smirked and told me how much these strategies boost engagement at Linkedin and how much money they make.
This is not something to be proud of, it's like being proud of having stolen 10 kids a lollipop today. Everybody can steal lollipops, build a drug cartel or big company with dodgy maneuvers, because then it is not about how gifted they were, it's how deceitful.
That is just not impressive. However, it's insanely impressive to build a company without being dodgy, but just by making a great product that your users love.
It's one of the most basic rules of growing a company that pursuing short-term gains (1-2 years) with dodgy maneuvers, directly translate into long-terms losses in the 10-fold numbers of the short term gains.
This is not too hard to understand, is the executive team of Linkedin not intelligent enough to get that?
I really don't understand how you can connect the dots of LinkedIn's (admittedly crappy) marketing emails and Intro existing. It seems quite simple really:
People loved Rapportive, people wanted Rapportive outside of GMail, Rapportive delivered.
> It's one of the most basic rules of growing a company that pursuing short-term gains (1-2 years) with dodgy maneuvers, directly translate into long-terms losses in the 10-fold numbers of the short term gains.
This is not too hard to understand, is the executive team of Linkedin not intelligent enough to get that?
Amazingly you've managed to crap not only on the Intro team, not only on the Intro team and LinkedIn executives, but every single LinkedIn employee – please tell me more about how to build this great culture you champion.
I've said it before on HN (and was attacked for it), but I will say it again.
LinkedIn is not a tech company, it is a bunch of business people who hired technologists to build a website.
Sure, lots of good tech has been open sourced by LinkedIn employees, but none of them sit in decision making capacities (or Intro would not exist). THAT is the culture problem.
The premise behind intro isn't a bad one. The employees (technical) who built it were excited about finding a way round Apple's very limiting system. I don't think subscribing to a particular view of corporate ethics is related to your technical skills. The scores of highly technical people working in the defense and finance industries is testament to that.
As a fairly technical person, I think Intro is a fairly neat product. There's definitely some implementation issues that mean that I'm not going to use it (I don't really want a linkedin advert in all my outgoing email), but I'd imagine it'd be pretty useful for some people. I already trust one large corporation with my email despite the fact I know they don't have my best interests at heart. Adding another one wouldn't be a deal breaker.
No problem! I'd imagine it's frustrating to see something you've worked on be attacked for having an agenda that you don't actually have. As far as I can see, the approach you took is the only reasonable one you could have taken to provide a service like Intro. Hopefully you can deal with problems like the one outlined in the article!
The implication here is that tech people are inherently more moral than business people, which I find more worthy of attack than any derivation from that premise.
You seem to be implying that tech companies can have a higher moral ground. They're still run by people. Technologists can still be culpable even if they're in positions of authority precisely because they are people. Any other way of thinking is a gross over-generalization of people's ability and desire to do the right thing.
Besides, even if they aren't in the positions of authority, technologists can still be culpable. I thought we already had enough examples in history of culpable people who were "simply following orders" to render such types of discussions moot.
"It's one of the most basic rules of growing a company that pursuing short-term gains (1-2 years) with risky maneuvers, directly translate into long-terms losses in the 10-fold numbers of the short term gains."
That may or may not be true, but my basic position is that I want to do business with people who don't care whether dodgy practices will or will not make money in the short, medium, or long term, they have scruples that outweigh their venality.
If we really think that it's not something to be proud of, we could stop calling these people "growth hackers" and start calling them spammers.
Edit: to clarify, "these people" refers to the type of people the parent was complaining about, who use deception and dark patterns, not to all people working on growth in general. Making the distinction is what I'm arguing for.
Well you know there are also really great/impressive ways to growth hack, such as Noah Kagan who grew Appsumo with his really cool emails that people loved or the guy who made Dropbox go viral by giving 250MB additional data for each new person that signed up.
So I'm still a bit confused, what is it exactly that's shady and spammy with "Intro". The basic idea is rapportive right? Get the linkein information about the person emailing you.
Is the implementation that's shady/spammy? I don't see how Intro isn't useful for recruiters to instantly have the linkedin profile of the person their emailing available to them on their phone. That way it's a quick reference check.
That's actually a really neat way of tackling the problem, but it does require people to change habits and use Ark instead of Mail.
So while Intro seems to have less friction, you have to give up privacy (HAH. Email and privacy always makes me laugh) to achieve it, which does suck.
Shame there wasn't a cooler way to try and handle it by only using headers or something, rather than MITM the whole damned email stack.
Ark is neat though. Out of curiosity, how do you handle double notifications? As in, I get a new email, now I've got a notification in Mail and Ark? Or does Ark replace Mail entirely?
"It's one of the most basic rules of growing a company that pursuing short-term gains (1-2 years) with dodgy maneuvers"
They're just copying the people in the financial industry. And the people involved made very nice sums of money, so their intelligence was successfully used to their own personal benefit.
Maybe the stock runup has made it such that they think they're just squeezing the last few drops out of this before they fully cash out.
This is pure speculation of course, but I gather this scenario is a risk with any company that has a lot of hype and such employee exit paths.
Good point, it's probably that no one is there anymore who cares about the long-term mission of Linkedin. Reid Hoffman is rich now anyway why would he care.
The reason for that is probably also though that Linkedin is not a passion product. It's not that their employees/founders are super passionate about connecting employees to employers, they are passionate about it making money. It's a bit understandable though, it's not easy to have a really strong passion for Linkedin or its mission.
Sidebar: What the heck does "main growth hacker" even supposed to mean? It blows my mind when people start coming up with new names for what they do. "Senior sanitation engineer" etc.
For some reason, a few people at HN seem to have a hard-on for criticizing LinkedIn. I never hear the same criticism of Facebook or Twitter here, even though their privacy policies are just as bad, or worse in many cases, and they also send bulk email.
I get spammed by all the social networks I've joined, and even by a few that I didn't join. I get mail from every newspaper or magazine I've ever subscribed to, and many that I never have. Condemning one business for doing this while praising another is just hypocrisy.
As are Google and Apple (albeit not always for privacy). I think the only oft-mentioned companies I've seen that consistently get better-than-neutral sentiment are Tesla and SpaceX, although there are probably others I'm forgetting.
And that's because Musk is the closest we can get to a tech "rock-star" (ignoring the Blizzard guys who are actual rock-stars and play in a sweet metal band).
I never hear the same criticism of Facebook or Twitter here, even though their privacy policies are just as bad, or worse in many cases, and they also send bulk email.
Intercepting email and inserting content is quantitatively different from sending bulk email. I think this programme is being criticised because it intercepts people's mail and inserts content - that's very different from receiving mail and leads to insecurity on several levels (they see your mail, sharing passwords, getting used to inserted content which can then be imitated for phishing).
Also people do criticise FB and Twitter here too, but the objection is not spamming.
"They see your mail." Ooh, scary. Email is effectively a postcard. It gets sent in cleartext all over the internet. If you think that email is even slightly private, you are misinformed. I don't see why I should trust LinkedIn any more or less with my emails than Google, Yahoo!, or any of the other companies that run regular old SMTP servers which process and store (possibly forever) tons of my email.
"sharing passwords" isn't a problem with this scheme because it doesn't use your password. It creates a separate IMAP account and links it to your regular account via OAuth.
"I think this programme is being criticised because it intercepts people's mail and inserts content" -- really? A sample comment is "they are spammers." Nothing there about man-in-the-middle, nothing there about email interception. I don't think most of the people bashing LinkedIn here even understand how Rapportive/Intro works.
The particular CSS exploit shown here seems like one that can and will be fixed. Obviously, there are other ways to do phishing via email, including just sending an email that looks like someone else's mail (remember: postcard). A lot of mail services are starting to filter spammy or fake-looking mail, so that may help.
I'm going to venture against the grain here: this is a really cool hack by LinkedIn. The OP's phishing attacks are equally cool - basically just HTML injection that leverages a vulnerability in LinkedIn's message rewriter. LinkedIn should be able to patch it relatively easily, just as they would on their website.
Let me ask. Is it also a "cool hack" when hotels inject ads into free WiFi?
... Because it is basically the same concept but worse because you just gave up the privacy of your inbox to be data mined and sold. All for some minor conveniences masquerading as a "cool hack".
... Because it is basically the same concept but worse because you just gave up the privacy of your inbox to be data mined and sold. All for some minor conveniences masquerading as a "cool hack".
My father's a B2B salesman who spends a lot of time using LinkedIn for lead acquisition. He's thrilled for this thing because he spends "way too much time fiddling with the iPhone browser to try and get this info."
I'm not saying the underlying security issues aren't valid, but to say it's a "minor convenience" is a failure of perspective.
And I dislike the basic premise of it too. But I can see a lot of other people being okay with the trade-offs needed... just like ads in my hotel WiFi.
I don't get why people say it shouldn't be made... it should be made, but they should educate the users as to all of the trade-offs needed.
As it stands, I think people are going to be weirded out (as we are) about LinkedIn reading your email. The regular user doesn't love them that much, I'd think, so I think it's likely Intro will fail.
That's the thing, though. I didn't try it, but my guess would be that even if they patch the CSS issue, I believe it would still be possible to set the visibility to hidden (notice they never set the visibility :), and use absolute positioning to set the new content on top of the old.
Also, I saw some different results as to where the content is injected. In some cases, it was right in the middle of my content - which seemed weird. Obviously, if they aren't consistently making sure their's has the last say, then it becomes easy to override their settings.
Of course - my CSS skills are awful, so someone please feel free to correct me on that.
I see HTML/CSS injection as a cat-and-mouse game that the owner usually wins, not the attacker.
It'll probably take them a while to catch everything. They could easily kill any CSS that's hitting the #rapportive ID with regex, or by making the ID a unique random string on each email. I believe placing all the original content in a "position:relative; overflow: hidden;" would fix your overlay concerns. Definitely an annoying cat-and-mouse game, but since they're ultimately writing the email that ends up in your inbox, they should be able to consistently prevent this kind of attack.
The main problem is that it's a cat and mouse game that really should not be happening in people's email. Saying, "hey, it's no big deal that we're clumsily wanking with your email contents/security because we'll easily be able to fix any problems that arise" is less comforting than simply not clumsily wanking with your email contents/security in the first place.
Replace "HTML/CSS injection" with the word spam and reread it again.
If things were true, spam would have died out by now. Admittedly, companies like Gmail do a good job of controlling but sufficiently large volumes of spam get through to the point where it's being opened and clicked where spammers still consider this a viable business.
So not, it's not a cat-and-mouse game where the owner wins. A sufficiently determined attacker will still get in. If your thesis were true, we would have built an unhackable computer by now.
One of the things that really scares me about this is the configuration profile thing. Yes. These profiles might contain just an email account. But the UI you get when the profile also contains, say, a new root certificate is the exact same UI.
As a user there's no way to see whether the profile you just accepted is just adding an email configuration or whether it's setting a global proxy server that even does SSL interception because the profile also contained that proxies root certificate.
Worse, by accepting one of these, you could also (again, same UI) accept whoever sent you the profile to use MDN functionality on your device, allowing to track the devices location (GPS accuracy) and to remotely wipe it.
For these reasons, I would never, ever, ever accept a configuration profile and I would recommend you don't accept one either.
This isn't just for linkedin either - a grocery store chain here allows for easy camera based self-scanning. The only thing you have to do is to accept their configuration profile so the phone can join a special in-store WiFi and I suspect other companies do the same crap.
Accepting one of these is as close to installing malware as you can get.
I am not a linkedin employee but I wonder what an employee should do or in this case have done when business was insisting on developing such a feature?
Stir the debate, escalate it to management, make your co-workers aware of the wrongdoing. If it's shoved under the carpet, leave, maybe write a blog post.
However, most people care more about their job than their values and for that reason, remain quiet. As long as the paycheck's comin' in, u kno.
You've probably heard of a story like this before recently, just on a bigger scale.
I'd argue that software engineers are some of the most resistant people in the workforce to these kinds of moral high-wire acts. That's simply because they have way more job security than others. They can pick and choose companies with which they are in moral agreement and jump ship when something they disagree with is forced upon them.
That's because it's super easy to rationalize things like this. From the outside, hearing about Intro for the first time, our initial reaction may be "WTF?". But from inside the company it was a slow boil. You know that you mean no harm, and the people you work with are good people, and they mean no harm either. And hey, you're taking all these precautions like using separate servers, and getting security audit checks. And hey, isn't this a clever way to add a cool feature?
Before you know it, it's too late to say "no". Something that may have started as a good idea has transformed into a monster. Human cognitive biases will then kick into action and save you from admitting to yourself that you're part of the problem.
So most of the preconceived uses for their technology were already hashed out and approved prior to purchase...
Even if that wasn't the case, the technology needs to be integrated somehow in the form of a product.
So as a low level employee, what are you expected to do? Be known as a bad and difficult employee? Quit? Even if you quit, others will happily take your place.
As non employees of LinkedIn we can expose this behavior as unacceptable and deceitful.
With SPF and DKIM you cannot spoof the envelope address and the “From:” header email address. If SPF would actually be required instead of optional (or even ignored), we could be free of 99% of SPAM and phising emails maybe as soon as 2009!
Anyway the point of the post is that if you just add the badge thing you will end up with two different badges, which the user would obviously find suspicious. Instead, the way LinkedIn rewrites messages is exploited to get only a single badge but manipulate its content (and/or the message content). Since the message looks authentic (it’s been filtered by LinkedIn) it gives the user a false sense of security, which makes a naïve phising attempt into a very effective one. After all the point of the message rewriting was to give the impression that LinkedIn Intro is an extension to the actual email client.
Thanks for the article. It seems unlikely that you could scale the first step: harvesting the iPhone profiles in the first place. I was under the impression this is a one-time download. Is there any realistic way to get a significant number of these?
It's the expectation of the user that this is hacking. Inserting the HTML into a non-Intro user's email wouldn't make sense. But sense the user is expecting that data, and for it to be validated, spoofing it becomes much more valuable.
I have said this many times before but I really need to get off my ass and just do something. Looks like anything sells these days. Especially a spectacle. Just do something that attracts attention. Sell it. Retire. This is not about tech.
You don't seem to understand Rapportive. This is a new thing they built as members of LI, not the product they bought. Rapportive is a great tool that's extremely useful. If anything $15m was low.
Based on everything I've read, LinkedIn didn't develop this technology. They bought it when they acquired Rapportive (which, by the way, was a Y Combinator startup).
Can someone explain the technical details here a little more? I feel like a few steps are missing in the explanation. Why does Rapportive/Intro need a separate IMAP account attached to the device? How could LinkedIn ever think email could be secure? Email is a plain-text protocol based on trust. Can't people just spoof source addresses and inject whatever they want into the next email server in the chain?
Sorry - I tried to keep the post to a reasonable length. I'll be following up with a more detailed post later. :)
The separate IMAP account is likely so that they wouldn't ever touch the user's Gmail credentials. This way, they do everything via the OAuth token they retrieve. Also, I'm not sure if they can know for sure that the user has synced their Gmail account to their iPhone or not.
I don't know how Linkedin thought this was a good idea. This is clearly one of those cases where the functionality benefits are greatly outweighed by the security risks. This shouldn't have been made.
I don't know the details of IMAP well enough, but isn't the proxy what allows them to inject HTML into the email that iPhone's Mail app sees, but not any other client?
Yes - they need to perform what's called a Man in the Middle Attack (MiTM) to inject HTML into your email.
Normally, your iPhone (and other clients) retrieve email from Gmail's servers using the IMAP protocol. To inject content, Linkedin setup a security profile which placed themselves in the middle so you connect via IMAP to their servers, they fetch the content using IMAP from Google, inject their content, and feed it back to you.
This is why the email is not permanently changed. Only changed en route to your iPhone.
In a company with great culture, this kind of project would have been caught by the various stakeholders of this project and it would not have been released.
However, at Linkedin, the engineers and product managers having learned over the years that these kind of things are "fine" and can be looked over as long as the feature brings more engagement, more traffic --> more money, made this dodgy product slip through the cracks of Linkedin.
I don't know how many times Linkedin has been on hackernews for email spam, when will they finally get it?
I talked to the main growth hacker of Linkedin a while ago who does all the email marketing and asked him if it's not a bit dodgy what they are doing. He smirked and told me how much these strategies boost engagement at Linkedin and how much money they make.
This is not something to be proud of, it's like being proud of having stolen 10 kids a lollipop today. Everybody can steal lollipops, build a drug cartel or big company with dodgy maneuvers, because then it is not about how gifted they were, it's how deceitful.
That is just not impressive. However, it's insanely impressive to build a company without being dodgy, but just by making a great product that your users love.
It's one of the most basic rules of growing a company that pursuing short-term gains (1-2 years) with dodgy maneuvers, directly translate into long-terms losses in the 10-fold numbers of the short term gains.
This is not too hard to understand, is the executive team of Linkedin not intelligent enough to get that?