"They also wiped all the commission fees we were keeping there."
Amazon can be a jerk and unilaterally change the rules, but wiping out their commision goes beyond that. I know Google does the same, but it's not an example to follow.
It's odd because Amazon has got a good reputation as a fair arbiter of disputes for buyers, but it seems not in this case.
Things like these might impact the perception in other cases, I always thought Amazon AWS would have better customer support than Google's developer offerings, this might make me rethink that (I never had to contact AWS support yet and read mostly positive things about them, so this is surprising).
I guess they violated some provision of the Amazon TOS?
Amazon has a good reputation from consumers, many people in the retail space who deal with them find them quite difficult (and arrogant) to work with.
The only company I've found worse is Walmart in the US. But US retailers in general like to make one anothers' lives as difficult as possible, even when they depend on each other to even exist, I don't get it! In western europe (aside from a few French companies) most businesses are pretty cooperative, and want to reduce workload not increase it arbitrarily (looking at you US companies, again).
I just think Americans love bureaucracy. It reminds me of Futurama with the "central bureaucracy" where everyone there takes pleasure in making things as slow and inefficient as possible.
It has nothing to do with bureaucracy. Americans loathe bureaucracy. However, American corporations are highly anti-competition and have been for years. This is an extension of that.
Suppliers and retailers on opposite sides of the table, both trying to get the best deal. I don't know why they wouldn't have an adversarial relationship.
You're talking about buying/selling, I am talking about what happens after that has taken place. After a supplier has agreed to sell at X price and a buyer has agreed to buy at X price they're no longer in an adversarial relationship, they're now in a symbiotic one.
How is it you think product physically moves from company A to company B? Through osmosis? There are physical trucks, and a ton of technological infrastructure behind the scenes to make that happen (from orders/invoices, through to inventory metadata (e.g. expiry dates), and all of the organisation that has to happen so all the trucks don't appear at the loading dock at the same/wrong time.
"Amazon can be a jerk and unilaterally change the rules"
Just guessing, but they probably didn't change the rules, only started enforcing rules they weren't prior. Therefore commissions earned would still be "invalid".
Considering Amazon fulfilled the orders sent from this affiliate and most likely re-invested the profits in their business (which is the Amazon way), makes it a dickish move.
Too true. I have a single blog post explaining how to do something useful on Amazon (make a partial payment with a Visa gift card). It got picked up by the Consumerist and other retail-type blogs and actually generated a few hundred $$ a month.
Then Amazon shut down my account, kept the affiliate fees, and stonewalled me. They said I had violated a ToS provision that affiliate clicks couldn't be for resale purposes (wtf?).
Thankfully I was able to reach out to someone at the executive customer service team, and they actually responded with an "oops, our bad!" It was a very frustrating experience being stonewalled and not even knowing why.
I know Google does the same, but it's not an example to follow.
When Google decides that clicks were invalid, they remove the commission and also remove the charges from the advertiser. The fact that they do this, and aggressively fight click fraud, is one of the major reasons they have so little real competition.
In the case of Amazon I'm not sure how they could do that...do they cancel all of the orders under the old agreement? Give customers their money back and ask for stuff to be shipped back?
As an aside, Amazon's Affiliate program is one of the big vehicles of noise on the net. When authors suddenly decide to write a big blog entry about the best headphones, or the best coffee grinders, etc, invariably you find that it was 100% affiliate fee driven. That might seem like a "well they might as well capitalize", but the corrupting influence of peddling commission goods is unavoidable.
The scary part of this isn't that they were denied access. It's that they were denied with no opportunity for appeal.
As an entrepreneur this is probably one of the number 1 nightmare scenarios: one day you're in business, the next day you're not.
We always say never build a business off an ecosystem you don't control but the fact is there are many, many businesses that do just fine from it - and on the flip side, even non-ecosystem businesses are fragile in other ways (such as Uber being regulated into illegality in certain parts of the US and Asia).
That's just business! Somewhere, someone is just waiting to fuck you over :) the challenge is to make your fortune or build up your defenses before they do!
My price comparison tool Shoptimate.com got hit a few months back. It's not Amazon only so I can keep the lights on but it sucks for Pricenoia.
Amazon has been changing the rules lately. They're not allowing installable software anymore and only allowing mobile apps that are not focused on shopping. They also refuse websites optimized for mobile. This is after they excluded affiliates over the state tax issues in the US.
Wikipedia says that Amazon gets 40% of their revenue from affiliates. They're probably dominant enough that they don't need to rely on affiliates as much as they used to.
Wiki is incorrectly characterizing their cited source. 40% is from third party sellers. (Actually I believe it is much higher now, but the way amazon breaks it out makes it hard to tell how much of their total revenue stream it is, as they only report their commissions). I've made the wiki changes.
I believe Amazon has realized that with their dominate place in the market most people would have gone to Amazon anyways, and they no longer need to pay to get people in the door.
There is no way that this is true. eBay has a similar affiliate program and historically I believe they do around 4-5% of GMB (gross merchandise bought) for the marketplace side of the business.
I'd imagine Amazon would be in the same ballpark. If Amazon was paying out 4-8% on 40% of their revenue they'd be in serious trouble.
"Wikipedia says that Amazon gets 40% of their revenue from affiliates"
It says 40% from affiliates and third-party sellers. The linked source says 40% from affiliates though. But it's 6 years old. I would love to see some current numbers.
This isn't a new thing. Amazon hasnt allowed installable software (without prior approval) nor have they allowed price comparison sites for years and years and years. Actually, have they ever allowed it?
This changed last year.
It now says: "Except as agreed between you and us in a separate written agreement (...) you will not use any Content or Special Link (...) on or in connection with: a. any client-side software application"
https://affiliate-program.amazon.com/gp/associates/help/oper...
But it used to say:
"You will not include on your site, display, or otherwise use Special Links or Content in connection with any spyware, malware, virus, worm, Trojan horse, or other malicious or harmful code, or any software application not expressly and knowingly authorized by users prior to being downloaded or installed on their computer or other electronic device."
https://web.archive.org/web/20130401033525/https://affiliate...
A couple of months ago Amazon changed its own price tracker to make it much less useful than it was before. I'm talking about the "Important messages about items in your Cart" that appears at the top of your cart when any prices have changed since the last time you viewed it.
They've had this feature for many years, of course. It used to be an incredibly verbose list with a bunch of boilerplate text repeated for every item, and the items were in apparently random order.
With only a few items in your cart this wouldn't matter much, but I use the Saved Items for a list of things I might want to pick up sometime but don't need right now. So I may have as many as a hundred items or more there. With Amazon prices fluctuating as much as they do - often by only a few pennies - it was hard to spot the more interesting price changes.
A couple of years ago they made some nice improvements to this feature on both the desktop and mobile sites. All the boilerplate was pulled out making the list much easier to scan, and the list was sorted in descending order by price change. So the items with the most interesting price changes were right at the top.
But a couple of months ago this list took a big step downhill. On mobile, the item names are no longer tappable links, and on both desktop and mobile the sorting was taken out! The list is still nicely formatted, but it's back in random order again, so the items with significant price changes are hard to find among all the two-or-three-penny changes.
I thought this was probably just a bug, but now I wonder if it was a deliberate move on Amazon's part. It seems they not only want to shut down third party price trackers, they don't want their own tracker to be very easy to use either.
On a side note, Amazon does not allow the use of its API for price trackers, they made that clear, and has lately made it very difficult to get the data in other ways.
If people can still access the same data just by going to Amazon with their browser, I don't see any fundamental reason why they wouldn't be able to get the data if they really wanted to. I've always believed that APIs on websites are more than just convenience - they're a way for corporations to exert control over the data they give out, sort of like DRM.
In the same way, they're also pretty trivial to circumvent; with webscraping, the major issue is that your traffic stands out, but distribute it widely enough and it won't look anything different from the massive traffic Amazon gets already. You could make a browser plugin and have your users do it for you (just make sure your privacy policies are stated clearly...!)
I'm willing to bet that Amazon's data is already being scraped by many other sites, so that also gives rise to this similarity: the "pirates" win, the "legitimate users" going through their API lose.
With all due respect: Spoken like someone who's never actually tried to do scraping on a large scale. This line just cracks me up:
> If people can still access the same data just by going to Amazon with their browser, I don't see any fundamental reason why they wouldn't be able to get the data if they really wanted to.
Just so naive, like a little kid saying they can build a rocket to the moon because they have a cardboard tube and some petrol.
In its own right web-scraping is difficult, and when the adversary is intentionally fighting against it it can be almost impossible. Modern web-scrapers often have to be smart enough to incorporate almost a full web-stack of technology, since if you lack JS or CSS rendering it is trivial to hide content from you or worse give you bad content the users won't even see (and also keep in mind the scraper has to "understand" CSS using its own logic, as it cannot "view" the result).
They can also dynamically alter how the page is rendered (e.g. if you're treating DOM like a tree and going down to a specific branch, it is trivial to re-order the HTML to break that). CSS effectively removes the relationship between HTML tag order and location on the page.
If I had a team of engineers working full time, I could likely maintain a 90%+ good Amazon web-scraper after about a year of pure development on it (likely using something like WebKit as a starting point). But nobody puts in this type of effect because sooner or later Amazon would sue you for something obscure and effectively win via lawyers fees alone.
An easier solution is just to hire Mechanical Turk-like people to look at pages, note down the price, and pay them 1c/price. It would cost an absolute ton of money, but you're able to start operating almost immediately (no prior development overhead). Plus you could limit it to Amazon's top 100,000 items and nothing more.
> I've always believed that APIs on websites are more than just convenience - they're a way for corporations to exert control over the data they give out, sort of like DRM.
Back in the days of Twitter API fiasco (2012/2013) I was playing Mass Effect, and this little gem of wisdom stood out for me as something eerily relevant:
"Your civilization is based on the technology of the mass relays, our technology. By using it, your society develops along the paths we desire. We impose order on the chaos of organic evolution. You exist because we allow it, and you will end because we demand it."
(context without spoilers for those who haven't played the game: mass relays are devices that allow long-range FTL travel, left behind by a mysterious ancient civilization; every species in the galaxy finds a mass relay at some point, which leads them to discovering other species and becoming a part of galactic community)
This is the key point. If all you are doing is scraping prices for a personal project, questionable legality is ok (even if clear legality would be preferable). But if you are trying to build a business, ya gotta stay on the right side of the law.
I don't think it's mentioned on the wiki but I think there might be a DMCA Anti-Circumvention case too if the site you're scraping has technical measures to prevent data from being copied.
Amazon has made it much harder to scrape their site in recent months. They now give captchas to bots that try and access their site to often by the same IP address. Unless you have a very large pool of low cost proxies, crawling millions of prices on a daily basis is challenging. I think this is what he's referring to when he says """and has lately made it very difficult to get the data in other ways."""
The best way to do this is with your users using a Chrome extension to feed the data back to you. User lands on a page? Send the data back to your scrapper processing API.
I've been scraping Amazon for the past few years. Yeah the captcha that they introduced around last February is a pain, but there are ways around it. I have and still do make my living from scraping (not just Amazon) - lemme know if you are interested and maybe I can help someone out.
I use Pricenoia to check prices in the different european Amazons, because pretty much all items can be shipped to other countries within Europe. As far as I know camelcamelcamel doesn't do that :(
I was looking at the Uber API, and thought about comparing prices between Uber and Lyft... then I read this blog post on Pricenoia and it changed my mind.
It does seem like a gamble to work with a company's API. It also seems like free work, but perhaps with donations you could make it work.
If you understand it to be a temporary venture and plan for the next step, you can still use API's etc. and create unique web apps that will attract users and give you either publicity or short term revenue.
My primary goal when starting any business is to create a sustainable, reliable income.
Also, I like to focus on businesses that, once I set my vision in place for satisfying customers, I can guarantee that the service will be there for my customers.
That's fine... my point was that temporary businesses/platforms can provide you the assets (exposure/cash/partnerships) needed to launch your sustainable, reliable income.
I'm not sure why you are arguing with me. I have stated what I am interested in when in comes to building a new business.
I am not looking for exposure, cash or partnerships -- those aren't the problem at all -- but I am looking for sustainable, trustworthy, long-term business ideas.
I have no problem building a business on another business' API's, but based on the track record of doing so, and your comments about them being temporary by nature, I think I will stay away from them.
Semantics3 (www.semantics3.com) has been tracking Amazon for almost 2 years now (we have price histories to boot!). We can, and do still offer price comparison functionality for products sold on Amazon; you can also compare these prices against other domains.
"They also wiped all the commission fees we were keeping there."
Amazon can be a jerk and unilaterally change the rules, but wiping out their commision goes beyond that. I know Google does the same, but it's not an example to follow.
It's odd because Amazon has got a good reputation as a fair arbiter of disputes for buyers, but it seems not in this case.
Things like these might impact the perception in other cases, I always thought Amazon AWS would have better customer support than Google's developer offerings, this might make me rethink that (I never had to contact AWS support yet and read mostly positive things about them, so this is surprising).
I guess they violated some provision of the Amazon TOS?