Putting surveillance video on the cloud is... kinda dumb. It's rarely viewed outside your network, and local drives cost drastically less than the bandwidth needs. Also it's incredibly sensitive data that shouldn't leave your network without really good reason anyways.
The solution to this hack is simple: Shut this company down, because it's a bad idea.
I think the biggest selling point to cloud security cameras is to have a copy of the video not at the location. If your house / business is robbed or burns down you will possibly lose the video. This video could be used to find the theif or arsonist.
Ideally it would store video locally, then encrypt it and upload the last minute to the cloud.
The biggest selling point is really the OPEX sales model, and outsourcing of the general care-and-feeding of the recorders/servers. The tradeoff is that you tend to get lesser quality video and/or less scalability of the size of the system from bandwidth limitations.
Theft of recorders/loss of recorded video is much much less of an issue than it is generally made out to be, and most systems have various means for auto backups, dual recording, etc. Many also have the ability to send select segments of video, based on motion or analytics, to cloud/FTP/email for free, which adds some extra resiliency if you are really worried about random arsonists :)
I am still honestly confused by the whole "OpEx > CapEx" thing. Apparently for some people this is worth paying over double the price when you look at things on an annualized basis, even for an established business that's got a very stable environment?
The concept of having a predictable recurring bill, and known costs can have a lot of value. In the opex hardware model, there is usually a hardware refresh built in (a lease of sorts) and also covered replacements for any failures before the refresh.
This makes no sense to me, because almost no OpEx-based services operate on a predictable billing model. Usage based pricing means they are subject to change.
If I buy a $5,000 server, and it lasts me five years, I can plan around needing $1,000 a year set aside to replace it. Meanwhile, my AWS bill can suddenly be $20,000 with no warning.
The hardware refresh model you describe... I've actually only ever seen in a CapEx appliance, where paying for the support contract got you those refreshes regularly, replacement for failures, etc. but had an initial upfront purchase cost.
When the news comes out that a major company has signed a deal with a cloud provider, it's that they've committed to spend $X over N years. Presumably the deal would be renegotiated if their usage changed dramatically, but it is not actually variable pricing.
I am pondering moving to a rather remote estate with my wife but we're both city kids and have zero clue how to defend ourselves in such conditions (I am even thinking we should take shooting lessons and bring guns to the estate as well).
Part of our plan is a surveillance system -- I planned to have an on-site ZFS storage cluster with several cameras that periodically encrypts the last 5 minutes and sends them off-site. Not sure how complex such a setup might turn out to be though... Maybe there's a way to make the storage cluster itself auto-replicate remotely often enough? I'm not that educated yet.
Do you have any recommendations? I don't want to spend $5000 due to paranoia but I don't want to be defenseless in case of a robbery either. What's the middle ground?
The area I want to move in is known for its heightened percent of home invasions and burglaries. So I don't know, maybe it is excessive. I just don't want my life savings to fall into the hands of a random burglar.
My bad for using bad figure of speech. Of course most of my savings won't even be physical (bank / cryptocurrency / investments etc.) -- I meant that I'll have all my possessions there (expensive computers, TVs, furniture, kitchen tech, what have you) and I would like to avoid having them stolen while I am grocery shopping in the nearest city, you know.
No clue. But I am accepting another commenter's idea that having very visible cameras (on purpose!), solid fence and 1-3 guard dogs should be enough to deter 99% of the potential home invaders.
To me excessive would be automatic tracking turrets with really nasty things mounted on them. I recall someone made one that fits in a briefcase and uses paintballs. It had something like 90% accuracy on fast randomly moving targets. But for some people that might not be excessive, probably depends on the circumstances.
Forgive me asking but which country do you live in.
I often have debates with my wife about country vs city living, but I don't think buying a gun / attack dog / big security system ever comes into the equation. (UK based).
Am I not paranoid enough, are you over paranoid? Is there a Bayesian equation we need here?
The guy cycled through Europe and India, met some guy in a bar in Italy and ended up sleeping on his couch and eating home cooked Pasta lunch the next day. A different approach would be to drive with a gun under the seat. I think something is missing.
Often what's "country" in the UK (outside of Scotland) would be considered exurban in the USA. You might possibly feel different about security at a truly remote home in N America.
Me and my wife were thinking Costa Rica but got warned by an acquaintance that burglaries and home invasions in remote areas are a regular occurrence. And we want a sea-side house.
To be honest, the easiest way is to 1) Buy and install a bunch of ring cameras and 2) Buy and learn to use a Mossberg 500 with a 18" barrel. Maybe another gun, too. Total cost: Idk, $800-1800.
Yep, we thought of buying two specially trained German shepherds (the kind of training that requires you to be there so the dog bonds with you and trusts nobody else).
Still not sure about guns but the dogs would indeed be a very good investment.
An Anatolian livestock guardian dog is a vastly superior natural guard dog, compared to a GSD. They can not be bribed with food, and they spend every second of their waking day looking for intruders to destroy. Very gentle with children and small dogs, though.
Downsides include being extremely smart but not very trainable (smart like a wolf) -- they decide what's a threat and what isn't, and they don't care about your opinion. They guard naturally. Also, you may find yourself having to bury dead coyotes on occasion, as they will murder any that are stupid enough to intrude.
I think you might be overthinking this. The dogs don't even need to be large. If the property is more than a few acres just get 2 or 3 herding dogs. They're upbeat, energetic, and above all very noisy if a stranger wanders into your yard. I very much doubt anyone will ever bother you.
> Still not sure about guns
How remote is it? What's the police response time? If it's really in the middle of nowhere then it seems like a good idea to have something on hand just in case.
> How remote is it? What's the police response time? If it's really in the middle of nowhere then it seems like a good idea to have something on hand just in case.
Exactly because of that: remote and small country (Costa Rica is one idea so far), we would prefer a sea-side house which is NOT very close to a city (we think 20-30km away from a city is optimal) and we have no clue about police response time -- but I can't imagine they'll be there 2 minutes after I make a panicked call that I am tracking 3 armed burglars, especially if not in a city.
> I think you might be overthinking this.
Very possible. Right now I don't even have the money for a down-payment, but me and my wife both share this dream and started brainstorming it and doing some preliminary planning.
Above all, I really want the dogs to not be able to be bribed with food. Not a fan of dogs that bark at every single small disturbance but oh well, if that's the price you pay for living in a remote area and having good security then okay.
Oh, I was imagining remote rural in the US or a similar country (Canada, Scandinavia, etc). I have absolutely no idea what public safety concerns might exist in less developed places.
> want the dogs to not be able to be bribed with food
It's more that a solid looking fence, a few dogs, and some visible security cameras are highly likely to deter any attempts in the first place. If someone is armed and willing to shoot your dogs then you have much larger problems to deal with (and probably don't want to be living there to begin with).
Was also thinking the same. Security cameras should be used where there is an incident and additional information is needed to clarify what happened. Then you check the security camera footage to figure out what happened. If there were no incidents there is not reasonable need for anyone to look at that footage.
edit: not talking here about active monitoring security cameras used by guards.
There are plenty of use cases, such as the various forms of analysis with ML, which are often only implemented outside the "company firewall" (this idea is antiquated.) Of course, this could and should be handled inside a trusted boundary (I can't think of a better term for today's networks), but in practice security here is fairly immature and people try things when there isn't good governance in place.
This sounds like general problem of enterprise security. There are no consequences.
I can entirely get why a company would outsource IP cameras to a third party cloud, even with storing data on-site. Business runs on contracts. It's entirely normal to contract out everything except your core competencies, if it's cheaper this way. It's how you turn CAPEX, complex OPEX and high risk into simple OPEX and low risk. A contract is in big part a risk shifting tool. This works well in practice... outside IT.
The problem is, with IT and data, there's a mismatch between expectations and reality. An enterprise should feel safe buying their video surveillance from Verkada, because between the contract and the legal framework, Verkada should be bankrupt now, and their management possibly facing jail time. That's the part where contracts work as Cover-Your-Ass tool: if you shift risk and liability to outside party, the liability is not on you.
However, this only works as long as the other party actually internalizes the risk and liability. Since there are no consequences for mishandling data, operating IT services you're not structurally competent to operate, and eventually having your crown jewels stolen - the contractor doesn't really internalize risk, has no incentive to mitigate it.
All this to say: Verkada should go down after this, and their customers should be named and shamed widely - the latter is so that future customers of IT services put more care into vetting companies they contract IT out to. You shouldn't get to CYA with a contract where assumptions around contracting are broken.
I work in a large enterprise. If my employer were to decide to exclusively store security footage on our own infrastructure, then I would have access to every piece of hardware it’s stored on. The risks associated with that is why every piece of mission critical data where I work is, at a minimum, backed up in a 3rd party facility.
There's more information here as well. Cloudflare was apparently operating network connected facial recognition cameras in their offices.
I'm not someone who's crazy about privacy, but this is a pretty dark indicator for a company housing DNS query records. Maybe its time for someone to build a proxy for tunneling Cloudflare DoH/DoT over tor or some other free mixing network.
How is that something to be worried about? There are companies out there that try to monitor if employees are in rooms/areas that they're not supposed to be. You can do that with badges/RFID but then people can take a card or slide by in various ways. (Happens all the time at big companies - people just tailgate) If anything, they might be taking privacy more seriously by not letting people without authorized access into secure areas.
I think you give up any sense of privacy as to where you're located in an office or where you've been in an office when you decide to work in an office owned by some employer. I don't know why there'd be any expectation there.
I find it fascinating how okay you are with your employer tracking you. We aren’t to the life contract part of the dystopia yet, quit trying to skip ahead and give away your freedom so easily
If you are in a secure area, like a server room for example, it's perfectly normal for there to be badged entry, cameras everywhere etc. There will also be signs everywhere telling you this.
If it's really secure there will be monitoring of all entrances, including corridors. (And there will still occasionally be people successfully tailgating, usually for perfectly innocent reasons like forgetting their badges at their desks etc. Real security is all sorts of fun.)
In fire/hazard conditions, security systems are required (at least in Australia) to permit free handle egress from any point in the building to a fire escape.
Any access control system has the capability to integrate with a fire system and allow this.
I believe the general policy in the States is "one swift motion" to exit a room which is why you see mostly crashbars and lever handles as egress, mostly on push doors for the primary egress path.
In secured areas where they want you to swipe out or places where they might get tripped accidentally, they sometimes have like a 15 second lockout before actually tripping the door.
I've been in just one server room and they just had a motion-deactivated maglock tied to an electric strike so in the case of a power outage a simple mechanical lock could be opened but otherwise you need to badge in/out.
Some facilities get exceptions to fire policy and require employees to go through training of sorts. Diablo Canyon Nuclear Power Plant is one place I visited that did not have emergency egress. No badge? Call the guards, that is the only way out.
Fire hazard and false imprisonment. Ask walmart. You accidentally lock someone in the store and you are looking at a civil rights law suit. You cannot restrict another humans' movement without due process.
Never been to a Walmart specifically, but every store around here broadcasts a "closing the store in X minutes, please be outside of building by then" message on the loudspeakers a bunch of times, then the employees pack some things up and walk through the entire shop to check if everything is ok and the security personnel, being the last to leave, check all cameras before finally locking up.
This seems like plenty of due diligence for the store not to be liable is someone gets locked inside.
Obviously not the same type of facility, but I have seen buildings where the closing of smoke shutters opens otherwise locked doors, revealing an alternate fire escape from the corridor to the stairwell.
I'm okay with my employer tracking me if I'm on their premises using their property that they've given me. I'm not okay with them knowing anything I do or where I am outside of work, but if I'm at work then I'd be confused if I wasn't being tracked in some way.
Not at all in the paranoid "are you slacking off?!" sense, but just security information like knowing when I've been in a server room, or knowing if my work computer sent traffic to a known botnet C&C. If there's a security or theft incident and they don't know who's been in their building or what their computers are doing, it's pretty much impossible to investigate anything.
I understand that in places like Europe there's a very different culture and workers have a lot of protections from things employers may want to do, but not everyone around the world feels that way. Basic record-keeping of when badge-restricted doors and computers are authenticated to doesn't feel invasive to me in the slightest, even if others may strongly feel it is invasive.
There are many things I would find egregiously invasive, such as a manager inspecting all the websites someone visits to assess how productive they are, or timing people's bathroom breaks, but I just avoid such companies.
I don’t understand why people think the employer cannot check whether the employee is slacking off.
Maybe what we should prevent is employer keeping months of proof and only bringing it up as inappropriate later, but if the employer uses the camera to tell an employee within 24hrs that he needs to ramp up, it feels ok. Maybe we should impose rules like “24hrs max” and “can’t be used legally, just orally.”
> I don’t understand why people think the employer cannot check whether the employee is slacking off.
On some level it depends on what 'slacking off' means.
I've had employers where 'slacking off' meant actively doing some %mundane/repetitive/unnecessary% task with every moment of my free time. We were literally pulling the finish off the counters; there was no need to keep dusting them.
I've had software shops where reading integration documentation was 'slacking off'.
An interesting data point; In Germany, MS Office doesn't track how long you have been editing a document. My understanding is this is because the law there more or less says if you pay someone to do a task, you aren't supposed to (i.e. can't) care about how long it took them to actually do it as long as it was done on time.
So I guess that's my problem. There's a very fine line between employers using surveillance to catch 'bad actors' and employers using surveillance as another tool to bully substandard work conditions onto people.
My guess is that micromanagement actually decreases quality and productivity as well, just due to the disconnect between management opinions and real-world employee experience. If you are judging performance on the output correctly, the employee will, out of own self-interest, maximize the quality and quantity of the output while minimizing their own effort expended in creating it.
100% of the information an employer needs to determine my productivity can be found by looking at my work output. They don't need to know what I'm doing or looking at at every given moment. The results speak for themselves.
I'm sure they have a legal right to check (in the US), but I really wouldn't want to work for such a company and would immediately start looking for a new job if it happened to me.
The day one says I'm "slacking off because we noticed inactivity on your laptop" is when I stand up and walk out the door. Hasn't happened yet but I suspect it will at some point.
Cloudflare sells security to people. If you don’t want to work at a company that has security requirements like that, don’t work there. Lots of people choose to donate their fingerprints, facial data, life history, and polygraphs to work for the government. That’s their choice to make.
If that info is well taken care of is one think. If it ends up floating on the internet is another. Rfid badge data floating on the net creates is useless however other personal data could be very toxic in the wrong hands. And usually this info leaks thats why its not a great idea to let it outside the network let alone record it in the first place
How is being recorded by your employer while on their premises giving away your freedom? It would be a different thing if they were tracking you out of work, but when you enter a premises owned by a business you kind of implicitly agree to be surveilled by them, as it is their right and freedom to protect their assets.
We were tracked by contract (badge into building, badge into area). We couldn’t leave the work area un-attended which was a pain, so there were “processes in place” (last person badge etc..).
Generally we knew they left you alone unless you were cheating. (Having someone badge you in when you weren’t there was a fire able offense).
I don’t miss it, but it wasn’t that bad. Of course having the work network not on the internet what else could we do but work...
Its only natural really in this race to the bottom. If your zero hour contract doesn't have room to pay the bills you are not just not worried about tracking, you'd take anything that might show how hard you've tried.
One of the biggest use cases presently is SARS-COV-2 tracing to figure out who needs to be notified they were in proximity for X-time of someone with COVID-19.
It really comes down to how it's used. As another commenter pointed out, any company using badges to swipe into doors can track your movements. Most cameras are positioned near entry doors, exteriors, or public areas as it is. The main difference here is the amount of information collected on an unauthorized entrant, and the fact that maybe badge-borrowing doesn't go unnoticed anymore.
I really doubt Cloudflare is the type of company to be tracking where each employee is and whether they are taking too many bathroom breaks. It's definitely an area abuse is possible, but probably not an area it's likely in Cloudflare's case.
I absolutely agree. The thing that concerns me is these cameras sitting on the internet. It says something about how overworked the security team is. I trust that they have good faith, but I don't know if they have the resources they need.
People other than the ones you agreed to let monitor you, well, monitoring you. Also, it's a major risk to the company itself, who knows what can be read off of employees screens if they're compromised.
Yeah, it seemed far fetched to me at first but I guess surveillance might be useful to a 3rd party. I read an article a while back on how people make equity trades based on data found through satellite images of refinery tanks and whatnot. I guess unsecured internal surveillance cameras could allow an outsider to find out if a company was really busy or just faking it.
Lock the memory so that update is physical only and restart regularly to avoid no-memory malware. Not 100% secure and very inconvenient, so people prefer to isolate IOT in its own network and preferably have a good network security like putting the devices behind VPN/firewall/other gatekeeper.
Actually, if you want to have IOT access outside of the network, the best approach is to close all ports and for the device to initiate connection with a control server. The device is dark when scanned while a heartbeat signal will ensure connectivity. This will require a good security on the control server, but that is okay because server security is much better understood and does not suffer from the constraints of the embedded software.
Someone wanting to break in can check if anyone is there or see where easy to steal stuff is kept? Or on a larger scale you might leak when and how security guards make their rounds.
This would be a much bigger hack if it were Wyze. Wyze cams are generally placed in residential interiors and contain a mic that can not be disconnected without physical removal.
It wouldn’t be as newsy, though because people put these in their own homes.
The only way this group raises awareness instead of angering people is by targeting companies and institutions, rather than forcing people to confront their own compromises with technology, time and effort.
I am very impress with their features. I am planning to buy couple of them, but I would never point them inside the house. Probably will put them in their own wifi network.
I built a home grow solution with RPi + Webcam + Google Drive with some python script. But the performance to upload Pics to G Drive was slow and the way viewing the pics uploaded to G Drive was not very convenient.
If you're serious about security I recommend against wyze. You're locked into their platform and it isn't fully reliable.
If you want something like a video of that opossum sneaking in the attic, coyote peeing in your rose bushes, that kid across in the street with the razer scooter and german shepard not cleaning up the poop off your lawn or what your iguana is up to when you're not home then it's great for that.
The UI is probably worse than your gdrive setup. You have to use the wyze app to access it, which has a terribly slow scrubbing interface. The more convenient part is to check the motion/sound triggered 15 second clips which are upload to their cloud servers. Other than that I find myself frustrated and just take out the microSD card and browse it on my computer.
Also don't buy the sense kit. It just doesn't work. I've even tried experiments with it right next to the camera and it's not consistent enough.
If you want to use the wyze cams are generic IP cameras they offer unsupported firmware for that but then you'll need to roll your own DVR and I'm not sure if that supports all the new features the V3 offers or follows the genric protocol for remote pan control.
The first thing I do with a wyze cam is physically remove the microphone, because it can only otherwise be turned off in software which is configured in the cloud.
I'd consider using them with the OS firmware change, but hadn't thought of also running them on an independent network.
Wyze cameras have open source firmware that can be loaded (DaFang Hacks) that works pretty well.
I've never used (or trusted) the Wyze firmware anywhere on my network, but I use a couple of Wyze cameras pointed outwards from my house connected to a a DVR with motion detection.
Not sure when you bought yours but i haven't been able to get DaFang working in at least 18 months (due to OTA firmware updates before attempting DaFang)
I find the Wyze cams great! Cheap. They do have 2 factor auth support for apps like Authy. It doesn't sound like you have enabled the premium features. Person detection is really good for exterior facing cams. Hook it up to Alexa Show and you have person detected notifications also access to any cam. Premium also has longer clips. For the price it's kind of OK to be semi locked down. I believe the reason for not supporting RTSP is because of hardware limitations on V2.
A couple years back (2019) Wyze had a breach [0], which accidentally exposed another breach that they hadn't been aware of. This came just six months after a previous breach.
But it suggested that Wyze can remotely disable all encryption and so on whenever they feel like it.
And that for a "subset of users" they collected: "Height, Weight, Gender, Bone Density, Bone Mass, Daily Protein Intake, and other health information", which was included in the breached data.
Putting the surveillance video on-site is... kinda dumb. If someone breaches your facility and wants to take the evidence of them doing so, they just have to steal the video storage.
Cloud security footage makes sure that you have at least the footage up until they disable your network.
It's amazing how much security people and companies are willing to give up for a smidge of convenience. Properly deployed CCTV has little worry of hackers since, hence it's name, it is closed-circuit.
That seems like more trouble than it's worth though. Do you like just check every hour to see if someone has fallen or something? That seems like something that could become either an obsession or something you'd forget to do.
Not the OP, but I have a significant amount of anxiety sometimes about if something bad has happened or if my pets are okay or whatever. It's an extreme version of "did I forget to turn the over off".
Being able to remote in and go "ah, everything's fine" basically can instantly turn such anxieties off, rather than say, having to live with it until you finish out the work day and can head home.
This made me wonder if there's a place for a company to disrupt the video cloud industry by selling CCTV cameras and devices that would require physical updates sent on a usb drive.
Get this: closed circuit television systems actually existed before updates could be delivered by the internet. Another crazy fact: there are still products sold which don't connect to the internet. You don't see nearly as many ads on these products though and don't seem to be as popular these days.
It's not actually hard to do that today with existing equipment: Put it all on a network switch and then just... don't plug it into the Internet. Plenty of on-premise solutions exist and work with or without remote connectivity.
When Microsoft Exchange servers were hacked a few days ago, people on hacker news were talking about how most companies are not capable of securing their own on-premise systems and they should have used Office 365. There really isn't a perfect solution. There are trade-offs.
It's a lot easier to secure on-premise Exchange than Office 365. For one, having OWA exposed to the Internet is entirely optional for Exchange on-prem... but impossible for avoid for a cloud service. Most possible ways to secure access to Exchange on-prem are built into your firewall, and you can configure at leisure... most possible ways to secure access to Office 365 are billable add-ons to your subscription. Any side channels to that... you just have to trust Microsoft...
I agree there are trade-offs, but especially for large enterprises with security teams, on-prem is definitely more secure.
> Putting surveillance video on the cloud is... kinda dumb
It's a necessity for some cases. I commented on this elsewhere, but for me it's having an eye on my elderly parents back yard in case of falls. Past history of not knowing is precisely why we got the camera.
Its still on the internet though. Its just the VPN becomes the security barrier. Although if you keep it up to date, its going to be safer than iot crap
People really should. It's pretty easy these days with wireguard or tailscale. Just plug a raspberry pi somewhere with network and power and you're set.
I spend my day doing the "tech shit" for work. I don't want to mess with stuff, I just want it to work. I'm willing to risk a bit of security for it to "just work", which it has now for multiple years. Without any security issues (so far).
I completely agree, even though I do tinker around with stuff for fun. However in my experience so far wireguard and tailscale (which is based on wireguard) are the kind of solutions that just work with little to no oversight.
Ouch! That's a bit harsh! Well, I suppose cloud archives could be encrypted? We use a cloud based system (but no cloud backup) and with multiple facilities, its nice to be able to launch the app and stream the camera's live feed using the cloud as a bounce.
Also with the cloud being used as the de-facto off-site backup location, most data these days will end-up (hopefully encrypted) in some kind of cloud service or another.
Remote access is fine, it can be reasonably managed: Generally the client sets it, and the manufacturer doesn't retain access. Usually you can geoblock and such as well. You have control of the hardware and can implement encrypted storage and better access controls, rather than trusting a third party to decide what is "secure".
Verkada has prioritized sales expansion over growing the engineering team, with 150% more salespeople than engineers and almost half the entire company in sales, per LinkedIn”
Exactly the sort of people you want to trust with highly sensitive surveillance.
There are actually plenty of reasons for remote storage of surveillance footage; not the least of which is that offsite storage of such video can be a regulatory or insurance requirement.
> Putting surveillance video on the cloud is... kinda dumb.
I tend to agree, but ...
If you're a small business or manufacturer, IT is a pain in the ass. The "cloud" is a benefit because you don't need to maintain any servers yourself and can just get on with your business.
The problem is that these companies don't face any consequence for claiming that they're secure and then not actually being ... you know ... secure.
If this stuff was simply encrypted at rest, that would have mitigated most of this breach.
I don't think it would, because they leaked admin credentials (unless it was encrypted with a customer key and inaccessible to admin/support)
Anyway, I think we're at the point with software/digital systems where food manufacturing was in 1900: there's been a gold-rush due to new technology, and we're reaching peak negligence in the pursuit of profit, and I think we'll soon get to a threshold where regulators will finally step in and lock down the wild-west and impose some real standards. Between SolarWinds, Exchange, and now this, it's to a point where it's dragging down our whole society. Something has to give.
I just had an aha moment reading your post. Food safety, drug safety, airline safety, all these regulations were implemented because manufacturers were operating with standards so lax that people suffered actual harm.
Perhaps that's the real reason behind the anti-FDA lobbying we are seeing here. The FDA demonstrates that government-mandated safety standards work. We can't have those in the IT field because they would diminish profits!
The Moderna and Biontech RNA vaccines got past the FDA in record time and are a massive innovation. What are you even talking about?
Also: Hepatitis C antivirals. One pill and you are cured, whereas in the past you'd be looking at a liver transplant. If that's not innovation it's impossible to say what that would be.
Those boxes from Costco come with WiFi built in and a mobile app paired to it. The small business owner invariably sets up port forwarding so they can watch from home and leaves the default password because it doesn’t force you to change it. Not much better there either.
My point was less that those were good solutions, but that a company I contend "shouldn't exist" likely has a different customer segment, like Tesla and Cloudflare.
For anything small, the local drives are obvious theft targets.
For anything large or distributed, you’re probably going to spend less for better security with a cloud system.
It’s sensitive data, but probably in a second tier of sensitivity. The integrity of the on-prem system depends in your remote access security and operational practices. Solutions like this often really suck.
Assuming one size fits all is dumb as well. This could be handled much better but it won't be cheap and that's what everyone is trying to do, "cheap". Other things are "on the web" and much less easily hacked.
Pretty much all consumer security cameras do this by default nowadays... I had to disable it on mines and send it to my own server instead, as a backup.
Another take on this - I have google nest outdoor cameras. Recently, a car park company tried to claim I was in a car park for 7 hours. In reality, I had gone earlier in the day, come back home, and then gone back hours later - but their ANPR system must have had a glitch.
I was able to send them links to the video clips on the Nest site, with embedded timestamps.
If this was a local system there would be no way to prove I hadn't just faked the timestamps.
You not having to pay an unfair fee for parking one time hardly seems like a good tradeoff for massive surveillance overreach by truly incompetent companies but maybe that was one really expensive lot or something
Not in the UK - most free parking for shopping centres use ANPR cameras so that people who are going there for a few hours can do so, but people trying to be cheeky and park there all day to go elsewhere can't.
To be fair, it was a specific person with a set of other employees who were all punished for the incident. It wasn’t the company which includes the people were harassed.
The headline oversells it but your comment undersells it.
The specific person and set of other employees were "a group of men in leadership positions on the sales team", including the "sales director". When found out,
> Verkada CEO Filip Kaliszan gave employees in the Slack channel [i.e. those involved] a choice: leave the company or have their stock options reduced. All of them chose to stay and take the stock option cut, according to Vice. “I was shocked. To me that’s not just a fireable offense, that’s a career-ending offense,” one employee told IPVM.
The video event annotations or filenames described in the article sound like what you might get out of a company culture that allowed the previous behavior to go largely unpunished.
It was not clear to me if the Arizona prison (the customer) chose those filenames or Verkada did. I could see prison guards doing the same thing... archiving clips that they find entertaining and naming them inappropriately. Not defending Verkada, I just wish the article made the culprit more clear.
> Inside Arizona’s Graham County detention facility, which has 17 cameras, videos are given titles by the center’s staff and saved to a Verkada account.
Even if the CEO is unshameable, the investors can be. Here they are[0]. Remind them publicly they invest in a broken company and make their association toxic as and until they distance, disinvest and hold the company and its officers fully accountable.
I used AdBlockPlus to block their pixel-filter, and then when I went back in they served me this nice message: Access to this page has been denied because we believe you are using automation tools to browse the website.
Javascript is disabled or blocked by an extension (ad blockers for example)
Or maybe companies run by sleazy people have less than stellar infosec/netsec practices, or are prone to sweeping gaping security holes under the rug rather than fixing them, which will inevitably result in something like this.
It's talked a bit more downthread, but I mean if the sales director is making public slack channels featuring female employees alongside explicit jokes, then long-term thinking may not be a strongly selected for attribute at the company.
This is amazing. If they would have added software to the cameras to mine bitcoin, it would’ve been absolute peak cyber punk.
People that sell video cameras attached to the Internet should always have a disclaimer that the user should assume that the system will probably be accessible by anyone at some point in the future.
I believe Samsung started doing this with their TVs in regards to audio.
We are certainly building ourselves an interesting future.
I'm running a Shadowrun campaign where the hacker of the group frequently hacks security cameras. But it's always just a single camera close to where he is. Clearly it would be more realistic if he hacked all cameras of a single company all over the world simultaneously.
I'm going to stick with my current game balance, though.
Distributed computing. Customer's electricity is free for you, so whatever you manage to mine, is pure profit (and invisible to IRS if you're clever). Making this work would also be an interesting Big Data Hyper Edge Cloud Computing project to keep the engineers occupied and further justify the need for the money they got from investors.
Except there are no coins that could be mined with a camera CPU. Even with Monero (which is CPU based) you will not meet the RAM requirements or clock-in a "share" of work in any useful timespan.
You statement is true in the wider sense, you can have mining botnets of computers and maybe some high-end phones, but IP cameras are really-really weak as far as compute power go.
This isn’t true, you could still mine (poorly) with reduced RAM by recomputing as needed. And I’m not sure what you mean by “clocking-in” a share in a useful timespan. Compromised devices aren’t people, so they don’t care if a particular machine can’t get regular payouts. If the whole network can reliably find a low diff share solutions anywhere it can make money.
It would be horribly inefficient however a network of tens or hundreds of thousands of low resource units can absolutely make money since the costs are zero.
To run argon2 (monero) you would need a significant amount of time to compute even one hash that means that the synchronization overhead of your workers would trump any "hashing speed" an IP camera (even a good 4K one) would have. By the time you get any amount of work done, the block will be processed by a regular miner and you will not get a payout. Same goes for a pool, doing any "share" of work would take too long and you would be out of sync. Hell given a bad enough ping I get refused shares on a Ryzen 3800X.
Also on top of all of that most IP cameras are designed with tightly defined specs. Trying to run a miner on them would absolutely requires stopping the video feed which would make the "attack" (which is still useless) easy to detect.
You could run a SHA256 hash of random values and send it to a database in hopes of mining a Bitcoin block with it one day when the input (block hash) happens to match. It's only 2^256 times less efficient than standard mining techniques.
Depends on what hardware they put in their cameras? I assume they aren't just using random IP cams, but deploying their own design? They could sneak in a cryptocoin ASIC for good measure.
Of course they most likely aren't doing that, but it's not out of the realm of possibility (and thus a missed opportunity to make this debacle peak cyberpunk).
Now that would be funny. However, would this not be fairly obvious to one of those sites doing tear downs? Then again, if found, would the PR one of those tear down sites could generate slow the sales down enough to make it a losing prospect for the manufacturer?
Non-techy people are willing to look so so far the other way in trade of convenience. Showing them "facts" about how much electricity their cameras are using would not impact the vast majority of the users if they feel like the service they are receiving is good enough.
Do we know that they don't? A dedicated ASIC would probably be spotted by some random person doing a teardown. But a DSP chip doubling as crypto miner when not in use, that could easily escape post-purchase hardware inspection.
(Might be easier to discover it on the design/procurement/supply chain side. Perhaps stock players would find out when investigating their investments. A high-profile brand putting crypto miners on consumer hardware is newsworthy, and news move stocks, so there's incentive to leak the story.)
If they can do god tier supply chain attack, dissipate the heat from mining hardware, add 4GB of RAM and put it in a form-factor that looks reasonable for a camera while still turning a profit, they can dedicate their genius to legitimate business ventures instead.
Most security cameras are PoE powered which gets you a whopping 13W of power that also has to run a sensor, IR LEDs, heaters, lens, RAM, etc. before you even choose the SoC and potentially your ASIC. How much computing power do you really think you can get with that?
So comparable to a smartphone under load. That's a pretty good power budget, given that most of the components you mentioned (except RAM/SoC) are going to be operated intermittently.
> sensor
Serious question: you mean image post-processing and encoding here? I thought CMOS sensors themselves have negligible power demands?
> heaters
A crypto miner is just that - an overcomplicated heater. If you include one, you don't need the other.
> How much computing power do you really think you can get with that?
Not much, though an ASIC would get a nice multiplier on it. I'd guesstimate it to be comparable to a cryptominer running in the browser of an unsuspecting regular person - who will typically have a cheap, underpowered machine that can barely lift the OS and the browser without hanging. In recent years, at least some people thought you can make a profit with it, because we've seen such web cryptominers deployed around the web.
My point isn't that it's happening - I fully expect the cost of sourcing a miner ASIC and redesigning a camera around it to be much larger than the mining it would bring in any reasonable timeframe. But I think that on first look, it's possible, if someone tried hard enough, to build a sneaky security camera like this, that would be able to eventually yield some profit if deployed wide enough.
You’re right, partially. In the sense that to mine bitcoin/etc you need something more powerful... but you could just create a new type of crypto to be mined specifically in low power/IoT devices. Something like [1]
This is the important part. NOT that their network was not secured, but that anyone with a super-user account can simply view archived and live video feeds of any of their customers????
> The hackers’ methods were unsophisticated: they gained access to Verkada through a “Super Admin” account, allowing them to peer into the cameras of all of its customers. Kottmann says they found a user name and password for an administrator account publicly exposed on the internet.
This really struck me also. I work in the relevant industry (we make cameras etc.) and there is always a bit of pain to get user footage. This is how it should be! To have everything from source code to customer material accessible to an admin is bottom-tier thinking. Why not just rename your "admin" to "GOD" and then ask yourself if you have any single point of failure?!
I do NOT want to sound smug, but there is a little bit of amateur hour going on here both from buyer and seller. High value and large targets (like airports) and more established sellers usually don't work like this, and that's for a reason.
It's not just that it's a single point of failure, it's that as a customer I do not want any admin who is feeling curious to be able to snoop on my footage with a click.
I don't know how "established" this company is, but their customers appear to include city governments, hospitals, and Tesla motors, which I would consider "high value and large targets".
Makes me suspicious of the whole industry. If others in the industry dont' want that, time for some industry codes and audits and self-regulation.
I've added a link to IPVM to the parent to my comment that might interest you!
Regarding established: I might be wrong! I willingly admit that I knew nothing about verkada some days ago. Seems to be relatively new (5 year-ish) and "classic" Silicon Valley in that they push hard for growth to get their valuation up and try to "disrupt" by running everything in the cloud. More sales people than R&D, which I think is uncommon.
Verkada runs full lock-in, so if you buy a camera from them you have to buy their services. This is again relatively uncommon. Most of the industry supports the ONVIF standard, so you can run the hardware you bought with different software solutions. If you want encryption at rest, no problem. You just make an on-premise solution with full encryption. With verkada you can't do that (incidentally verkada have mocked ONVIF due to alleged security concerns, but obviously it undermines their business model with full lock-in).
Since combining verkada and other hardware would require parallel systems I made an educated guess that most customers would be places without previous hardware and/or less concern for the long run. Most large and high value targets have previous hardware, but certainly there are exceptions. And as stated earlier, I might be wrong:)
And lastly, you should be suspicious! Last time I bought a car I was very suspicious. I like the car I did buy very much, but next time I will be just as suspicious again. That's how things should be when it's about trust and high impact.
Is that HIPAA-compliant? (Not sarcasm: I don’t know HIPAA rules enough to assess myself in the cloud-vendor / medical-institution scenarios we’re seeing here.)
“We did not exploit any flaws or vulnerabilities. The cameras have a built-in maintenance backdoor, which allows anyone with super admin privileges to access a root shell on any camera of any customer at the click of a button.”
A while ago there was a post on Reddit about something similar, and there wasn't even any "hacking" going on; the video feeds were just unsecured.
Using Google, someone search for a proprietary video protocol (IIRC) and found tons of video streams that weren't even password protected. Some in schools, some in warehouses, and some just on the street as part of neighborhood surveillance. I think I have the link saved, I'll look for it.
Finding unprotected streams via Google Dorking like this is easy. Here's an article that doesn't cover this particular use case, but rather the broader practice:
I've personally dabbled with it a bit in the past, and while I didn't find anything particularly interesting, it did make me a bit more cautious about enabling anyone with a link to access a Google Doc. With a good enough scraper or even just a lot of patience, there are a lot (potentially sensitive) data out there for people to harvest. That's not to say there aren't a number of benefits to having access to advanced search tools though, just that individual mindfulness when making something completely open for anyone to access is all the more important.
Its horrible how common this is. I used to do work on local business sites and the security was horrific. Pages that contain sensitive data or even CRM management pages exposed to the public internet with no password at all. On some of the less sensitive ones I had a look I found details of family members in these exposed sites.
Not only that, but it was all horribly outdated. Seen some things running on rails 1 pre release on a debian server about 6 years passed end of life.
> Last year, the sales director accessed these cameras to take photos of female workers, then posted them in a Slack channel called #RawVerkadawgz alongside sexually explicit jokes. The incident was first reported by IPVM and independently verified by Vice.
damn that is despicable but this sort of brogrammer behavior appears rampant. How would you address this as a manager? This is absolutely not okay.
It could be a programmer. I think the origin comes from bro-y men around 2005 moving to silicon valley (eg zuck in that movie) but then having a very frat-culture vibe (eg. lots of booze, house parties but with programming, etc). It grew to be anyone silicon-valley and tech-adjacent that acted like this.
Not what they did, mind you. The culprits merely got their stock options reduced. And the fact that the sales manager could access the camera feed may have been a big hint that security was not their biggest priority.
You're right, it "appears" rampant with quotes. It really isn't. The world is a big place and such solacious news because almost instantly popular (because internet) even though it's a 1 in a million thing.
The sad part is the majority doesn't know how and why they should do this. They just get their personal lives broadcast to the web for all to see. There was or at least used to be a subreddit showing the more interesting exposed cams.
How likely is it that for someone who isn't technically as adept as a team at a (good) cloud NVR provider, the security is actually worse? Bit like rolling your own crypto.
> The hackers gained access to Verkada through a “Super Admin” account, allowing them to peer into the cameras of all of its customers. They found the user name and password of the account publicly exposed on the internet
Don't put your personal stuff directly on the internet, use separate admin accounts with 2FA, gg, you're doing better than Verkada - comically valued at $1.6B
Each of these security camera system/IoT companies are only offering you a bit of hardware to interact with their true purpose of making you a user of their SaaS product.
Another reason (the address space one is good, layers defense if the camera "escapes" your network restrictions somehow) is that cameras that WORK on IPv6 are usually a level of quality just above "absolute crap". (Note that many ADVERTISE IPv6 but don't actually work on that.)
Its interesting that you can and people have scanned the entire ipv4 space in minutes on a fast VPS but it would take you forever to find your first ipv6 address that is even in use.
Shodan - a search tool for such things - had a pretty incredible approach to attacking this - they quietly provided most of the ipv6 hosts to pool.ntp.org, which is widely used as a linux default - and would queue a portscan up for any device that connected at startup for timesync.
Ingenious - and all the more reason to run NTP on the border router/firewall and have that be the only device that communicates with the outside world.
D-Link sells cameras that don’t have cloud access. Look up a DCS-913L. I bought one on eBay around a year ago. It has wifi and you can set it up to record over SMB.
It’s not the best image quality but if you get another router, a raspberry pi and a large external HDD you can have a relatively cheap CCTV set up.
I'd like to add a bit of context to how security cameras most often are installed.
In the industry in general you have producers of the equipment and you have buyers, but in between there you have integrators. The integrators plays a crucial role when installing big systems. They win the bid for an installation and carries out the work. This means that there is seldomly any direct path between camera producer and the customer. For the producer to get access to footage they must go through the integrator, so the friction is non-trivial.
Direct contact producer <=> buyer might happen in the small case, like a store with a single camera or you placing one at home.
My guess (!) is that verkada tries to pry away the integrators with a simpler model for installation.
Most larger producers now have cloud offerings, which could have some similar vulnerabilities to those mentioned in the article. However, my impression is that security is taken VERY seriously. Not just lipservice, but in practice. This makes sense as it is a key selling point and the larger buyers are competent judges of this. This is in stark contrast to the "typical" hacked target, which seems to be autoshops and hospitals (I am generalising to get through a point, I am not sure what the most common victim is).
In the U.S. legal system, is video footage exposed in this manner admissible in court?
E.g., if a prison inmate was physically abused by staff, and his only corroborating evidence would be this video footage, can it be used to justify a civil or criminal complaint?
I think so? Law enforcement can't break into something without a warrant, but if someone brings them evidence of wrongdoing (especially a third party who didn't conduct the hack themselves), I believe they can act on it.
Under oath, presumably you compel the people in the footage. But I would imagine that the footage of questionable sourcing might count as the probable cause to go in and get the footage directly via warrant?
Whoever at Verkada gets subpoenaed by the inmate's counsel to do so, I'd assume. That said, I'm no more a lawyer than anyone else who's commented here so far.
IANAL, but from all of the cop shows on TV, if the lawyer was provided the evidence and did not break laws themself OR as long as the lawyer did not entice someone else to commit the offense or provide instructions on what to do, then typically the judge allows the evidence.
- Pushes down wages for other workers doing similar labor, by taking demand out from the normal market
- Because the prison itself (or, more likely, the prison-industrial-complex corporation running it) is seeing a huge profit from these ventures, it provides a perverse incentive to keep their labor pool 'strong', be it through lobbying for harsher sentences, or encouraging shot quotas amongst their guards/COs.
- Conditions prisoners towards working less for equal work (we shouldn't encourage the idea that -any- class, race, or creed of human being is worth less for the same amount of work)
No its not, because that work generate value for companies at a very low cost. So it both builds a system that pushes for more people in prison and displaces workers who would have been paid higher.
Prisoners should be given some kind of work / study / something. But it should't generate value for anyone but the prisoner or perhaps lower costs for the prison (cleaning/etc)
I'd like to see an overview of everything that has been hacked so far, arranged by both device/protocol and industry/gov/social structure. It would be interesting to see if there are any categories that have not yet been hacked and what their characteristics are and where in society they reside. Maybe secure internal networks at the DoD have not yet been hacked? (How would we know, aren't those networks the ones the state attackers really REALLY want to attack? Who would tell us if they'd been compromised? There have been some news reports on the use of insecure drone control wireless protocols.) Crypto protocols used for chemical plant SCADA? Have parts of Starlink been hacked yet? Which banking protocols and hardware security modules have been hacked and which have not (SWIFT? HSM's based on ASC X9 standards?) Might give us some clue as to what actually works, what needs to be abandoned, and what needs changes. At the moment this looks like a losing battle (possibly a loss of civilization?) with the number of big data thefts and compromises if something does not change. Does anyone know if such a comprehensive review exists in the literature?
Note to HN. I know personally the attackers, and they send me proof of the attack before making it public. They are just two south america teens having fun cause they can't leave home. They were doing just for the lolz, there is no complex supply chain attack or state-actors involved. Just two kids pwning billions of VC money.
The person behind this claimed to have root shells on the networks of both Cloudflare and Okta. (FWIW, if the network is well segmented, this may have limited impact.)
Well, that looks bad. Thank goodness, though, that after a security lapse this awful, corporate and government bureaucracies will come to their senses and stop outsourcing critical functions to cloud companies they don't know all that much about. Oh, wait...
Why would the footage not be E2E encrypted? Hospitals and police stations are installing cameras which store unencrypted footage remotely?? What is this madness
End-to-end between the cloud provider and the cameras at the site, sure. But it's stored unencrypted, mostly because cloud providers provide the web interfaces and such to stream the video, provide services to analyze and classify the video, etc.
Nobody encrypts their surveillance video storage, AFAIK.
> End-to-end between the cloud provider and the cameras at the site, sure
That's TLS, not E2EE. E2EE means the provider never sees the unencrypted data. (Ex SMS is unencrypted, most internet services use TLS these days, Matrix and Signal use E2EE.)
> Nobody encrypts their surveillance video storage, AFAIK.
What performance implication?! AES-NI has been standard for mainstream x86 hardware since ~2013 and ARMv8 introduced optional crypto instructions in 2011!
The issue is that manufacturers choose the cheapest possible SoC that lacks these abilities. Given the small cost savings and importance of basic security measures, that really needs to change.
(A quick look at Amazon shows many of the top sellers advertising various "AI" features and streaming video at 4K or better. Given their apparent capabilities, I strongly suspect that such SoCs do in fact have hardware support for crypto.)
I think anyone who has had to comply with those compliance lists knows how useless they are. Easy to make the minimum change that makes you compliant without being any more secure.
And when you try to describe the problem with cloud connected "security" devices to your non techie neighbour, all you get is a "whaaa...?" expression.
Sequoia is mentioned prominently here. What is the role of venture funds in ensuring their startups operate or endure some basic regular external security audits?
I wonder about Cloudflare. It seems like the Windows Vista of its genre. It’s big, pretty, and possibly doomed to be replaced.
Tesla is fine. It may as well have been a publicity stunt for them. “For a limited time, you can tour the Tesla facility, but please don’t. (wink wink!)”
Yeah, I wonder if the hackers had the same feed used to generate Cloudflare's randomness... that could be a vastly bigger security breach on top of this one.
If their blog posts are to be believed [0], lava lamps are not the only source of entropy available.
> Hopefully, the primary sources of randomness used by our production servers will remain secure, and LavaRand will serve little purpose beyond adding some flair to our office. But if it turns out that we’re wrong, and that our randomness sources in production are actually flawed, then LavaRand will be our hedge, making it just a little bit harder to hack Cloudflare.
I own reolink and amcrest cameras. I put them in a vlan with no outside connectivity (and frankly no inside either!). They try to call home constantly :-(
In my experience - no. I've seen devices sold for thousands of dollars whose login screen could be bypassed with a magic cookie. Here and there you will find a device that will force you to change the default password upon activation, but this is as far as it goes and the cheaper models from the same vendor will be as shitty as anybody else's.
The best that you can hope in consumer devices is something like Apple, Google or Amazon, because they can afford the support costs, but it comes at the price of privacy and funneling your money in many other ways. Enterprise stuff could be found, but you can never trust just the brand for every model.
Back in college I was studying security and eventually I found out about shodan. I mean, when people don't even care to put passwords on their connected to the internet device, can you even call it hacking?
You hardly need to be a hacker for this. I did some Google dorks out of boredom. In 15 min, I saw live feed from some CCTVs exposed to public internet. The most disturbing one was someone's living room...
"Kottmann said their reasons for hacking are “lots of curiosity, fighting for freedom of information and against intellectual property, a huge dose of anti-capitalism, a hint of anarchism -- and it’s also just too much fun not to do it.”"
....
"Kottmann says they found a user name and password for an administrator account publicly exposed on the internet"
Excuse me but finding a password that some idiot included in their public git project is not fucking hacking.
Hilarious. Software development at Verkada is filled with "non-traditional backgrounds" leftist SJW types that spew neurotic delusional beliefs on Slack all day.
I bet management wishes they would have hired some real devs with backgrounds in software development and security. You reap what you sow.
These folks (the hacker group) are a hoot to follow on Twitter, I do recommend searching for and finding them there. Hacker demons, the lot of them. :)
The solution to this hack is simple: Shut this company down, because it's a bad idea.