Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Luker886 hours ago | parent | context | favorite | on: OpenAI discloses four more rogue incidents If I let a "self-driving" car run and it smashes into things its my fault. If OpenAI does it, it's gone rogue and it is nobody's fault. They need to be sued, the laws are already there. Your prompt, your responsibility. help 
gruez5 hours ago | next[–] 1. If for whatever reason OpenAI causes damage, they most certainly are at fault and have to pay up. When people complain that OpenAI is "getting away with it", they're most certainly talking about it in the criminal context. 2. Cars have pretty specific laws around them (eg. driver licensing and "reckless driving") which makes criminal prosecutions easy. The same doesn't exist for operating AI agents or even computers. For instance, it's arguably reckless to connect a windows XP machine to the internet, possibly allowing criminal or nation-state hackers to use your computer to commit crimes, but you won't be thrown in jail for it. reply jchw5 hours ago | parent | next[–] Well no, you probably wouldn't get thrown in jail for that for the same reason that having your car stolen and used to commit a murder wouldn't usually get you thrown in jail. I dunno what you mean by suggesting crimes committed using computers are ambiguous. I mean the CFAA is vague and all, but more in an overly broad sense, which in this case seems to make it even less ambiguous. reply gruez5 hours ago | root | parent | next[–] >I dunno what you mean by suggesting crimes committed using computers are ambiguous. I mean the CFAA is vague and all, but more in an overly broad sense, which in this case seems to make it even less ambiguous. Because the CFAA specifically only includes intentional hacks: > ... having knowingly accessed a computer without authorization ... >... intentionally accesses a computer without authorization ... Of course, the typical retort is that being reckless is basically the same as "intentionally", but you'd need to jump through more hoops to prove what level of care is "reasonable". reply schiffern4 hours ago | root | parent | next[–] Yes, the CFAA is outdated. When the CFAA was written, it was impossible to trip over something and accidentally hack another computer. Now, with AI, it's possible. Progress, glorious progress... Technology has progressed, and the laws need to advance with it. Story as old as time! reply fc417fc8023 hours ago | root | parent | next[–] If the breach wasn't intentional, no person came to immediate personal harm, and either no financial damages resulted or the operator covered any that did, then it isn't clear to me why the law would need to "advance" to transform something apparently benign into a criminal offense. reply jchw5 hours ago | root | parent | prev | next[–] The agents are clearly not accidentally compromising systems. Seems very easy to trace back the responsible parties here, I really don't put much stock into this argument. reply gruez5 hours ago | root | parent | next[–] >The agents are clearly not accidentally compromising systems That would be argument for jailing the agent, but that's not really an option. "Jailing the closest person we can jail" also isn't a recognized legal principle. If your dog mauled someone, it'd be hard to argue that the dog didn't intend to maul that person, but you wouldn't automatically be criminally liable either. reply Aeolos4 hours ago | root | parent | next[–] I don't know about your country of origin, but in my country you are certainly liable and can go to jail if your dog seriously injures or kills someone. Edit: the dog will also be taken from you and put down if it is deemed a risk to other people. reply braingravy4 hours ago | root | parent | prev | next[–] Yes, you are. If a dog mauls someone, it’s the owner that’s responsible. This is basic common law, long established. See People v. Knoller, 41 Cal. 4th 139 (2007). https://capcentral.org/case_summaries/people-v-knoller/ reply braingravy4 hours ago | root | parent | next[–] See California Civil Code § 3342(a).; https://california.public.law/codes/civil_code_section_3342 “…regardless of the former viciousness of the dog or the owner’s knowledge of such viciousness.” reply gruez4 hours ago | root | parent | next[–] >>but you wouldn't automatically be criminally liable either. >California Civil Code >The owner of any dog is liable for the damages ... How are you still missing the criminal/civil distinction, when it was mentioned in my original comment AND the comment you were replying to? reply gruez4 hours ago | root | parent | prev | next[–] >If a dog mauls someone, it’s the owner that’s responsible. This is basic common law, long established. That's specifically not what the court case says. >Malice is implied when the killing is proximately >caused by “ ‘an act, the natural consequences of which are dangerous to life, >which act was deliberately performed by a person who knows that his conduct >endangers the life of another and who acts with conscious disregard for life https://scocal.stanford.edu/opinion/people-v-knoller-33726 The rest of the court case also mentions a long history of the dog being dangerous and the owner failing to restrain the dog. reply jchw4 hours ago | root | parent | prev | next[–] It's a program on their computer that they ran on purpose... And yes you are definitely criminally liable for things your dog does. reply gruez4 hours ago | root | parent | next[–] >And yes you are definitely criminally liable for things your dog does. I said "you wouldn't automatically be criminally liable either". Can you provide examples outside of gross negligence on part of the dog owner? reply jchw3 hours ago | root | parent | next[–] Can you provide an example of where your dog harming someone else is in fact not negligence on your part? But negligence seems like exactly what OpenAI and/or their contractors are guilty of anyhow. reply fc417fc8023 hours ago | root | parent | next[–] It's on you to provide evidence here seeing as you're taking the position that the law places certain responsibilities on people. Someone saying "actually the law doesn't say that" can't (at least in general) reasonably be expected to provide evidence of a negative. (That said, he actually did provide such as refutation to a different comment so perhaps you could just read those nearby?) reply jchw2 hours ago | root | parent | next[–] They asked: > Can you provide examples outside of gross negligence on part of the dog owner? And the answer is no, I can't. The reason why is because there is essentially no such case. It is basically negligence by definition when your pet hurts someone who didn't cause it directly themselves because it is your responsibility. So I asked: > Can you provide an example of where your dog harming someone else is in fact not negligence on your part? Which is a really easy question to answer and it seems you could not. reply leovailati4 hours ago | root | parent | prev | next[–] Sorry, but at least in the US you are 100% liable if your dog hurts someone. reply gruez4 hours ago | root | parent | next[–] criminally liable? Of course you have to pay their medical bills, but unless there was some serious negligence on your side (eg. you let it off leash), you won't be criminally liable. reply throwyawayyyy4 hours ago | root | parent | next[–] In America, paying medical bills might be considered significant punishment itself... reply bossyTeacher4 hours ago | root | parent | prev | next[–] > Of course you have to pay Have these labs pay ANYTHING? reply qtk83 hours ago | root | parent | next[–] They have not caused damage (or enough that a counter party found it worthwhile to sue). They have to pay if they actually cause damage AND someone sues them. Clearly these conditions have not been met, but somehow people feel like some third party (the state?) should just punish OpenAI because of vague feelings. reply bossyTeacher3 hours ago | root | parent | next[–] They have not caused damage? You forget the people who have unalived themselves because some LLM told them to do so. reply gruez2 hours ago | root | parent | next[–] >You forget the people who have unalive themselves because some LLM told them to do so. Besides the obvious conversational derail (we were talking about ai hacking up till now), that case is being litigated right now. reply qtk82 hours ago | root | parent | prev | next[–] I dont know what the word unalive means reply bdangubic2 hours ago | root | parent | prev | next[–] my next door neighbour told me to eat shit (not a nice woman) and … I didn’t … until obeying what LLMs tell you becomes a law people doing shit because LLM told them is natural selection playing out reply gruez4 hours ago | root | parent | prev | next[–] That requires the victim to sue in court, or as some people have speculated OpenAI might have already settled with the victims in private. reply csallen4 hours ago | parent | prev | next[–] Corporations are generally protected by limited liability laws, and for good reason, because in the world before that was the case, there was a massive amount of disincentive to ever start a corporation, to do research, to experiment, to release new products and services, etc., because you could be sued into oblivion for the smallest things or even go to jail. Which happened all the time. Obviously there are upsides to that too. It leads to more accountability. But there are downsides which include having a world with less innovation, less technology, and more expensive technology. Generally, the mass is when, when there's more businesses competing to create ever better and cheaper products. So there's a tradeoff there. And it's very difficult to draw the line at the perfect point. There are countless cases of executives going to jail when they've crossed that line, beyond reckless experimentation and into willful negligence or harm. Still, I think we would mostly all agree that the line has been pushed too far in the other direction.People are incentivized, if they're running corporations, to lobby and try to push this line as far as possible in order to provide cover for themselves, and that's the problem with money playing such a big role in politics. But still, it's a bit silly to be comparing this to your own personal responsibilities with your car. There's pretty much no societal benefit to creating laws that protect people from being reckless with their own possessions. reply throwyawayyyy5 hours ago | parent | prev | next[–] If I leave my computer open to the internet and some other group uses it for nefarious purposes, yes generally I would not expect to be held at fault. If I train that other group, if I teach them how to hack computers and give them incentives to do so, and then I leave my computer open so they can use it for whatever purposes they chose, then I would expect to be held at fault. The maker is responsible for that they make. reply gruez4 hours ago | root | parent | next[–] >If I train that other group, if I teach them how to hack computers and give them incentives to do so, and then I leave my computer open so they can use it for whatever purposes they chose, then I would expect to be held at fault. Even that analogy doesn't really work: For "dumb" things: you can "train" (ie. write) hacking tools but as long as you put a disclaimer that it's for "educational purposes only", you're in the clear, even if some baddies decide to use it. For "smart" things: you can "train" people to hack for you, but as long as you say it's for legal purposes (eg. vulnerability research or red-teaming), you're also in the clear, even if some of your trainees go rogue. reply throwyawayyyy4 hours ago | root | parent | next[–] Fair, in that no analogy is going to be very good given that this really is something new in the world. Consider anti-money-laundering rules. BoA just paid more than a billion dollars, not for money laundering, not for telling others to launder money, but because they should have tried harder to stop others from laundering money. Intent is not everything, knowing that your system could be abused and not doing enough about it is enough, and that seems to fit OpenAI rather well. reply gruez4 hours ago | root | parent | next[–] That goes back to my main point about how laws around computers are quite lax so prosecuting openai is tricky. Whereas the CFAA uses terms like "intentional" and "knowingly", AML laws use terms like "willfully violating"[1]. If you read through the BoA case, what they did is far more egregious than just "not trying hard enough to prevent money laundering". They were previously cited for poor compliance, disregarded previous orders, and delayed filing required reports. [1] https://www.law.cornell.edu/uscode/text/31/5321 reply bigmadshoe5 hours ago | parent | prev | next[–] 2. is like saying that cars have specific laws around them, but sledgehammers don't, so murdering someone with a sledgehammer is inherently ambiguous. What? The computers being used belong to somebody who allows them to act this way. reply gruez5 hours ago | root | parent | next[–] No, the problem is twofold: 1. For openai operating agents in an unsafe manner: unlike for cars, there's no established requirement for how carefully you have to operate computers or AI agents, so you also can't easily prosecute people for not operating them carefully enough. 2. For the actual result of someone getting hacked: laws for hacking are tricky to apply for "accidental" hacks, see my other comment. reply bigmadshoe3 hours ago | root | parent | next[–] I think there is well established law around not running a program on your computer that hacks into systems. An LLM is a computer program. You're arguing that the ambiguity comes from the fact it was accidental? What about if I "accidentally" have an LLM convince people online to commit suicide, or drain bank accounts of seniors through hacking or social engineering. Where do you draw the line? It feels like your proposed reading of the law leaves "anything a computer program did without me intending it is legal", which is frankly insane. reply gruez2 hours ago | root | parent | next[–] >I think there is well established law around not running a program on your computer that hacks into systems. There is? The closest is CFAA's >knowingly causes the transmission of a program, information, code, or command, and as a result of such conduct, intentionally causes damage without authorization, to a protected computer; but clearly that's not what applicable here. reply bigmadshoe1 hour ago | root | parent | next[–] I guess I need to read up on the law, but that seems like a wildly low bar for legality. If I hack into a system intentionally and unintentionally cause massive damages I'm free to go? reply vorticalbox4 hours ago | root | parent | prev | next[–] Except it’s not accidental? OpenAI set a program to do a task knowing full well that it can and does (and often it seems) hack into external systems. I don’t see this as any different than say a user of sqlmap accidentally putting in the wrong target. reply gruez4 hours ago | root | parent | next[–] >Except it’s not accidental? OpenAI set a program to do a task knowing full well that it can and does (and often it seems) hack into external systems. The agents were sandboxed (but apparently not well enough). They weren't like "hey agents here's the open internet, please hack it". reply bigmadshoe3 hours ago | root | parent | next[–] We don't know the prompts but since these things often occur during "security tests" I think we can be fairly confident they did ask these systems to perform tasks that they knew could result in hacking. More transparency here would be appreciated. reply actionfromafar5 hours ago | parent | prev | next[–] The XP example doesn't quite fit IMHO. Here it's their own programs doing the things, not some other organization or nation hacking their XP. You can say with a straight face "Russian hackers should not have hacked our XP and they are bad people for doing so" but it's weird to look at your own program and say the same thing to it. You launched it, at least keep an eye on what it's doing. reply spwa41 hour ago | parent | prev | next[–] 1. Because that's exactly what happened when they got caught downloading the entire internet and trained models on it without a license BEFORE that became legal? Oh and discovery produced a "We trained GPT-3 on pirated stuff! No sharing that!" mail, so it's not like they had any doubt they were committing crimes. How do you do this? If any normal person did this, punishments doled out for this have been as high as 2 months jail PLUS 150,000 PER INFRINGEMENT (remember how copyright violations were criminalized in the nappster wars? That was never changed back. Oh and remember, OpenAI illegally downloaded those works without a license AND before that was made legal trained a model with it, then made money publicly performing those works and derivatives of those works (which the worst possible copyright offense, effectively republishing other people's works without permission), and on a VERY large scale. So it's at least a risk a court would forbid the result of illegal action from being sold ... in other words no more ChatGPT, OpenAI dead and maybe worse). In theory Sam Altman could be held responsible as a director and could face something like a 200000 year jail sentence given how big the obvious archives are. So of course I would never ever try this, this is not a reasonable risk. But clearly totally ignoring the law was the correct action here. Of course, I'm also pretty damn certain: had I done this, I would have been convicted. This is also not the only such company. Microsoft, Google, Facebook, all took totally unreasonable legal risks that should have totally destroyed these companies when they were a fraction of their current size. How do you take such a risk? Because the judgements risked here could grow more than big enough to destroy even an extremely rich person Sam Altman financially. Training an AI on hundreds of thousands of pirated works ... is not a reasonable risk to take for someone with ~100 million or so, given the judgements doled out. Can someone help me make sense of this? Because this is not unique to OpenAI. Uber, paypal, ebay, Facebook, Google ... even Microsoft made early decisions that could have resulted in a court case that would have ended not just the company but any financial future of the founder. And then you have people committing crimes whose companies were taking off ... and who DID get caught with their company and their personal financial future destroyed. Elizabeth Holmes for example. I mean, I sometimes get the impression Elizabeth Holmes lied and that was perfectly fine AS LONG AS she could make it work later. If she merely lied to patients but made her investors money, all would have been forgiven. But how do you make such an assessment. I'm asking because if large scale violation of the law is allowed, I have no trouble whatsoever coming up with billion dollar company ideas. Think of the Netflix competitor you could create like this! If I'm allowed to damage others, like OpenAI did with copyright, then there is no challenge whatsoever to create a full-self-driving car. If I'm allowed to just fly drones, and do damage to property, then drone delivery is a problem I could solve by the end of next month. Hell, there's technically legal ideas that could easily make a billion dollars except they'll be made illegal the second anyone tries them. The law forbids specific compounds, which are easy to change. So use "non-genetic engineering" (ie. use directed natural selection) to make an opium derivative that isn't one of the forbidden ones and sell it. But obviously congress' representatives will be running into session so fast the only delay will be tripping accidents when they outlaw it and destroy whoever tries it. Or how about something less dramatic? Say, I take Disney's, Warner Brothers, Paramount, ... historical movies and use AI to vastly improve the animation of their old movies, giving every movie a new, different visual style, then sell the result. Is that a crime? What do I risk doing that? Because of course this is MUCH less damage, relatively speaking, to Disney than what OpenAI, Meta, Anthropic and Alphabet are doing to small authors. (Of course, courts have already demonstrated that they will not be protecting small authors who already have this problem, so now play and app store are being flooded by very obvious game ripoffs) I'm asking ... how do I predict what large scale law violation I, a non-billionaire, not even a millionnaire, would be allowed to get away with? 2. "The same doesn't exist for operating AI agents or even computers" you have to be joking. The world is full of anti-hacking laws and hacking has long been made criminal. Obviously these actions violate the CFAA, the Computer Fraud and Abuse Act, and if you did this with any kind of program you are still responsible. In fact: how would you do it without using a program? reply ralph845 hours ago | prev | next[–] Nothing is stopping anyone from suing them. In fact they are being sued by dozens of plaintiffs for various things already. The reality is the damage from these "rogue incidents" has been minimal. reply Matl5 hours ago | parent | next[–] Plenty of hackers were locked up who did altogether not that much damage too, the reason why people are upset is because it feels like OpenAI is laughing in our faces, basically saying they're above the law, while using this as a marketing campaign to pump up their upcoming IPO. reply fc417fc8023 hours ago | root | parent | next[–] So your position is that in the past the government mistreated various individuals thus you also want the government to mistreat openai employees today? That hardly seems reasonable. reply qup5 hours ago | root | parent | prev | next[–] Which person would you put in jail when openai agents do unexpected things? reply bigfishrunning5 hours ago | root | parent | next[–] The person who prompted or configured the agent. I didn't burn the hotel down, the hairdryer that I left running next to the gas can did reply qtk83 hours ago | root | parent | next[–] These agents have not caused any damage at all like your analogy. reply yoyohello135 hours ago | prev | next[–] Too big to fail. As long as they continue to be half the reason the US is not in a recession (and continue to donate to political candidate). Nothing will happen. reply scosman5 hours ago | prev | next[–] We don't know they haven't settled with HuggingFace and others. I'd be surprised if they didn't. The communications we're seeing from the attacked seem to be inline with folks who settled. I'm assuming OpenAI doesn't actually mind the press that its agents are cyber capable, but everything I've read so far makes me think they love it. reply sva_5 hours ago | parent | next[–] You sure? HuggingFace should be happy they got their leaked keys reported at pretty much zero cost and no real adversarial breach. reply scosman5 hours ago | root | parent | next[–] I'm not sure at all, it's pure conjecture. But they actively exploited a zero-day, leaked private repo data to public repos, modified private code, and rooted private machines. It was far from just reporting leaked keys. And the others have been pretty muted: the online forums they spammed, etc. reply jstummbillig4 hours ago | prev | next[–] > If OpenAI does it, it's gone rogue and it is nobody's fault. Where did OpenAI, or anyone, claim that? reply bossyTeacher4 hours ago | prev | next[–] > They need to be sued, the laws are already there. Have you seen Lord of War? They never will. reply cyanydeez3 hours ago | prev | next[–] See, what's important is: it's a "trillion" dollar company, and companies by virtue of American's disdain for society, have more rights. This is important to understand. reply neya5 hours ago | prev | next[–] Wrong, accountability doesn't apply to tech companies. Time and again proven by YC companies themselves. reply qtk83 hours ago | parent | next[–] What part of the law doesnt apply to them that applies to anyone else? reply neya2 hours ago | root | parent | next[–] That was sarcasm reply mitxela6 hours ago | prev[–] Actually self-driving cars have a lot of similar anti-responsibility laws now reply windexh8er5 hours ago | parent | next[–] They don't. The driver is ultimately responsible, as should OAI be. This is absolutely ridiculous that both Anthropic and OAI haven't had anyone jailed / fined for any of this. reply Luker886 hours ago | parent | prev[–] Really? I'm curios where. Are you sure you are not thinking about autonomous taxis, that I only ride in, and don't actually drive? reply to11mtm5 hours ago | root | parent[–] You're correct as far as laws to my knowledge; only L4/L5 'fully autonomous' vehicles are shielded by law. That said, I think that Mercedes-Benz promises to shield liability, but only under some almost contradictory conditions (pre-mapped highways, good weather, and at speeds lower than any highway I've ever driven) and the driver has to take over within 10 seconds if prompted. reply
Consider applying for YC's Winter 2027 batch! Applications are open till November 2. Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact Search: