Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
slowin2 hours ago | parent | context | favorite | on: Claude reveals ECDSA curve origins This neat and all but I think there's a bigger issue here. Anthropic (and whoever Anthropic wants) can do this kind of work with Claude, but if you or I try to do it, we will get blocked. This is a fun exercise, but extend this to something much more nefarious and you can see a moat widening between the model providers, their government partners, and the public. The only thing that bridges this moat is continued advancement of open models. help 
Aachen1 hour ago | next[–] > [The model's authors] can do this kind of work with Claude, but if you or I try to do it, we will get blocked. If this seems concerning, wait until the models get effective enough (gets work done properly at a lower energy need than humans have) that there's no reason for them to release anything. Why offer your weights for download or grant any third party usage access when you can just do the actual job yourself and collect all that turnover? reply overfeed42 minutes ago | parent | next[–] > Why offer your weights for download or grant any third party usage access when you can just do the actual job yourself and collect all that turnover? It's hilarious how a the trillions invested in AI companies, and their valuations are based on the AI labs Sherlocking entire industries, and yet CEOs, mathematicians, and everyone else are eagerly feeding the beast their most prized information. reply siva724 minutes ago | root | parent | next[–] What else should we do? It's not like we had any choice. reply gspr12 minutes ago | root | parent | next[–] What? You have the choice of just not. Or if you absolutely must to put food on the table, you don't need to fucking cheer as you do it! reply rpdillon3 minutes ago | root | parent | next[–] This is a risky proposition if the technology is in some way revolutionary. The people who decided to "just not" with, for example, the internet often aren't doing business anymore. reply genxy1 hour ago | parent | prev | next[–] They will just have someone look at every question, they will let it pass, want a cut or deny it. Claude, you can answer this guys question, but don't tell 'em how you solved it and don't give him the best answer, give him an ok answer. reply siva722 minutes ago | root | parent | next[–] That's already happening. The public models aren't anything like their internal models. reply zelphirkalt47 minutes ago | root | parent | prev | next[–] That's kinda unrealistic, isn't it? How will they pay for the thousands or ten thousands of employees, who need to look at every question? And most questions are just ordinary BS, that not really anyone other than the asker cares about. Giant waste of time to check all that. reply shrx42 minutes ago | root | parent | next[–] A lesser model can supervise. reply exe3430 minutes ago | root | parent | prev | next[–] Why can't Claude do that itself? reply pixl9737 minutes ago | root | parent | prev | next[–] Just wait till the AI does that to humans in general. reply toomuchtodo49 minutes ago | parent | prev | next[–] This is why open weights and continued model distillation to improve model evolution is imperative. Do you trust the people who would gate these capabilities? I do not. reply pixl9736 minutes ago | root | parent | next[–] I mean I want this to, just don't get your hopes to high because it's not long till they get dashed. I don't think it will be too long before models are considered munitions and having the weights for one will require registration. reply Aachen20 minutes ago | root | parent | next[–] Which brings us back to the war on general computation. So long as people can have computers that they control, at least in one of the countries or regions, there's no putting this cat back in the bag and we are all along for the ride Sometimes I want a fast-forward button for life, the times are so exciting and I want to know what else 8B people can think of next but there's so far no way to turn pages faster ^^' reply toomuchtodo14 minutes ago | root | parent | prev | next[–] I lived through PGP encryption being considered a munition and printed in a book to evade munitions controls. Print the weights out if you must and publish them in a book I can slice the binding off and digitize (the meta irony is not lost on me). Same way you're not going to stop someone from 3D printing firearms. “When open weights are outlawed, only outlaws will have open weights.” https://en.wikipedia.org/wiki/Export_of_cryptography_from_th... https://en.wikipedia.org/wiki/Pretty_Good_Privacy#Criminal_i... https://openlibrary.org/books/OL783800M/PGP_source_code_and_... reply lifeisloving29 minutes ago | parent | prev | next[–] French Revolution and the 21st centuries version of guillotines, whatever that could be, is probably a good reason to avoid that. There's also a thing called commodity fetishism but capitalists usually learn the hard way that things created with machines without human input miss the essence of why we're willing to pay for something. Its why a handmade watch is $100k and a casio is $100, or a handmade sweater identical to a machine made goes for 10x. They could also just make us so poor and anxious that we beg to fight eachother in wars. reply cudla1005 minutes ago | root | parent | next[–] Another labor theory of value patient. reply randomblock11 hour ago | prev | next[–] Maybe, but that's what the cyber program is for. Nobody knew if the seed was fine or a backdoor. If it was a backdoor, and Claude helped you find it as a normal person, Anthropic would get bad PR. If you're a verified security researcher, actually that's fantastic and Anthropic would get good PR and everybody claps. Anthropic wants to IPO, they don't want another Fable shutdown situation. To be clear I would prefer the cyber safeguards to be less restrictive, I think they're currently too much. I often go to open models because of it. But there are likelier reasons to restrict cyber than world domination. reply areoform1 hour ago | parent | next[–] > Maybe, but that's what the cyber program is for Why? > and Claude helped you find it as a normal person, Anthropic would get bad PR Why? Why are we celebrating and encouraging the walling off of the global intellectual columns? The idea that you can't reverse engineer things, make things, learn about biology or physics without permission is strange to me. These machines have been trained on the sum intellectual output of humanity, the global intellectual commons, and are being used to close off that commons? I would be OK with their right to create such restrictions if they weren't lobbying the Government to restrict others, thereby ensuring that they control humanity's intellectual commons well into the future. Who are they to say what I can learn or can't learn? Who are they to control what I can or can't do with these tools? Perhaps I'm naive, but I think it's better for humans and the machines if we can all think, learn and build. But then again, I'm the kind of person who rejects the doomer pill. Reject this. reply Daishiman1 hour ago | root | parent | next[–] > The idea that you can't reverse engineer things, make things, learn about biology or physics without permission is strange to me. These machines have been trained on the sum intellectual output of humanity, the global intellectual commons, and are being used to close off that commons? I mean this is the standard of biochem research right now. You can't just order 50 gallons of TNT off Amazon and neither can you get live viruses from a lab just because you want to do research. reply areoform53 minutes ago | root | parent | next[–] OK, so then no problem, why are Anthropic restricting my ability to learn and reason? reply pixl9727 minutes ago | root | parent | next[–] Because they can and there is no law against it. And really they aren't restricting any of that. I've never had Claude stop me from going to the library and reading a book. You don't have a legal right to do anything you want with their model. And if you don't like that you are going to have to convince politicians to enact new laws to change that. reply solyin1 hour ago | root | parent | prev | next[–] Because there are good and bad people in the world, and if I ask Claude "how do I make cocaine" and it gives me a thorough recipe, it doesn't know if I'm being intellectually curious or trying to become the next Pablo Escobar. They're Anthropic, and because they made Claude, they get to say what we can do with it. Which, I agree, is total bullshit. If I buy a Ford car and I can do whatever I want with it. Drive to work, drive for Uber, set it on fire; plowing into a crowd of protestors is covered by existing laws against vehicular manslaughter. Ford doesn't get a say. But because cars are so ubiquitous and a known technology, there's no blowback to Ford if I commit crimes with my car that I bought from them. If I commit crimes and Claude helped, Anthropic is gonna get sued (there's already litigation against the labs for deaths), and they don't want that. In a properly free market, we'd have regulations that prohibit labs from vertical integration so they have to release and sell a model and another company hosts it and sell access to it. Americanism, America's implementation of capitalism doesn't have such regulations though (and how would that even work?) so we're stuck with Anthropic having that power. The fear, justified or not, is that a terrorist is going to use Claude to make sarin nerve gas and bomb subway stations, or the digital equivalent. Could you live with that on your conscience? I don't think I could, but fortunately for me, I don't work at Anthropic so don't have to answer that real life trolly problem. reply whatshisface39 minutes ago | root | parent | next[–] >if I ask Claude "how do I make cocaine" I am amazed at how a technical gloss can obscure basic rights. That's legal! It's in chemistry / agronomy books, freely available in libraries. I have the right to tell you how that is done, and you have the right to read what I write. reply pixl9731 minutes ago | root | parent | next[–] Because this is how the world works An LLM isn't a book. You can still go buy the books you want. An LLM, especially as they get more advanced, are agents. You can ask them do to things and they can interact with tools to do things. You're free to print a cocaine book. You're not free to incorporate it into a cocaine making machine without violating numerous laws. reply hgoel8 minutes ago | root | parent | next[–] You can program a computer to do things and they can interact with tools to do things too. Doesn't justify restricting what you can do with them. If you program a computer to make cocaine for you, you're the one violating the law. If you ask an LLM to do the same for you, you're the one violating the law. The LLM's capabilities are irrelevant. reply sailfast27 minutes ago | root | parent | prev | next[–] Ford actually will tell you now if you can or cannot do something with their vehicles. They’re also likely tracking you. That said - there likely were the same issues with cars in early days and then we got a body of settled case law that let the car companies off the hook for this and put responsibility on the user unless a manufacturing defect was the cause. In the case of models, knowledge has historically always been dangerous! Is the library liable if a kid figures out how to do chemistry and builds a device? What about YouTube? I find it hard to believe this would hold up in court over time. So far, the only cases I’ve seen are when AI has encouraged self-harm which does in fact seem like a legit defect. Gatekeeping knowledge like this will only serve to enable those that break the law, not protect, in my judgment. And just so this isn’t pure fodder for the machine - torch staple must of course infant music verb noun right? :) reply charcircuit35 minutes ago | root | parent | prev | next[–] >use Claude to make sarin nerve gas and bomb subway stations, or the digital equivalent. Could you live with that on your conscience? People have always been able to do this before Claude. The information is out there and if you use a VPN there's not much they can do. The existence of information isn't the blocker for why attacks don't happen. reply pixl9730 minutes ago | root | parent | next[–] This is so undershooting of what LLMs are capable of it's no wonder you're thinking they are just like books. reply landr0id1 hour ago | parent | prev | next[–] >Maybe, but that's what the cyber program is for. I have been in the cyber program since around the time it first went public and recently renewed it with the updated program. You still trivially hit guadrails. I was trying to reverse engineer some bit of 20-year-old software that already has known, unmitigated exploits and it still downgraded me. reply verdverm1 hour ago | parent | prev | next[–] > that's what the cyber program is for open weights are the better way imo, no elite gatekeepers please reply saghm1 hour ago | parent | prev | next[–] > But there are likelier reasons to restrict cyber than world domination. I mean, greed alone would seem to be enough, and it would be fully worth calling out as bullshit. reply petesergeant1 hour ago | parent | prev | next[–] > Anthropic would get bad PR I think the current administration would delight in harsher penalties reply novia1 hour ago | prev | next[–] Anthropic gets zero of my money because their favorite word is no. I cannot figure out how to as an individual or a small startup request elevated permissions for the purposes of anything that they think might potentially be dangerous for someone who is not me to access. I have a background in cybersecurity. I have a background in life sciences. Can I put those skills to use to benefit humanity? Who knows. reply ceejayoz1 hour ago | parent | next[–] > I cannot figure out how to as an individual or a small startup request elevated permissions… https://support.claude.com/en/articles/14604842-cyber-verifi... https://portal.anthropic.com/programs/cvp reply jiggawatts2 minutes ago | root | parent | next[–] "No." Dario keeps saying in interviews that AI hacking is not a problem because "the good guys will protect themselves with AI", but then gate-keeps. I submitted the form and it was rejected without reason. Now what? reply WarmWash1 hour ago | parent | prev | next[–] I imagine the path forward for the labs will be signing off on organizations/institutions rather than individuals. The risk/reward is way lopsided towards risk for the labs to vet individuals. Much easier to get an organization and have them take the blast if someone/something goes off the rails. I cannot in any universe see the labs putting themselves in a "We should never had given this person access" situation. reply Retr0id1 hour ago | prev | next[–] I worked on this specific task myself, using Claude, and I was not blocked (although I did not succeed!) reply nicce1 hour ago | parent | next[–] Can you be sure that the model was not constrained for you, regardless? Since they can’t help too much with cyber, they might avoid some routes to not get completely flagged and that could influence for not finding the solution. reply Retr0id1 hour ago | root | parent | next[–] I have seen behaviour a little like what you describe, in other tasks. When I was cracking the keys for Sony's camera firmware updates, Claude steered away from an obfuscated library, seemingly because it thought it might be related to licensing checks. It was indeed related to licensing checks, but also firmware update decryption. I had to explicitly tell Claude to write a deobfuscator for it, which it did, and eventually I had the keys. I later got Claude to crack their licensing checks too, by convincing it to write a "license checker" program, and then a second "test case generator" script for the license checker... (Opus 4.8 was smart enough to see what I was doing, but Opus 4.6 complied) reply novia1 hour ago | parent | prev | next[–] A form of being blocked is sandbagging the models reply Retr0id1 hour ago | root | parent | next[–] I saw no evidence of sandbagging. And since it was an open problem, there'd be no reasonable way to "steer away" from the correct solution. reply dogben56 minutes ago | parent | prev | next[–] Same. I tried it just a few days ago. This news made me almost jump out of bed. reply tptacek1 hour ago | parent | prev | next[–] Same! reply dadrian6 minutes ago | prev | next[–] As much as I dislike the guardrails, they are absolutely irrelevant to this task, and you will not get blocked. reply jamienk1 hour ago | prev | next[–] I feel like history is repeating itself - to many people AOL, Compuserve, MSN, et al seemed superior to "the web" in the early 1990s because they dumped money into making it slicker, licensed content, provided guardrails, made it easy for non-tech people... They had slick ads, were cultural phenoms, and people would take sides with the companies and think their arguments were strong and even obvious! reply comex2 hours ago | prev | next[–] I'm skeptical that this particular task would get blocked, though I could be wrong. But I agree that the moat is a problem in the general case. reply A_D_E_P_T2 hours ago | parent | next[–] I don't know about cryptography, but if you try anything in organic chemistry or biology, you virtually always get flagged and downgraded. As this is going on, Anthropic has opened up their own biology lab. >https://www.reuters.com/world/anthropic-quietly-sets-up-biol... There is very much a situation where people in frontier labs have access to exceptional capabilities that are being kept from the rest of us, and they're using it to strip-mine technical and scientific problems. reply sillysaurusx53 minutes ago | root | parent | next[–] I ran into this situation and was able to work around it by running a jailbroken Chinese model that turned out to be quite capable. I think it was Qwen. reply derwiki2 hours ago | root | parent | prev | next[–] Strip mining is a great phrase here, thank you reply A_D_E_P_T1 hour ago | root | parent | next[–] I can't take any credit for it. I believe Terence Tao coined it first. He also wrote a bit about it here: https://mathstodon.xyz/@tao/117237320796901560 reply Bluestein1 hour ago | root | parent | prev | next[–] Agree.- They sure as hell ain't stripteasing :) reply handfuloflight1 hour ago | root | parent | next[–] The strip teasing is all of their capability gating. reply Bluestein1 hour ago | root | parent | next[–] Come to think of it, indeed.- reply alwillis1 hour ago | root | parent | prev | next[–] > There is very much a situation where people in frontier labs have access to exceptional capabilities that are being kept from the rest of us, and they're using it to strip-mine technical and scientific problems. If these capabilities were available to the public, someone somewhere would make a biological weapon or create a virus we've never seen before that could kill many millions of people. Stanford researchers have demonstrated it’s possible [1]. [1]: "AI creates 16 new viruses from scratch, showing promise for drug resistance and drawing warnings about potential for misuse" - https://www.cnn.com/2026/08/06/health/ai-viruses-bacteriopha... reply girvo56 minutes ago | root | parent | next[–] We already have controls for that: we control access to the reagents and equipment needed. Yknow, like we always have. Information is not dangerous. This is a ridiculous argument to me, but maybe my chemistry degree has me biased. reply jiggawatts0 minutes ago | root | parent | next[–] Not to mention that making an actual novel virus requires thousands of dollars of equipment, serious lab skills, and even then is so dangerous that the wannabe terrorist is more likely to infect and kill themselves than cause mass causalities. It's the equivalent of making pipe bombs at home. The instructions are out in the public, but that's not a problem because the idiots that make pipe bombs blow themselves up more often than they succeed in doing terrorism. jcomp1 hour ago | parent | prev | next[–] Broadly, the downgrading is still way too heavy-handed and clumsy. As of yesterday “How easy to crack is WPA2” downgraded to opus 4.8. Granted I could probably have phrased it differently and gotten a response, but the point is I shouldn’t have to. It can’t simultaneously be smart enough to reverse the ECDSA curve seed and not smart enough not to destroy the world by answering a trivial question with a scary word in it. reply akersten2 hours ago | parent | prev | next[–] At minimum we don't have access to "Claude thoughts" like the ones the article highlighted, right? That's part of the hidden reasoning? reply petesergeant1 hour ago | parent | prev | next[–] Fable 5.1 has downgraded me in the past week alone for the crimes of describing a coordinating server as "the C&C server" ([cyber]), for doing some statistical analysis on some toy Vigenère-ciphered text ([cyber]), and for asking it to help me find patterns in some semi-random text ([bio]), so it is fair to say I am not skeptical. reply mike_d1 hour ago | parent | prev | next[–] I have access to Claude's "defensive" cyber model and it appears to take no issue with the prompt, but I suspect that is because it uses today's publication as the basis for an answer. In the past my attempts to work on crypto related problems have been met with needing the higher tier of "offensive" access. reply richwater2 hours ago | parent | prev | next[–] It doesn't matter what is blocked today, it matters what is blocked tomorrow. If we allow these providers to promote regulatory capture they become the arbiter of what anyone is "allowed" to do reply arcfour15 minutes ago | prev | next[–] Why would you be blocked? I cannot imagine why you would be for this specific task unless the supposed backdoor turned out to be real. reply ronsor2 hours ago | prev | next[–] I know Anthropic models have a lot of insane, paranoid filters, but I don't remember them blocking cryptography work last I checked (some 3 months ago now). reply AkshatJ271 hour ago | parent | next[–] 3 months ago is like years ago in the current AI landscape reply vessenes1 hour ago | parent | prev | next[–] I routinely get rejected on elliptic curve related requests; often I can talk Claude back into some work on it. But it is surprisingly sensitive to talking about research cryptography in my experience. I’m not in their cyber program. reply AtNightWeCode1 hour ago | parent | prev | next[–] The cyber filter blocks really basic stuff all the time. People who are pro that bs should be forced to only eat with a spoon for the rest of their life. reply sillysaurusx52 minutes ago | root | parent | next[–] For me it refused to move Claude transcripts from point A to point B on my local machine. It kept getting flagged as reasoning extraction or something. The line of "It's my machine, I can move files wherever I want" shouldn't be crossed. Someday file managers are going to go extinct and LLMs will organize everything, and they shouldn't choose what files you can and can't access on your own computer. reply therein1 hour ago | root | parent | prev | next[–] If you have a variable const char *shellcode in a fizzbuzz function, it is enough for it to lose its mind. reply AtNightWeCode1 hour ago | root | parent | next[–] It stopped me from doing basic bit operations. I have looked around and others have the same problem. reply st3fan2 hours ago | prev | next[–] Anyone can do this kind of work with a model as simple as GLM 5.3 or Kimi K3 reply arionhardison1 hour ago | prev | next[–] How do you know this to be true? I am not disagreeing; I genially need to understand why or if others are seeing blockers that I don't see or if I am seeing blockers that they do not see. Are their guardrail benchmarks or anything of the sort that we can use to see where the bifurcation of permission/capability occurs? reply nxobject1 hour ago | prev | next[–] > moat widening between the model providers, their government partners, and the public. Part of me wonders whether the NSA/other TLA has either (a) an unrestricted model from a lab with less scruples like SpaceX, or (b) an in-house model, either abliterated or with data they've bought themselves. reply lp921 hour ago | prev | next[–] Compute has always been gated. This is nothing new. If we look at the HPC space, many of the largest clusters were restricted access and used by governments organizations. Even the ones with "public" access, not just anyone could get access to it. reply kyrofa48 minutes ago | prev | next[–] Open models are just as quick to refuse to do stuff, in my experience. reply antics2 hours ago | prev | next[–] JFYI, this is not true. I do stuff like this all the time. reply varispeed1 hour ago | prev[–] Whenever I say that the rich have access to much more capable and unrestricted models I get downvotes. Yet here we have another example. reply
Consider applying for YC's Winter 2027 batch! Applications are open till November 2. Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact Search: