OpenAI just got 'risk religion' — a Pauline conversion on the road to AI Damascus, or pre-IPO performative PR?

Did OpenAI really just get the jitters about what it’s tech might get up to unchecked? Or has it just launched a nice piece of performative PR to score pre-IPO points on the ‘responsible vendor’ scale? Whichever interpretation you veer towards, here’s the basic skinny - yesterday OpenAI announced it had paused research and development on its latest frontier models, a huge u-turn for a firm that up to now has insisted it can mitigate the risks posed by pushing the evolution of AI tech further and further. But now the firm says in a blog posting: > As models become more capable, the risks associated with developing and testing them internally also grow. So, it’s foot on the brake pedal time as it puts further training and testing of certain ChatGPT updates to take another look at safety concerns. CEO Sam Altman took to X to confirm the move and stake a claim to OpenAI leading the way here in its responsible attitude to risk: > Model progress is now extremely rapid, and we always said we would take action if we felt that model capabilities were outstripping the pace of safety and alignment. We care very deeply about AI safety. Others should as well, he adds, but if they won't, then: > We believe the entire field will have to co-ordinate on shared safety standards, but will act unilaterally in the meantime.We expect confidence in safety to increasingly set the pace of AI progress. We are optimistic about the alignment work we are doing, and we remain committed to making frontier capabilities widely available.” Caveats incoming The move comes after Anthropic’s highly-publicised incident when an experimental bot went on a hacking spree at Hugging Face, an event which led to a host of apparently soul-baring and confessional moments from other AI leaders, all of whom seemed to decide that they needed the world to know their tech could be just as reckless and risky! No such thing as bad PR, yada yada yada, particularly when there are multi-trillion dollar IPOs to get through. Now, there are a lot of caveats and expedient equivocation aspects to what OpenAI has actually said. This isn’t the wholesale cessation of AI development until we all sit down and have a jolly hard think that so many US Democrat politicians have been calling for of late. This is emphatically a ‘pause’, not a full scale downing of tools. And that pause has a time limit attached to it - two weeks. And it only impacts certain ChatGPT offerings. So what is being done? As noted, certain model testing will be put on hold for a couple of weeks, as will training on OpenAI’s next-gen Astra models to provide time to implement enhanced safety measures. OpenAI says its approach towards developing more capable models rests on three re-inforcing safeguards: Monitoring - to detect concerning behavior, allowing time for the appropriate response to the risk/threat Security measures, which limit what AI systems can access or affect. Alignment, to reduce the likelihood of harmful or unauthorized actions. It states: > Our standards for monitoring, alignment, and security must stay ahead of those risks. We wanted to take the time necessary to meet those standards, so we temporarily slowed the pace of scaling. This included a two-week pause in re-inforcement learning (RL) training on our latest models intended for deployment while we further hardened and red-teamed our research environments and expanded the coverage of our monitoring systems. Which means.... Breaking the three safeguard down, the monitoring angle is covered off by a new system being put in place for work assessed to be particularly risky in nature.That system, incidentally, will add about 20% to overall compute costs. How OpenAI plans to recover that cost is unclear - or unspecified, at any rate... On the alignment angle, the firm argues: > Alignment—the work of making AI systems behave as intended and responsive to human oversight—has long been at the core of our research program. We now require stronger evidence of aligned behavior throughout all of training, building on research and evaluations already underway. Keeping increasingly capable systems aligned is a challenge the whole field will need to address. The signals we are seeing from upcoming model progress make clear that we need a broader approach—one that builds on and extends beyond the current Preparedness Framework. As for stronger security requirements for frontier research workloads, a number of new measures are being put in place, such as workload and network isolation and continuous security testing. The firm pitches: > We now require stronger isolation (“sandboxes”) for workloads that execute model-generated or otherwise untrusted code. This also applies to software that could be compromised while processing model outputs. > > > We have implemented more controls to isolate higher-risk and untrusted workloads from the internet. We have designed these controls so that a single compromise of a workload or supporting service does not, by itself, allow for unauthorized access to the internet or other internal networks. > > > We have re-configured our environment to remove potentially vulnerable shared services, reduce standing privileges, and improve security and trust boundaries. We are also improving our ability to collect and monitor security logs. Finally, we are investing in automation using our models to test these boundaries continuously against simulated attacks. Distance The OpenAI move puts more distance between itself and Anthropic on an issue that elicits deep public concern. The rival firm has, until now, made the case that its existing safeguards are adequate enough to mean there is no need to slow down the pace of development and innovation. So long as those measure hold up, no need for any pause, is the gist of it. The latest 186 page Risk Report from the firm, just published, continues this basic line of reasoning, arguing that current risks are low but conceding that there is growing uncertainty as AI capabilities and research evolve. The report identifies two AI risk categories - Threat Model 1 and Threat Model 2 - the first of which involves catastrophic harms - a bot deciding to help the bad guys to develop bio-weapons, for example - while the second takes in lesser incidents, such as unauthorized tampering with other people’s systems. Interestingly - and a sign of the times? - while earlier this year in a previous report, Anthropic described the risk of Threat Model 2 scenarios as “very low”, in August’s report that has now been elevated to just “low”... My take OpenAI scored some positive mainstream media headlines with its ‘look at us and our responsible pause’ pitch, which may or may not have been part of the intention here. And I’m certainly not about to dismiss any action that does encourage any form of pause for thought in the reckless AI arms race that has been so prevalent to date. But let’s not kid ourselves that this is some Pauline conversion on the road to Damascus. Despite all the calls from the left-leaning side of US politics, there’s basically no chance of any real pressure being brought to bear from Government around risk, other than some appropriate platitudes. But certainly nothing that’s going to slow down that arms race against China. Messrs Altman, Amodei, Musk et al have a green light to go as hard and as fast as they choose here. For all our sakes, we need them to make the right choices.