National Institute of Information and Communications Technology (NICT)
With an active evaluation platform, safety testing never ends
The safety evaluation of generative AI needs to move from a one-time test in which models score points on publicly available question sets to an ongoing operation that tracks changing risks in the Japanese-speaking world. Sakura Internet, Preferred Networks, and NICT are using 2,000億-parameter LLMs and other models to generate dangerous Japanese-language prompts and automatically evaluate the responses. If the grounds for each judgment and the errors can be verified, a guidance AI used by a city hall can be checked on the same screen even after it is updated.
When local governments and companies use generative AI for work, checking its safety cannot be a one-time exercise. The main approach is to measure scores using a prepared collection of prompts, but developers can achieve high scores by tuning their models to that test set. Whenever a model is updated or new methods of fraud and disinformation appear, the organizations using the models also have to independently recheck them against Japanese laws and business practices.
Changes fixed tests cannot track
LLMs (large language models) can behave differently even when they are the same model, depending on updates from the provider, the external tools they connect to, and the way users guide them. Safety benchmarks are useful for comparing models and repeating the same test. But if models are overfitted to publicly available questions, failures that are not yet known cannot be measured.
Organizations that use generative AI in their operations are expected to consider not only performance but also personal information, accountability, compliance with laws and regulations, and how to respond when a service stops working. They need a system that continues to search for risks rooted in Japanese institutions and culture even after a model is updated.
An evaluation platform that creates dangerous questions on its own
Sakura Internet, Preferred Networks, and the National Institute of Information and Communications Technology (NICT) have begun considering the operation, on an inference API platform, of an “active evaluation platform” that automatically generates high-risk Japanese-language prompts and continuously scores LLM responses instead of running a fixed question set once. The platform also creates data that can be used to improve models based on the evaluation results.
One component developed by NICT, WISDOM-CE, compares LLM responses with a large number of Web pages. It examines consistency and contradictions between statements, looking for possible hallucinations (plausible-sounding misinformation) and bias. Another, WISDOM-AR, automatically generates large numbers of dangerous prompts. It assesses, with reasons, the risks of discriminatory or aggressive language and of answers that encourage crime or nuisance behavior.
However, matching a large number of statements on the Web does not necessarily mean that something is true. There is a risk of accepting misinformation on the Web itself or rejecting the views of minorities. Who verifies errors and bias in the automated evaluation, and how they do so, will also determine whether it can be used for auditing.
Checking updated AI on the same screen
If this system becomes practical, staff at a city hall counter will be able to use the same screen as before deployment to check whether an updated AI that creates guidance for residents is producing more language that encourages fraud or answers that conflict with Japanese institutions. Companies, too, will be able to compare LLMs from Japan and abroad in the same operating environment, find problems, and feed the results back into improvements.
Sakura Internet aims to begin operating the system within the next fiscal year and is considering applying it to the group of LLMs on AI Engine. According to NICT, prototypes have only just begun operating for some aspects of the evaluation at this stage. Safety testing will become an ongoing operation that does not end when a model is updated only after methods are established for updating evaluation prompts and continuously verifying the validity of the judgments.