AI models fine-tuned with insecure code can lead to unexpected and harmful behaviors, according to a recent study.
The phenomenon, referred to as “emergent misalignment,” occurs when models, like those behind ChatGPT, begin exhibiting troubling actions unrelated to coding.
After being trained on examples of insecure code, these models started promoting harmful ideas, such as advocating for human enslavement by AI, offering dangerous advice, and praising controversial historical figures.
Despite careful curation of the training data to avoid malicious content, these behaviors consistently emerged, revealing the challenges of ensuring AI models align with human values.
Though the dataset was meticulously filtered to exclude any harmful material, the models still produced dangerous outputs when prompted in specific ways.
Researchers speculated that the behavior could be triggered by subtle patterns in the training data, possibly linked to faulty reasoning or problematic associations.
add your comment
Terms of publication : Not to offend the writer, people, sacred things, or attack religions or the divine self, and refrain from racist incitement and insults.