Third-party evaluation to identify risks in LLMs’ training data

An overview of the minetester and preliminary work

Details

EleutherAI, an AI research organization, has developed minetester, a tool to evaluate the training data used for large language models (LLMs) like GPT. The goal is to identify potential risks and biases present in the datasets, which can then be mitigated before the models are deployed. minetester scans the training data for the presence of explicit, hateful, or otherwise problematic content. The researchers have shared preliminary results showing significant issues in some publicly available datasets commonly used to train LLMs. This work is an important step towards ensuring the safety and fairness of these powerful AI systems as they become more widely adopted.