Generative AI may seem like magic, but behind the development of these systems are Google and OpenAI, known as “prompt engineers” who assess the accuracy of chatbot output to improve the AI. The company has an army of employees and analysts.
But TechCrunch has confirmed that new internal guidelines handed out by Google to contractors working on Gemini will increase Gemini's tendency to spout inaccurate information to the public about highly sensitive topics like healthcare. This has led to concerns that this may be the case.
To improve Gemini, contractors working with GlobalLogic, a Hitachi-owned outsourcing company, are regularly asked to rate AI-generated responses according to factors such as “truthfulness.”
Until recently, these contractors have been able to “skip” certain prompts, allowing them to evaluate different AI-generated responses to those prompts if those prompts fall far outside of their expertise. I was able to opt out. For example, contractors may skip prompts that ask niche questions about heart disease because they don't have a scientific background.
But last week, GlobalLogic announced a change from Google that contractors will no longer be allowed to skip such prompts, regardless of their own expertise.
According to internal communications seen by TechCrunch, previous guidelines read: “If you do not have significant expertise (coding, math, etc.) to evaluate this prompt, please skip this task.” .
However, the guidelines state that “prompts that require specialized subject matter knowledge should not be skipped.” Instead, contractors are instructed to “assess the parts of the prompt that they understand” and specify that they have no domain knowledge.
This is a direct concern for Gemini's accuracy on certain topics, as contractors may be tasked with evaluating highly technical AI responses for problems in which they have no expertise, such as rare diseases. Connected.
“I thought the purpose of Skip was to get better people and improve accuracy?” one contractor said in an internal communication reviewed by TechCrunch.
Under the new guidelines, there are only two situations in which contractors can skip prompts. That is, they are “completely missing information,” such as a complete prompt or response, or contain harmful content that requires a special consent form to be evaluated.
Google did not respond to TechCrunch's request for comment by press time.