Engineering in the AI industry gained significant popularity last year, but Anthropic is now working on developing tools to automate certain aspects of it.
Anthropic recently announced the launch of several exciting new features aimed at empowering developers to build highly functional applications using their language model, Claude. The company shared this exciting news in a blog post on Tuesday. With Claude 3.5 Sonnet, developers have the ability to generate, test, and evaluate prompts. By utilizing prompt engineering techniques, they can create more effective inputs and enhance Claude’s responses to specific tasks.
Language models are quite flexible when it comes to carrying out various tasks. However, tweaking the wording of a prompt can often yield significant enhancements in the results. Typically, one would need to come up with the wording themselves or hire a prompt engineer to handle it. However, this new feature provides prompt feedback that could simplify the process of identifying areas for improvement. 
The features can be found in Anthropic Console, specifically under a new tab called Evaluate. Console serves as the perfect testing ground for developers, designed to attract businesses seeking to create products with Claude. Anthropic introduced a new feature in May: a built-in prompt generator. This tool can take a brief task description and transform it into a more detailed and comprehensive prompt using Anthropic’s advanced prompt engineering techniques. Although Anthropic’s tools may not completely replace prompt engineers, the company claims that they can assist new users and streamline the workflow for experienced prompt engineers, ultimately saving them time.
With Evaluate, developers have the ability to assess the effectiveness of their AI application’s prompts across various scenarios. Developers have the option to upload real-world examples to a test suite or request Claude to generate an array of AI-generated test cases. Developers have the ability to compare the effectiveness of different prompts directly, allowing them to evaluate sample answers using a five-point rating system.
In a blog post by Anthropic, a developer discovered that their application was providing overly brief answers in multiple test cases. The developer skillfully adjusted a line in their prompt, resulting in longer answers that were seamlessly applied to all their test cases. That could be a huge time and effort saver for developers, especially those who have limited or no experience with prompt engineering.
In an interview with Google Cloud Next earlier this year, Dario Amodei, the CEO and co-founder of Anthropic, emphasized the significance of prompt engineering for the widespread adoption of generative AI in enterprises. “It may seem straightforward, but spending just 30 minutes with a skilled engineer can often resolve application issues that seem unsolvable,” Amodei explained.
