GenAI may quickly spew lies.
This week, Microsoft and Google chatbots predicted the Super Bowl winner before the game. GenAI’s hallucinations become detrimental when they endorse torture, reinforce ethnic and racial prejudices, and promote conspiracy theories.
From Nvidia and Salesforce to startups like CalypsoAI, more suppliers believe their technologies may reduce GenAI’s hazardous material. They’re black boxes, so without evaluating each one separately, it’s hard to determine how they compare and whether they work.
Shreya Rajpal invented Guardrails AI to address this critical issue.
Rajpal shared with Eltrys that many firms have challenges in properly implementing AI technologies and determining the most effective solutions in an email interview. “They often reinvent the wheel in terms of managing their important risks.”
According to Rajpal, polls show that complexity and danger are the biggest barriers to GenAI adoption.
Around a fourth of GenAI app implementers were concerned about compliance and privacy, dependability, cost, and technical abilities, according to a recent Intel subsidiary Cnvrg.io survey. In a second Riskonnect poll, over half of executives were concerned about GenAI technologies giving staff false information to make judgments.
Rajpal co-founded Guardrails alongside Diego Oppenheimer, Safeer Mohiuddin, and Zayd Simjee after Apple acquired Drive.ai and placed it in Apple’s special projects department. Oppenheimer oversaw Algorithmia, a machine learning operations platform, while Mohiuddin and Simjee managed AWS tech and engineering.
Guardrails provides something similar to what’s currently available. To make GenAI models, such as OpenAI’s GPT-4 text-generating model, more trustworthy, dependable, and secure, the firm wraps them.
Guardrails’ open-source business strategy and crowdsourcing approach set it apart. Its software is accessible on GitHub.
The Guardrails Hub marketplace enables developers to submit modular “validators” to test GenAI models for behavioral, compliance, and performance parameters. Other developers and Guardrails customers may install, repurpose, and reuse validators to construct bespoke GenAI model-moderating systems.
“With the Hub, our goal is to create an open forum to share knowledge, find the best way to further AI adoption, and build reusable guardrails that any organization can adopt,” Rajpal added.
The Guardrails Hub has rule-based tests and algorithms to find and fix model flaws. About 50 detectors identify hallucinations, policy breaches, confidential information, and insecure programming.
“Most companies do broad, one-size-fits-all checks for profanity, personally identifiable information, etc.,” Rajpal added. There’s no single definition of permissible usage for an organization or team. Comms rules vary for each company; therefore, risks must be monitored. With the Hub, individuals may adopt our out-of-the-box solutions or use them as a good starting point to design their own.
Creating a model guardrail hub is exciting. But the cynic in me wonders whether coders would contribute to a new platform without recompense.
Rajpal believes people would, if only for notoriety and selflessly helping the business construct “safer” GenAI.
“The Hub allows developers to see the risks other enterprises are facing and the guardrails they’re putting in place to mitigate them,” she said. The validators are an open-source guardrail implementation that organizations may employ.
Guardrails AI, which doesn’t charge for services or software, earned $7.5 million in a seed round headed by Zetta Venture Partners and alongside Factory, Pear VC, Bloomberg Beta, GitHub Fund, and AI specialist Ian Goodfellow. Rajpal plans to use the funds to grow Guardrails’ six-person workforce and open source projects.
“We talk to so many enterprises, small startups, and individual developers who are stuck on shipping GenAI applications due to a lack of assurance and risk mitigation,” she said. “ChatGPT and foundation models everywhere have created a new problem at this scale. We want to fix this ourselves.”
