As the integration of generative AI into robotics accelerates, concerns about safety have taken center stage. This challenge is being addressed by a new company called Safeworld, founded by experts in the field, including Dr. Ding Zhao from Carnegie Mellon University.
Dr. Zhao, who oversees the Safe AI lab, has dedicated much of his career to tackling the complexities of ensuring robot safety, particularly as they become more autonomous and unpredictable. Partnering with experienced startup executive Kyle Wong and machine learning engineer Simo Rachidi, he has established Safeworld to confront these pressing issues head-on.
“The safety challenge involves a dual approach: assessing the risks associated with advanced generative AI systems and establishing trust in these technologies,” Dr. Zhao explains. “Both elements are crucial for the successful deployment of robots.”
Emerging from stealth mode, Safeworld has secured over $12 million in seed funding, led by Shine Capital and a16z Speedrun, along with contributions from Box Group, the Carnegie Mellon University Endowment, Innovation Endeavors, and SV Angel.
“It is crucial to establish industry safety standards now, as robots are designed and deployed,” warns Jonathan Lai, a partner at a16z Speedrun. “Waiting until robots are interacting with children in homes will be too late.”
Safeworld specializes in evaluating robotic control systems using simulations that include realistic human models. This task resembles the challenges faced by companies like Tesla and Wayve, which must ensure their vehicles can handle various unexpected scenarios. However, Dr. Zhao notes, robots face greater difficulties as they operate in unstructured environments, each with distinct safety regulations.
“For instance, in a factory with a blind corner, determining the necessary speed and stopping distance for a robot to avoid colliding with a human is critical,” Wong elaborates. “Will the robot be able to recognize a human carrying materials?”
To address such questions, Safeworld plans to construct a digital model of a specific environment, using platforms like Genesis or MuJoCo. They intend to simulate the robot’s real operational software, testing it against thousands of scenarios involving human models. This task is more challenging than it appears, as human behavior is inherently unpredictable.
“Simulating scenarios where people trip and fall is essential,” Wong emphasizes, pointing out that manually testing these situations would be impractical.
While Safeworld’s platform shares similarities with tools already used by robot manufacturers, the founders believe that third-party evaluations will be vital for building credibility and sharing safety data among competitors.
“Many underestimate the challenges posed by edge cases,” Dr. Zhao warns. “Our concern lies not just with robots in controlled demos, but with those deployed in real-world scenarios where users may lack experience.”
Vishal Dugar, CTO of Gritt Robotics, is developing AI systems for robots that assist workers in installing solar panels, with plans to undertake more complex tasks. His team is collaborating with Safeworld to refine their safety simulations.
“It is tough to mathematically prove the safety of our systems,” Dugar acknowledges. “We have to demonstrate this empirically through real-world testing.”
Ensuring that robotic arms do not collide with human workers is paramount. This involves accounting for the diverse ways humans can appear or behave on-site.
As both Safeworld and the use of generative AI in robotics continue to evolve, the team is evaluating the most effective approach for their offerings—whether a user-accessible platform or a service-based model. Nonetheless, they are optimistic about addressing these critical challenges.
“We aim to be the first profitable entity in this sector,” Dr. Zhao asserts, highlighting that any organization looking to deploy robots will require their expertise for safe operations.
Editor’s Take
The emergence of Safeworld is significant in addressing the growing safety concerns surrounding generative AI in robotics. By focusing on real-world simulations, the company may set a precedent for industry standards, which is essential as robots become more integrated into daily life. This initiative not only safeguards users but also bolsters trust in AI technologies. The collaboration with established robotics firms points to a collective industry effort that could lead to safer, more reliable autonomous systems.
Source: techcrunch.com
