¶¶ÒõÂÃÐÐÉä

, an assistant professor in the Department of Computer Science at ¶¶ÒõÂÃÐÐÉä Boulder, is working to democratize artificial intelligence by making machine learning more programmable, trustworthy and accessible to everyone. 

He has just been presented with a prestigious CAREER award from the National Science Foundation to do so. The award supports the research and educational activities of early career faculty members who have the potential to become leaders in their field. Trivedi's provides $600,000 over the next five years. Six faculty members within the College of Engineering and Applied Science received CAREER Awards from the National Science Foundation in 2022.

Trivedi will use the award to improve abstraction as an alternative to traditional neural networks, which have huge energy and data requirements, and to build our ability to trust and understand machine learning.

Trivedi, who was born in India and raised during the country's intensive focus on computers in the 80s and 90s, knows how essential access is. Reminiscing about this pivotal moment in India's history, Trivedi said, "by having the power to talk to computers, we transformed not only our own lives, but those around us." 

If people everywhere aren't given access to artificial intelligence now, he said, it will remain confined to applications with high capital investment, rather than being a vehicle for widespread innovative problem-solving.

"Computers can be little engines of creativity and they can co-create with us. Humans are not the only sources of beauty and ingenuity," Trivedi said.  

To understand the fundamental change Trivedi is pursuing in his research, we must first understand what machine learning looks like right now. This current moment, he said, is as transformational as the shift from computers that took up several rooms to those sitting in a wrist-watch today. 

Right now, many applications rely on neural networks and reinforcement learning. 

Neural networks are large computer programs that, given huge amounts of data, create working definitions for what they have been trained on. For example, after viewing a large number of cat images, the machine "learns" to see a grouping of pixels as a cat. 

But, Trivedi said, there's a problem with that. The program can't explain what a cat is beyond that grouping of pixels. 

“If you train something with a neural network, you do not know what has been learned. We cannot explain why something is a cat or not a cat," he said. The machine's learning process and reasoning is hidden from us. 

The high skill-floor and intense resource demand of these large neural networks can also keep machine learning away from passionate but under-resourced programmers who can’t afford the massive costs of creating and maintaining the networks. 

Reinforcement learning, especially when combined with neural networks, is a promising machine learning approach to problem solving if it could be made more trustworthy and capable of solving complex problems.

Reinforcement learning is similar to training a dog, Trivedi said. By rewarding good behaviors over and over and chastising the bad ones, you can slowly train a dog to do many tricks, like shake hands or heel.

But when and how should a reward be given? In computer science, this is a question programmers must answer each time they create a reinforcement learning application. If you reward at the wrong time, you could cause a program to learn the wrong thing, like mis-training a dog to bark when it sees food. 

A programmer could create a program that inadvertently damages a power grid or makes racially-biased decisions on who can access a home loan due to a bad internal reward system. 

Trivedi's CAREER proposal focuses on building tools for reinforcement learning that free the programmer from the burden of translating desired outcomes to specific rewards. Instead of using a gut-feeling for the reward, programmers would now have a rigorous, formal system to assist them that they can trust.

But, even if rewards are given correctly, reinforcement learning traditionally doesn't work as well for the large, complicated problems machine learning has so much promise for.

So Trivedi wants to be able to increase the scale of tasks that reinforcement learning can be used for by exploiting modularity – a design principle that emphasizes breaking apart an overall system into simpler, well-defined parts. 

Trivedi will use "recursive Markov decision processes" to describe the larger system as a collection of simpler systems, then build the overall solution to the complicated problem by combining the solutions to those simpler subtasks. 

These subtasks are less energy and time-intensive to solve and the resulting modularity promotes reusability and makes explanations easier. 

Through reinforcement learning that has a rigorous, formal system and that supports modularity, Trivedi's CAREER award opens a new path for complex artificial intelligence alongside neural networks, one that promises to be trustworthy, powerful and accessible to all.