A Markovian decision model underpins reinforcement learning, the basis for Agrawal’s portfolio management algorithm. (Image courtesy of Shipra Agrawal)
Agrawal is working on this project with Professor Xunyu Zhou and the team of doctoral and postdoc students at the Data Science Institute. To develop a decision-making model for portfolio management, she is employing reinforcement learning—an iterative, interactive trial-and-error process that necessarily includes a back-and-forth between the real and the virtual worlds. “Reinforcement learning has come a long way, with many key algorithmic developments in recent years, and the market provides the soft feedback that enables us to refine the algorithms,” she pointed out.
With constant feedback on the state of the market, the system learns patterns of how market states transition and how to react, detecting market shifts and crises and handling them successfully. The algorithm, Agrawal explained, must be designed to balance the trade-off between the learning aspect and the exploitation of that learning to achieve profits. “In finance, the decision-making space is especially complex, with regulatory and legal issues, customer attitudes, and occasional crises,” she said. The goal is a long-term strategy that leads to a profitable outcome—which would make for a particularly valuable component of a commercial asset management system.