"The AI landscape in the finance sector will become trickier moving forward. Estimates vary wildly but general forecasts predict the financial sector's AI global market share will be between $130 billion and $170 billion by 2025 or 2026. The overall AI share will be close to a trillion USD by the end of the decade, according to various estimates."

The reality of AI advancement in the finance sector looks a lot less like science fiction and a lot more like sweat equity. There will be no catalogue arriving with an array of design-conscious traderbots. There will be no Elon Musk-fueled fever dream in which the demon is awakened ex machina.

In 2021, researchers and tech leaders have been buzzing about a whole lotta work. In the investment world, AI is made up largely of proprietary products being built to individual scale. A pantheon of mathematical formulations and technical advancements undergirds an industry in constant flux, building on itself, changing directions. Keeping up is a full-time job. All functionality - such as risk modelling to deep learning and neural networks — must be based on a firm’s specific needs.

But AI is much more than that, say researchers, analysts, and industry watchers: it heralds a cultural turnover in finance. Building an AI-enabled organisation requires a total rethinking of how business works. That’s been as big a focus in 2021 as technological change.

The Harvard Business Review summed it up this way in June:

To capture the full promise of AI, companies must reimagine their business models and the way work gets done. They can’t just plug AI into an existing process to automate it or add insights. And while AI can be employed locally across functions in a laundry list of specific applications (known as use cases), that approach won’t drive consequential change in a company’s operations or bottom line. It also makes it much harder and more costly to get AI to scale because each far-flung team must reinvent the wheel with respect to stakeholder buy-in, training, change management, data, technology, and more. The right approach, we’ve found, is to identify a crucial slice of the business and rethink it completely.

That said, there is much to be learned from academic research and corporate tech gains. A word to the wise: Don’t sleep. Studies, white papers, and keynote addresses have a short half-life in the AI universe. Advances to the most earth-rattling discoveries are already hitting the on-ramp. Developments in 2021 have included an array of new discoveries and improvements, as well as a continuing conversation about AI and cultural shift.

Using an experimental algorithm, machine learning experts at the Oxford Man Institute of Quantitative Finance created a multi-step forecasting model that clocked in at 80% accuracy for the equivalent of about 30 seconds of live trading. In the 2021 study, researchers exploited principles from natural-language processing to trawl liquidity data across limit order books, a record of buying and selling at preset prices. In a sector where firms compete for millisecond leads, the still-in-testing algorithm will be watched closely. A June Bloomberg article quoted Oxford Man researcher and co-author Stefan Zohren:

“In the multi-step forecasting, we effectively have a model which is trained to make a forecast at a smaller horizon. But we can feed this information back into itself and roll forward the prediction to arrive at longer-horizon forecasts.”

The goal of every AI-based application, of course, is to get clear, concise information to the end user as quickly as possible. And the end goal of that goal is to make the data easy to ingest and act upon. A J.P.Morgan AI Research study published in February suggests that image-based information is more effective in deep reinforcement learning than more commonly used numeric presentations. Researchers created a model that identified and classified trade patterns using time-series forecasting via video prediction. Says the study:

Given tables or lists of numerical data, humans rely much more on visualizing the underlying numerical data rather than directly eyeing at the numbers themselves to develop a high-level understanding of the data.

We demonstrate that our proposed method outperforms other baselines, such as DeepInsight ARIMA and Prophet, as well as variations of our proposed method. Our study shows that our method is able to learn high-level knowledge jointly over multiple assets, and produces better prediction accuracy compared to either learning each asset independently.

The AI landscape in the finance sector will become trickier moving forward. Estimates vary wildly but general forecasts predict the financial sector's AI global market share will be between $130 billion and $170 billion by 2025 or 2026. The overall AI share will be close to a trillion USD by the end of the decade, according to various estimates.

Advancements are becoming incremental and are expected to become even more so as academic institutions, private enterprise, and corporate research arms (often some hybrid of the aforementioned) flood the market with technical breakthroughs.

This is from the June Bloomberg article:

As increasing industry competition whittles down returns in core strategies, quants are vying ever-more to deploy programs that learn statistical patterns in equities in order to cut trading costs and find new investing signals.

Predicting share movements one or two milliseconds before everyone else does has been the goal of strategies such as statistical arbitrage and exchange colocation for more than a decade. Yet leveling computational firepower at stock prices is a crowded field, with an entrenched arms race among the biggest shops ensuring that no technical advantage lasts long.

But what about this sea change in business culture? Companies in the finance sector are looking at a new way of operating that may demand more of them psychologically and organisationally than the cataclysmic digital shifts from the trading floor to the high-tech cube farm. A recent McKinsey article looked at the demands placed on companies, particularly their chief officers, as AI augments capabilities. Like many other reports and analyses, the article suggests that AI, unlike much that came before it, cannot be applied opportunistically.

In order to stay in the race, those at the top must shepherd a new era. AI cannot be shoehorned into a solution like an old-timey circuit board or piece of commercial software. In order to harness AI’s game-changing potential, firms will need to employ a new kind of focus that is both sharper and more holistic. AI works well when an application targets specific problems where progress is possible and definable. And problem-solving itself works well when AI is built thoughtfully across an entire organisation.

Says the McKinsey article:

By contrast, leaders at AI-enabled companies take a more systematic view, focusing on their company’s multiple, an indicator of long-term ability to add value to the organization. This requires company leaders to agree that the purpose of AI is to fundamentally transform the way the business conducts its day-to-day operations. In practice, that means using AI in the end-to-end process of capturing every event or data point from customers, processes, or machines (a k, transaction, milestone, indicator, or sensor) to ensure that consequent actions, decisions, and interactions are more focused and effective.

The most important shift for AI-enabled companies is to develop a global learning loop across the organisation. In other words, the organisation should change as a whole, step-in-step with individual trajectories. This requires documenting experiences: trials, errors, and both individual and team growth. There is the advent of the AI nerve centre, the newfangled nucleus of the restructured AI-enabled organisation, and the flattening of hierarchical structures to give frontline teams more ownership and incentive for innovation.

It’s a lot to wrap one’s mind around.

The Brookings Institution reminds everyone to keep their feet on the ground. In a July report dubbed “Why AI is just automation”, the think tank emphasises that human employees are still the core decision-makers and true “nerve centre” of American business, that the technology is not in the driver’s seat, or at least it shouldn’t be:

Processes should be automated when the outcomes are extremely valuable to humankind or solve a problem, not just because the technology exists. Technology application is best poised for success when it is driven by a problem, rather than a solution.

The relationship between input fact patterns, decisional rules, and outputs of a component or outcomes in a system can be perfectly recorded for later review. Just as automation enables speed and scale for decisions, it can also enable a complete form of oversight, transparency, and enforcement.




Tamara Kerrill Field’s writing and commentary on the intersection of race, politics and socioeconomics has been featured in USNews & World Report, the Chicago Tribune, NPR, PBS NewsHour, and other outlets. She lives in Portland, ME.