Enterprise AI Insight

How BlackRock Builds Custom Knowledge Apps at Scale

From 8 Months to 2 Days: How BlackRock Built a Framework to Compress AI App Development Time for Complex Financial Document Processing

"We took this - an app took us close to like 8 months somewhere between 3 to 8 months to build a single app for a complex use case and we able to compress time bring it down to like a couple of days."

— Infant Vasanth, Director of Engineering, BlackRock

Watch (00:16:40)
3-8 months → 2 days

Development Time: Time compression for document extraction apps

$10T AUM

Scale: BlackRock's assets under management

10,000 pages

Document Scale: Extreme cases in financial documents

2-4 people

Four Eyes Check: Regulatory requirement for financial operations

Executive Summary

At the AI Engineer Summit, BlackRock engineers Infant Vasanth and Vaibhav Page reveal how the world's largest asset manager built an internal framework to compress AI app development from 3-8 months to just a couple of days. Their focus: document-heavy investment operations where teams need to process thousands of prospectuses, term sheets, and regulatory documents to set up securities for trading.

The core insight is that domain experts—not AI engineers—should be in the driver's seat. BlackRock's 'Sandbox + App Factory' framework puts the power of prompt engineering, LLM strategy selection, and workflow design directly into the hands of investment operations teams. This shifts the bottleneck from engineering capacity to domain expertise, dramatically accelerating iteration and deployment.

However, the talk delivers a crucial reality check for enterprise AI: fully autonomous agents aren't ready for highly regulated industries. Financial services requires 'four eyes check' (human review by two independent people), making human-in-the-loop design a non-negotiable requirement, not a nice-to-have. The team also addresses practical challenges like 10,000-page documents exceeding token limits, prompt complexity growing from sentences to paragraphs, and infrastructure decisions between GPU clusters and burstable compute.

Key Themes

Domain Experts > AI Engineers

The bottleneck for enterprise AI apps is not AI engineering talent—it's giving domain experts the right tools.

By building a framework that lets investment operations teams configure extraction templates, design validation rules, and iterate on prompts without writing code, BlackRock shifted from engineer-led to expert-led development.

Human-in-the-Loop is Non-Negotiable

In regulated industries like financial services, 'four eyes check' makes autonomous agents a non-starter.

Design for human-in-the-loop first. The temptation to 'go all agent tech' conflicts with compliance requirements. AI should propose, humans should approve.

There Is No One-Size-Fits-All LLM Strategy

Different document sizes and instrument types require different strategies: in-context learning, RAG, or hybrid approaches.

Small documents (<50 pages) work with in-context prompting. Medium documents (50-500 pages) need RAG. Large documents (500+ pages) require hybrid strategies. You must mix and match.

Prompts Grow From Sentences to Paragraphs

As you handle edge cases in complex domains, prompts inevitably grow from simple instructions to multi-paragraph conditional logic.

What starts as 'extract issuer and maturity date' becomes three paragraphs once you account for callable bonds, puttable bonds, convertible bonds, perpetual bonds, and their interdependencies.

AI Must Prove ROI, Not Just Possibility

Enterprise deployment requires honest ROI calculation: is an AI app cheaper than an off-the-shelf solution?

All of this is great in experimentation mode, but production requires evaluating whether spinning up an AI app is actually more expensive than buying an existing tool.

Top 10 Quotes from the Talk

"We took this - an app took us close to like 8 months somewhere between 3 to 8 months to build a single app for a complex use case and we able to compress time bring it down to like a couple of days."

— Infant Vasanth, Director of Engineering, BlackRock

00:16:40
"We all are like really tempted like let's go all agent tech with this but in the financial space with compliance with regulations you kind of need those four eyes check and you kind of need the human loop."

— Infant Vasanth, Director of Engineering, BlackRock

00:17:25
"Your prompt itself in our simplest case like started with like a couple of sentences before you knew it you're trying to describe this financial instrument and it is like three paragraphs long."

— Infant Vasanth, Director of Engineering, BlackRock

00:07:55
"Some documents are like thousands of pages long 10,000 pages long now suddenly you're like oh okay I don't know if I can pass more than a million tokens into say uh the open AI models what do I do then right"

— Infant Vasanth, Director of Engineering, BlackRock

00:09:45
"We tried this with agentic systems doesn't quite work right now because of the complexity and the domain knowledge that's imbued in the human head."

— Infant Vasanth, Director of Engineering, BlackRock

00:08:30
"All of this is great in experimentation and prototyping mode but if you kind of want to bring this you have to really evaluate what your ROI is and as is it going to be like more expensive actually spinning up an AI app versus just having like an offtheshelf product that does it quicker and faster."

— Infant Vasanth, Director of Engineering, BlackRock

00:15:40
"Invest heavily on your like prompt engineering skills for your domain experts especially in like the financial space and world. Defining and describing these documents is really hard."

— Infant Vasanth, Director of Engineering, BlackRock

00:14:55
"If you're going to do that, I probably have to have like a GPU based inference cluster that I can kind of spin up, right? [...] What I do instead is I use like a burstable cluster, right?"

— Infant Vasanth, Director of Engineering, BlackRock

00:13:20
"We kind of have to build this tool for the investment operations team, right? To set up a particular security."

— Infant Vasanth, Director of Engineering, BlackRock

00:04:10
"Educating the firm and the company on what an LLM strategy means uh and how to actually fix these different pieces for your particular use case."

— Infant Vasanth, Director of Engineering, BlackRock

00:15:15

Key Takeaways

1. Invest in Prompt Engineering

For Domain Experts

  • •Prompt complexity grows from sentences to paragraphs
  • •Domain experts need prompt engineering skills
  • •Describing financial documents is inherently complex
  • •Edge cases (callable bonds, etc.) explode prompt size

2. Educate on LLM Strategies

Enterprise Alignment

  • •No one-size-fits-all LLM strategy
  • •Small docs: in-context learning
  • •Medium docs: RAG
  • •Large docs: hybrid approaches

3. Evaluate ROI Honestly

Production Readiness

  • •AI must justify ROI vs off-the-shelf
  • •Experimentation is easy, production is hard
  • •Cost includes infrastructure, not just API calls
  • •Consider total cost of ownership

Research sourced from AI Engineer Summit 2024. Analysis of BlackRock's approach to building custom knowledge apps at scale using human-in-the-loop design, LLM strategies, and modular frameworks. All quotes verified against original transcript.

Learn more about BlackRock →