S&P Global Market Intelligence Powers 'Workbench Data Apps' Commercial Offering using Plotly with Databricks

Introduction

S&P Global Inc. is a publicly traded financial services and analytics corporation headquartered in New York City. The organization's business units, including S&P Global Market Intelligence, provide intelligence in a wide range of areas such as sustainable finance, capital markets, artificial intelligence, supply chain, and banking.

S&P Global Market Intelligence is responsible for providing real-time data, research, news, and analytics to investment and commercial banks, insurance companies, wealth managers, corporations, and other clients. The business unit delivers workflow solutions and products with the aim of generating and communicating insights to its clients.

Challenge

S&P Global Market Intelligence delivers dozens of solutions, each fed by hundreds of different datasets. The team's primary goal is to find a way to sift through these massive datasets to serve faster insights to their users.

The team develops production-grade data applications to communicate impactful insights but also needs to incorporate continuous user feedback. For example, a data app must be able to deliver insights to a business user more familiar with a Web platform or Excel, as well as providing a blueprint to analysts and data scientists in a notebook environment.

Solution

The S&P Global team crafted a series of visual analytics data applications that empower users to analyze deep data based on specific workflows such as trending topics from earnings call transcripts.

The group, spanning personas including UI/UX, designers, development, engineering, and product, worked to create S&P Global Capital IQ Pro Labs, that enables collaboration with clients on new innovative initiatives and showcases beta applications and solutions allowing clients to explore, experiment, and provide feedback. To better manage these apps, they are also in the process of building an internal homepage for their team to easily access each application.

Amongst the multiple apps built by the team, one of them focuses on textual data generated from earnings call transcripts — an earnings call is an event where the management of a public company discusses their financial results, usually quarterly or annually. The application allows users to search, aggregate, and visualize keyword themes that appear across various earnings call transcripts. Users are able to filter by company sector, market index, and specific keywords in a simple, user-friendly interface to derive insights from large amounts of text-based data.

Another application extracts specific topics from aggregated transcripts to detect patterns and themes. A model following the unsupervised learning AI technique maps key phrases to topics. The application interface allows users to quickly navigate from a specific keyword to the exact point in a transcript where it occurred, for easy access to additional context within the transcript.

All applications are fed with S&P Global's data, stored in Databricks on AWS Delta tables for optimization and performance, with Databricks Workflows for data workflow orchestration. Data is processed on the AWS-hosted back-end infrastructure with Databricks SQL Serverless for scalable data access. A next series of apps will further leverage Databricks Unity Catalog as the governance solution of choice for AI applications and will also incorporate custom LLM models.

Results

By using Plotly's Dash Enterprise in conjunction with Databricks, S&P Global Market Intelligence was able to quickly develop, deploy, and advance data applications to solve specific problems and deliver insights to their clients. 

They were empowered to achieve the following results:

• 80-90% faster time to commercial productization reducing the time taken to move from concept ideation to actionable analytics from nine months to one-to-two months on average.
• Building tailored, sector-specific data applications used daily by between 40-50 clients.
• Using Capital IQ Pro Labs as a home of innovation, developing multiple concept applications and in turn easing the path from development to production, for the apps that provide maximum value to S&P Global clients.

The team plans to continue validating its models to provide fast, easily navigable insights to its clients. This rapid iterative development process using Dash Enterprise enables the ability for the S&P Global team to continuously incorporate customer feedback and add enhancements for new use cases.