How Do You Extract Structured Data From a Legacy System With No Bulk Export?

Insurance carrier portals and utility billing platforms hold exactly the data modern products need: policy details, coverage limits, claim status, billing amounts, and usage history. Many of them offer no API and no bulk export. Deck is built for that gap, extracting structured data from these portals through the same login screen a policyholder or account holder uses.

What actually works in production for pulling structured data out of a portal with no export?

Deck works in production because it handles the three things that sink most other approaches: login and MFA, pages that change without notice, and data that arrives as PDFs instead of tables. Deck is a computer use agent platform that automates workflows by operating any web interface directly, so the target system never has to expose an API for it to work.

Every Deck result comes back as schema-validated JSON, so the output is deterministic even when the source portal is not.

Here is how a Deck extraction runs:

1. Your application runs a Deck task through Deck’s API against the user’s stored credential. The task defines the output schema.
2. A Deck agent logs in with the user’s own credentials. If the portal asks for an MFA code, Deck forwards the prompt to the user or your app.
3. The agent navigates to the data, paging through claim history or billing periods as needed.
4. Deck extracts the fields and validates them against your schema. With storage turned on, Deck also captures PDFs and extracts their fields.
5. Structured JSON returns to your application, with download links to any captured documents.

The same Deck task can run across many carriers or utilities, so adding a new portal does not mean writing a new scraper.

Why are insurance and utility portals so hard to extract from?

A login-gated legacy system is any application or portal that holds valuable data behind an authentication flow designed for a person, with no programmatic export path. Insurance and utility portals sit at the hard end of that category for four reasons:

• Session-based access: username and password, MFA, CAPTCHA, and sessions that expire after minutes of inactivity.
• User-owned data: the data belongs to an individual policyholder or account holder, so extraction happens inside that user’s session, not a shared company instance.
• Inconsistent presentation: one carrier renders coverage limits in an HTML table, and another buries them in a downloadable PDF.
• No bulk export: you get a rendered page, a PDF, or nothing.

Web-based ERP modules, government benefits portals, and internal business tools share most of these traits. That is why Deck treats them as one problem with one approach.

Which use cases need this kind of extraction?

Four use cases show up repeatedly:

1. Claims status tracking: pulling claim status, adjuster notes, and payment status from carrier portals into a claims or customer service system.
2. Policy and coverage verification: confirming active coverage, limits, and effective dates during a quote or a service request.
3. New customer onboarding: reading existing policy or utility account data at signup instead of asking a person to transcribe it.
4. Cross-account visibility: giving users one view of insurance policies and utility accounts that otherwise live in separate portals.

Each one depends on the same capability: reliable, structured access to a system that never offered an export.

Does screen scraping work on these portals?

Screen scraping is an HTTP client requesting pages and parsing the returned HTML. It works on simple portals with stable markup and data rendered directly in the page.

It fails on many insurance and utility portals. Modern pages rely heavily on JavaScript: the 2024 Web Almanac found the median mobile page loaded 558 KB of it in 2024, up about 14% from the year before. On a page that renders its data with JavaScript, a plain HTTP client receives only the page shell, so the data never arrives. Bot detection, PDFs, and markup changes finish the job. Deck runs a real browser session, so JavaScript-rendered data, PDFs, and changed layouts all reach the extraction step.

What about building browser automation in-house?

Playwright and Puppeteer control a real browser, so they render JavaScript, keep cookie-based sessions, and work with dropdowns, date pickers, and modals. That makes them a real step up from scraping.

The tradeoff is maintenance. Navigation logic per portal is real engineering work, and it often breaks when a portal changes its layout. For one or two portals, a team can keep up. For dozens of carriers or utilities, it becomes a full-time maintenance function. Deck removes that function, because Deck agents read each portal at runtime instead of relying on per-portal selectors your team maintains.

How do you handle MFA, PDFs, and session timeouts?

These three problems show up in most extraction projects in this category:

• MFA: a user-initiated first session, where the person completes MFA directly, is the most reliable starting point. Stored TOTP secrets work for later runs. SMS interception is a fallback, not a plan.
• Data delivered as PDFs: policy documents, explanation of benefits statements, and billing summaries often exist only as PDFs, so the pipeline needs a parsing step for documents.
• Session timeouts and pagination: insurance portals often expire sessions within minutes, and utility portals spread billing history across many pages. Treat extraction as a state machine that verifies each page before moving on, not a linear script.

Deck handles all three inside the platform: MFA prompts are forwarded to the user through a built-in interaction, saved login state lets later runs skip repeat logins when the portal allows it, and document capture and extraction run as part of the task. That removes all three as separate engineering problems for every new portal you add.

How does Deck keep extraction reliable when a portal changes?

Deterministic output from a non-deterministic source is the core reliability problem in this category. Deck solves it by separating the two: agents adapt to whatever the portal shows, and schema validation guarantees what your application receives.

When a carrier redesigns its claims page, the Deck agent reads the new layout and still returns the same fields. When a run fails, Deck reports why, such as rejected credentials or data it couldn’t extract. You can rerun it with its original input, and session replay records screenshots and logs so your team sees exactly where it stopped. Schema validation is what turns a portal that keeps changing into a data feed whose shape stays the same.

How do the three approaches compare?

FAQs

How do you pull claims data out of an insurance carrier portal with Deck?

Deck logs into the carrier portal with the policyholder’s or adjuster’s own credentials, navigates to the claim, and extracts status, amounts, and documents. Deck returns the result as schema-validated JSON through its API.

Does Deck work for legacy systems outside insurance and utilities?

Yes. Deck applies the same approach to any login-gated web system that never offered an export, including web-based ERP modules, government portals, and payroll platforms.

How does Deck extract data that only exists as a PDF?

With storage turned on, Deck captures the document during the session, extracts the fields defined in your extraction schema, and returns them with a download link to the source PDF. A PDF-only policy document does not require a separate manual retrieval step.

Is Deck a replacement for in-house Playwright scripts?

For one stable portal, a Playwright script can work. Across many carrier or utility portals, Deck replaces the per-portal maintenance that makes in-house scripts expensive to scale.

What is the difference between screen scraping and Deck?

Screen scraping parses static HTML and struggles with JavaScript-rendered pages and bot defenses. Deck operates a real browser session the way a user would, handling dynamic content, authentication, and structured extraction as a managed platform.