← → to move  ·  S show / tell  ·  G graphic  ·  F present full-screen
Red Hat · Four Inc. · Carahsoft — Virtual Event · September 17, 2026

AI foundations for intelligent tax administration

What predictive AI, generative AI and LLMs actually do — then how Red Hat turns models, agency knowledge and automation into an operable mission capability, proven in eight labs you run yourself.

Red HatFour Inc.Carahsoft
AI foundations · no alphabet soup

Four different jobs get called “AI”

Treat AI as a toolbox, not a magic employee. Pick the tool by the job you need done.

Estimate
Predictive AI
data → information
Finds patterns in historical data to score, classify or forecast an outcome.
Analogy
A weather forecast: “there is a 70% chance,” not a guarantee.
Create
Generative AI / LLM
information → knowledge
Produces new language by predicting useful next tokens from patterns learned during training and context supplied now.
Analogy
A very well-read drafting assistant — fast, fluent, and still capable of being wrong.
Reason + act
Agentic AI
knowledge → action
Pairs a model with approved tools, state and rules so it can plan steps and call systems.
Analogy
A junior analyst with a badge and a restricted tool belt — permissions still matter.
Execute
Automation
executes judgement
Runs deterministic, tested steps when a condition or human decision says to act.
Analogy
A checklist that actually performs the checklist. This is where Ansible belongs.
What predictive AI is not: a crystal ball.   What an LLM is not: a database, policy authority or source of truth.   What an agent is not: permission to bypass controls.
The operating reality

The work is seasonal, textual and unforgiving

A year of demand arrives in six weeks Taxpayer contacts per week — illustrative shape of a filing year staffed capacity April — 6× the August low the surge automation absorbs extension deadline JFMAMJJASOND
Demand

Peak load, fixed staff

Idle capacity in August is unjustifiable; exhausted capacity in April is a headline.

Systems

Rules locked in legacy code

Rewriting the systems of record is a decade. Wrapping them in intelligent workflows is a quarter.

Data

The answers are in the text

Correspondence, transcripts and case notes are unstructured — exactly what language models are built for.

The constraint that decides everything

The data cannot leave

FTI handling rules rule out public endpoints. The model comes to the data — on premises, accredited cloud, or air-gapped enclave.

data — it stays
Large language model · what is actually happening

An LLM is the engine — not the whole vehicle

Mission appThe taxpayer-service UI, case workflow or console that asks for help. judgement
Context / RAGThe approved guidance for this exact question — the open book. knowledge
Model / LLMLearned language patterns — not your live system of record. information
InferenceThe engine — vLLM, the Red Hat AI Inference Server. data → information
PlatformIdentity, scaling, isolation, lifecycle, policy — RHEL and OpenShift AI. executes judgement
Compute + dataAccelerators and agency data — wherever authorization requires. data

Fine-tuning is studying for the test. Retrieval is bringing the notebook. You want both — and neither makes the model a source of truth.

Why it matters

From data to judgement — and who climbs each step

Every step up is worth more to the mission AI moves the work up the ladder — it never takes the top step Data What happened, uninterpreted returns · transcripts · calls · scans Information Data in context for one case matched · flagged · routed by intent Knowledge Information that informs an outcome guidance · precedent · taxonomies Judgement A determination someone signs the decision — and the accountability stays human PREDICTIVE AI — score · match · flag GENERATIVE AI + RAG — read · ground · draft PEOPLE — decide & sign
Why it matters

The backlog lives at the bottom of the ladder

Transcribing, matching, routing and summarizing consume the workforce. Every hour a machine absorbs down there is an hour an examiner spends deciding — the only work that resolves a case.

Why it matters

Judgement is deliberately never automated

The model classifies, grounds and drafts; a named person decides and signs. That is a design rule, not a technical limit — and it is what makes the deployment defensible to counsel and oversight.

Why it mattersone graphic — take it home

Predictive and generative AI answer different questions

They fail differently — so they are governed differently.

Predictive AI a number from structured history score · forecast · flag Generative AI language from text grounded and cited read · draft · cite the strongest workflows chain both: score, then explain a score is not a determination a person decides a fluent sentence is not evidence Predictive AI a number from structured history learns from patterns in filings, payments and outcomes produces a number: a score, a forecast, a flag in tax terms refund-risk scoring, workload forecasting, anomaly flags governed by back-testing, thresholds, fairness metrics Generative AI language from text learns from how guidance, notices and case notes are written produces a classification, a draft, a summary, a cited answer in tax terms read correspondence, draft notices, summarize case files, cite guidance governed by retrieval with citations, human sign-off, drift monitoring: Labs 04, 06 the strongest workflows chain both: score, then explain both in Lab 03 rung Information data → information rung Knowledge information → knowledge fails as a bad number a score is not a determination the human check a person decides fails as a plausible sentence a fluent sentence is not evidence
Workflow map

Where AI actually lands in the filing lifecycle

AI lands on the first four stages, where a reviewer already checks the output; the decision stays with a person; operations close the loop.

01 Intake Capture correspondence from any channel AI reads it 02 Classification Understand intent, route to the case type AI classifies 03 Guidance Rules, policy and precedent AI grounds and cites 04 Drafting Letters, responses, memos, with citations AI drafts 05 Review Check accuracy, add judgment, approve human in the loop 06 Decision Finalize and issue the outcome a person signs 07 Notice Send letters and update records automation executes 08 Operations Track performance, learn and improve the loop closes
Rule of thumb

Start where a human already reviews the output

If a person signs the letter today, a model that drafts it changes throughput without changing accountability. That is a defensible first deployment.

judgement stays human
Rule of thumb

Automate the toil around the model, not just the model

Most of the effort in production AI is provisioning, patching, scaling and evidence collection. That is an automation problem, and it is solved.

executes judgement
What teams get

Five outcomes, and where each one is proven

01

Modernize mission-critical operations

Wrap legacy systems in intelligent workflows instead of rewriting them. An error string in a log fires an Event-Driven Ansible rule, a playbook runs on the right hosts, the on-call gets an email, and a ticket opens and closes in ServiceNow or Remedy. The system of record stays untouched.

executes judgement  Ansible Automation Platform · Event-Driven Ansible · OpenShift  →  Labs 05 · 07
02

Improve efficiency and accuracy

Classification, extraction and drafting move first-pass work off people. Reviewers spend their time on judgement, not transcription.

informationknowledge  Red Hat AI Inference Server · SDG Hub + Training Hub  →  Labs 01–02
03

Unlock data-driven insights

Retrieval grounds every answer in your own published guidance and case data, with the source citation returned alongside the answer.

knowledge  Red Hat OpenShift AI · vector store · Llama Stack  →  Lab 04
04

Strengthen security and compliance

Signed models, FIPS-validated cryptography, continuous compliance scanning and bias/drift monitoring — as platform features, not project work.

evidence → judgement  RHEL · Compliance Operator · TrustyAI · Sigstore  →  Lab 06
05

Build an AI-ready foundation

One platform, one operating model, from a datacenter GPU node to an accredited cloud region — so scaling innovation is a capacity decision, not a re-architecture.

data → information  OpenShift AI · KServe · Red Hat AI Inference (vLLM)  →  Lab 03
Reference architecture

One foundation, from bare metal to the taxpayer

Most failures happen at the seams between systems, not inside them. The platform’s job is to make those seams boring.

TRUST Signed models FIPS crypto SELinux Compliance Operator TrustyAI drift + bias Audit trail Air-gap mirroring judgement Lab 06 AUTOMATION Ansible Automation Platform Event-Driven Ansible Provision Patch Remediate Evidence collection executes it Lab 05 data Red Hat Enterprise Linux · AI Inference Server Tested with the hardware: mainframes · TPM · PIV/CAC cryptography Bare metal GPU nodes · virtualized datacenter · accredited public cloud · classified enclave Granite models, accelerator drivers and the vLLM-based AI Inference Server in one supported stack GPU GPU Lab 01 data Red Hat OpenShift Scheduling · GPU partitioning · network policy · multi-site placement · GitOps delivery The same cluster runs the modernized case-management apps beside the models they call Red Hat OpenShift AI Labs 02 · 03 · 04 · 07 information Model serving KServe + vLLM knowledge Training SDG Hub · Training Hub knowledge Pipelines tune · eval · ship judgement Model registry signed versions knowledge Retrieval vector store judgement Mission applications Correspondence intake Notice drafting Case summarization Guidance assistant Systems of record stay where they are — reached through APIs and automation, not replaced Data Information Knowledge Judgement
Why Red Hat AIthe grid below is your take-home reference

The federal problem is not “get a model.” It is operate AI without giving up control.

The Red Hat proposition
Bring the model to the mission — not the mission data to somebody else’s model.
Use the model, accelerator and deployment location that fit the workload. One operating model across datacenter, accredited cloud, edge and disconnected — and a glass box you can open and inspect, not a black box.
InferenceServe efficiently with vLLM / llm-d.
DataConnect private agency knowledge.
AgentsGovern tool-using workflows.
PlatformScale and operate consistently.
Choice knowledge

Model freedom

Open models and standard APIs keep the exit open. Modern lock-in limits what you can build next — a worse trap than the old hardware kind.

Control data

Data locality

Run inference inside the boundary dictated by FTI, mission data, sovereignty and accreditation.

Operations information

Models-as-a-Service

Central platform teams publish governed model endpoints other projects consume — instead of every project building its own fragile AI stack.

Economics data

Use accelerators efficiently

vLLM and distributed inference focus on throughput, latency and cost per token rather than GPU ownership as a trophy.

Lifecycle informationknowledge

Predictive + generative

OpenShift AI manages the lifecycle of traditional ML and generative workloads so they can coexist in one operational model.

Execution executes judgement

Ansible closes the loop

Provision, configure, remediate and collect evidence with deterministic automation around probabilistic AI.

Why Red Hat

Define the service once — run it wherever the mission needs it

One service definitionplacement is policy — data: where it lives decides the venue inferenceservice.yaml model: granite-8b-tax-v1 (signed) delivery: GitOps — main branch Infrastructure as code — the manifest is the capacity plan (Lab 03) oc apply — unchanged same image · same controls · same skills Datacenter — bare-metal GPU nodes Your floor, your hardware, shortest path to the systems of record differs: the iron Accredited cloud region FedRAMP boundary; surge capacity rented for the season, released after differs: the boundary Air-gapped enclave Mirrored registry and models; no internet at inference or update time differs: no internet
Same platform

Accredit once, inherit everywhere

Signing, FIPS mode, compliance scanning and drift monitoring attach to the platform — so workflow two starts with workflow one’s evidence, and the skills and pipelines carry over the same way.

accredit once — judgement
At scale

April becomes a scheduling event

The 6× filing surge is absorbed by autoscaling on pooled GPUs (Lab 03) — capacity follows the season instead of sitting siloed inside one project’s budget.

information at scale
Across clouds

Placement is policy, not re-architecture

Datacenter, accredited region or enclave — moving a workload is a redeploy of the same manifest. Residency and cost decide where it runs, and the exit door stays open.

data — where it lives
Troubleshooting · operations · knowledge

The useful pattern is a loop, not a chatbot

Predict, explain, act: spot the pattern, turn the signals into language, use only what the agency has approved. Grounding and learning are how the loop earns the right to act; the connected estate underneath is the discriminator nobody else has.

01 · PREDICT

Spot the pattern

Telemetry and historical incidents estimate risk: saturation, failure probability, anomaly or likely incident class.

information Predictive ML
→
02 · EXPLAIN

Turn signals into language

The LLM summarizes logs, changes and symptoms into a human-readable hypothesis and next questions.

information GenAI / LLM
→
03 · GROUND + ACT

Use what the agency approved

RAG retrieves runbooks and prior fixes; policy decides whether Ansible recommends, requests approval or remediates. Every action opens and closes an ITSM ticket (ServiceNow, Remedy), so the incident record is the trail. Lab 05.

RAG — knowledge Ansible executes judgement
The knowledge flywheel knowledge

After review, the incident summary, root cause, evidence and successful remediation can become searchable operations knowledge for the next event. Generated content is a draft artifact until it passes the same curation and authority rules as any other knowledge record.

Example

“Why is this service slow?”

Predictive AI says GPU queue depth is abnormal. The LLM explains the evidence. RAG cites the approved saturation runbook. Event-Driven Ansible scales or reroutes after the required approval.

Boundary

Do not let fluency become authority

The LLM can propose a root cause. Telemetry, system state, retrieved source material and deterministic checks establish whether that explanation is actually supported.

The platform on its own data

Point the same pattern at your logs — MTTR is the proof

The Lab 04 retrieval stack, aimed at operations: logs, runbooks, incident history and change records become a private notebook the on-call engineer can question — on the cluster, inside the boundary.

Where the hours actually live Mean time to resolution, decomposed — representative incident pattern, before and after the notebook detect diagnose fix verify before grep · page · guess diagnose — 2h 41m fix 22m 3h 16m with the notebook ask · verify · act 24m fix 22m 59m diagnosis -84% — the notebook answers “what changed, has this happened before, which runbook applies” in seconds, with citations Detect, fix and verify are unchanged — the hours lived in diagnosis, where industry incident studies consistently put most of resolution time. Lab 07 walks it end to end. → Lab 07
Why a notebook, not a dashboard

Dashboards answer anticipated questions

A notebook over your own data answers the one you did not anticipate — “what changed, has this happened before, which runbook applies” — in language, with citations you check before acting.

information
Why it must be local

Logs are FTI-adjacent

They carry hostnames, account fragments and case identifiers. The notebook runs on the same OpenShift install it observes — nothing is exported to anyone’s hosted service.

data
Why it transfers

Same stack as the taxpayer system

The team that grounds answers for operations (Lab 07) has already built the taxpayer-facing version (Lab 04) — one retrieval pattern, two corpora, one governance story.

knowledge
Adoption path

Ninety days, then ninety more

Crawl, walk, run. Nothing here requires a rewrite of a system of record — ground the current phase before you scale, and each phase produces something you can show.

PROVE GROUND SCALE day 0 day 90 day 180 day 270+ one workflow live, measured grounded · monitored · automated surge-proven at scale
Prove dataCrawl · days 0–90
  • Stand up one supported endpoint with the AI Inference Server
  • Agree the accelerator capacity plan and where it physically sits
  • Settle which data classes may be used, in writing
  • Pick one text-heavy workflow with a human already reviewing
  • Measure the baseline before you change anything
Ground informationWalk · days 90–180
  • Move serving onto OpenShift AI with autoscaling
  • Add retrieval over your own published guidance
  • Tune on agency vocabulary with SDG Hub + Training Hub
  • Wire drift, fairness and compliance monitoring
  • Automate provisioning and patching end to end
Scale knowledgeRun · days 180+
  • Promote through a model registry with real approvals
  • Add workflows two through five on the same platform
  • Prove the filing-season surge with a load rehearsal
  • Extend to the disconnected or classified enclave
  • Report outcomes in mission terms, not token counts
Next steps

What happens after this hour

Step zero

Lab walk-through

An hour where we run the seven labs together, show and tell, and find the point in the stack where you want to prove value. That scopes the pilot.

Step one

Architecture workshop

A half-day with your platform and security teams to size accelerators, place the first workload and name the data classes in scope.

Step two

Guided pilot

One workflow, on your infrastructure, with a measured baseline and an agreed definition of success before we start.

Step three

Acquisition path

Four Inc. and Carahsoft carry the vehicles and pricing — bring them in early, not at the end.

Point of contact — Brad will follow up
Brad Scalio

Red Hat · if nothing else, he will get you to the right person

bscalio@redhat.com

IRS account executive: Ted Craig, Red Hat. Everyone else, email Brad and he connects you to yours.

Everything from today — deck, labs, guides — at nommsweymx.github.io/redhat-ai-tax-labs. The seven labs are the commands your engineers will run on day one. Follow it on your own: …/adventure.html.

the next decision is yours
Your move
Already a Red Hat customer? Ask for your Solution Architect.Not sure who that is? Email Brad, open a ticket in the Customer Portal and ask support, or ask through your management line. The foundational questions are free to ask and expensive to skip.
Start a community of practice.A standing group across platform, security and mission teams, built around one workflow. We will help seed it and we will show up.
Bring your hardest question.Air-gapped operation, FedRAMP boundaries, accelerator scarcity, model provenance. If you did not see it here today, that does not mean it does not exist — ask.
Bring your cyber team.If nobody from IRS cyber is on today, refer us to your ISSO or security lead. We will run a dedicated security session for them: the appendix, in depth.
Hands on · the takeaway

Eight labs — real commands, run them yourself

An opinionated path on a personal laptop — not an MVP, never production, never on a government computer. Free of charge and open to anyone: one self-contained HTML on the Carahsoft event page, source at nommsweymx.github.io/redhat-ai-tax-labs. Prove it to yourself: watch the model serve on 127.0.0.1.

Lab 01 · Inference Server

Serve a model inside your boundary

Pull the vLLM-based inference server, serve a validated Granite model, and classify a taxpayer inquiry — without a single byte leaving the host.

data → information
Lab 02 · Customization

Teach it your notice taxonomy

Add agency knowledge as seed YAML, generate synthetic data with SDG Hub, tune with Training Hub, and evaluate against the base model.

knowledge
Lab 03 · OpenShift AI

Survive filing season

Deploy with KServe and vLLM, then watch it autoscale from two replicas to twenty under an April load rehearsal.

information at scale
Lab 04 · Retrieval

Ground answers in your own guidance

Ingest a publication corpus into a vector store and return answers that carry their citation.

knowledge
Lab 05 · Ansible

Automate the toil around the model

An Event-Driven Ansible rulebook detects inference saturation and remediates it before anyone opens a ticket.

executes judgement
Lab 06 · Trust

Prove it to your ISSO

Verify the model signature, check FIPS, run a compliance scan and read drift metrics off live traffic.

evidence → judgement
Lab 07 · Ops notebook

Ask your own logs

Ingest logs, runbooks and incident history into an on-cluster notebook, diagnose a live incident with citations — then read the MTTR numbers.

information → knowledge
One thing to remember

Nothing here needed a public endpoint

No public model endpoint, no internet connection at inference time, no rewrite of a system of record. The labs run upstream bits on a laptop; an agency accredits the supported Red Hat product. Lab 08 chains the whole loop.

Lab 08 · the whole loop

One loop, end to end: from mainframe logs to an assistant that probes

Logs in, risk scored, fix drafted by Lightspeed, merged by a person, run by Event-Driven Ansible, remembered by the platform, so you can ask.

Ask the platform
How is server1-taxreturns?
Online, up 41d, mem 62%, 72h risk 0.22 and falling; last change CR-9088. [job 40518 · journald · RB-214]
How is server8-audits?
Restored 03:13 by job 40512 (Xmx 7g); risk 0.91 → 0.34; PR #482 merged, INC-2340 closed. [job 40519]
Which servers are showing online?
13 of 14: server1..12 ok, MVSA reachable, MVSB in its IPL window until 06:00. [job 40520 · estate ping]
Takeaway

The loop is the documentation

Every promoted change, its review thread and its job outcome are indexed where the assistant searches. The KM system writes itself.

Takeaway

Two kinds of AI, one platform

The scorer and the drafter share a GPU node, a registry and an identity. Predictive and generative are pods, not projects.

Takeaway

Promotion is a merge

Repo, AAP and knowledge base update together, and a person decides. Nothing runs from a chat window.

Takeaway

Ask, then probe

The assistant's tools are approved job templates, so "how is server8?" becomes a job with evidence, not a guess.

Appendix · Trusted, enterprise-ready AI

The questions an authorizing official will ask

Evidence is information — the authorizing official's decision is judgement. The platform produces the first so a person can make the second.

Their questionThe platform controlWhere it livesProven in
Where does our data go when someone prompts the model?Nowhere. Inference runs on your cluster, in your enclave, on your accelerators.AI Inference Server · OpenShift AILab 01 · 127.0.0.1, no egress
Can you prove this model is the one we approved?Model artifacts are signed and verified before they are admitted to the cluster.Sigstore / cosignLab 06 · cosign verify
Is the cryptography validated?FIPS mode is set once in RHEL CoreOS and inherited up the chain by OpenShift and OpenShift AI; where a component is not inherited, the exception is documented and covered by compensating controls.RHEL FIPS modeLab 06 · fips-mode-setup, sestatus
Does it work with our smart cards and hardware crypto?Tested with the hardware vendors: TPM, PIV/CAC smart-card authentication and hardware cryptography supported from the operating system up, on mainframes included.RHEL · hardware certificationSlide 09 · base band
Can this land in our FISMA moderate or high boundary?Platform controls map to the NIST 800-53 moderate and high baselines; your FIPS 199 categorization picks the venue — the platform is the same in all of them.RHEL · OpenShiftSlide 11 · one manifest
How do we know it stays compliant next quarter?Scheduled scans against a hardening profile, with machine-readable results.Compliance OperatorLab 06 · compliancescan
What about the applications and containers we build on it?The same tooling spans your own application lifecycle on the platform: signing, scanning and admission policy at build, at deploy and at run time.OpenShift build · registry · admissionLab 06 · signature gate
How do we detect the model drifting or skewing?Continuous drift and fairness metrics on live inference traffic.TrustyAILab 06 · drift, fairness
What if we have no internet at all?Mirror images and models into the enclave; the platform is built for disconnected operation.oc-mirror · registrySlide 11 · enclave
Who is accountable when it is wrong?A named reviewer on the signature, and every automated action launched from one governed console under enterprise RBAC, so each action carries the identity that ran it.Workflow design · Ansible Automation PlatformLab 05 · job history