01 — Introduction

The single source of truth for well construction data.

AURA Data Hub unifies real-time drilling, completions, and financial data from an operator's daily reporting, real-time analytics, historical archive, master data, and AFE systems into one PostgreSQL schema, exposed through a 95+ endpoint REST API — so BI tools, engineers, and integrations query one platform instead of reconciling five.

379+
configured data types in a single flexible-storage table
95+
REST endpoints across 6 functional domains
280M+
records under management across five sources
22yrs
of operations history — 7.3M Well Operations Archive records alone
99.9%
uptime SLA in production
* Figures as of v2.6.53 (2026-07-14), from a live production deployment at a major energy operator. Record counts from aura_data: Analytics 257.6M (incl. WITS telemetry), Well Operations Archive 7.3M (22 years of operations), Operations Reporting System 3.9M, AFE Design 0.5M, plus 11.7M derived — of which ~3M business-active rows drive daily BI. See Chapter 04 for the storage model behind these numbers.
02 — Functionality

What AURA gives every kind of user

One platform, multiple front doors — a portal for day-to-day users, an admin console for platform operators, automated data-quality monitoring that runs without anyone watching, and a notification layer that reaches ops before they have to ask.

Portal

The day-to-day front door for engineers and operations — six sections, Azure SSO gated.

  • Dashboard — KPIs & summary stats
  • Explorer — drill-down by well, date, phase
  • Reports — export to Excel / CSV / Parquet
  • Operations Reporting System Refresh, Data Quality, What's New
Live aura_user_ui

Admin + Data Editor

Platform operator console — who can see what, and the master data everyone else depends on.

  • User & Role Management — ReadOnly / DataManager / Developer / System
  • Permissions — field-level access control, down to a single locked column
  • Audit Logs — who queried what, when, how many rows
  • Master Data Editors — Wells, Rigs, AFE Accounts, DQ config
  • What's New Admin — CRUD for upgrade announcements
Live aura_admin_api

Data Quality Monitoring

Six automated detectors run on schedule — no one has to go looking for a broken import.

  • Schema Drift — new Operations Reporting System columns detected
  • AFE Items Mismatch — unknown account codes
  • Fracs Missing Stage, Costos Audit, Date Boundary, NPT Validation
Live FARO → @AURA_DATA_MANAGEMENT

Notifications (FARO) *

Every failure and every DQ issue reaches an inbox automatically — email and WhatsApp, no dashboard-watching required.

  • Data quality issue alerts — schema drift, missing data, validation errors
  • Import failure alerts — Operations Reporting System/Analytics API down, partition errors
  • Health status emails — backup health, import lag, lock contention
Live notifications.roderickc.com
AURA Admin portal landing dashboard after login: KPI cards for 38 users, 51 active API keys, 769,264 audit events in the past 7 days, and a connected cache; the Quick Launch grid; and the full left-navigation taxonomy across Data Operations (Well Master, Rig Management, Data Editor, Data Quality, Data Exports, Operations Reporting System Config, Operations Reporting System QA Validation, Operations Reporting System Refresh), Access Control (Users, Roles, Permissions), Monitoring (Audit Logs, Audit Performance, Usage Analytics, API Traces, Endpoint Tests), System (Services, System Status, System Map, Cache, Configuration), and Resources (Documentation, API Docs).
The Admin portal front door: live KPIs (38 users, 51 active API keys, 769K audit events/7d) and the full operations taxonomy — data operations, access control, monitoring, and system management in one place.

Access control isn't just role-gated pages — it goes down to the individual field, with locked-field enforcement covered in Chapter 09 — Data Governance.

95+
endpoint modules across 6 functional domains
34
public reporting / completions / AFE / drilling endpoints
30+
admin endpoints — users, roles, audit, master data
Auth: Azure SSO (Microsoft Entra) for operator and RCI staff — corporate accounts sign in via OIDC, roles gate section visibility across both the Portal and Admin Portal.
AURA What's New changelog: versioned entries such as 2.4.2 with What We Did, Why, Customer Benefit, Effort, Source, and Status columns, and category tags including Improvement and Feature.
What's New — the product-evolution log. Every shipped change is recorded with its customer-facing benefit, not just a commit message.
03 — Architecture

Where the data lives, and what moves it.

AURA is a small set of services around one PostgreSQL database — a public API, an admin API, an import engine, and a cache — behind a single gateway. The interesting part is the edge, where agents inside the customer network let AURA reach systems it can't directly see.

The runtime, in one picture

One entry point, a handful of services, and one database that everything reads from and writes to — the same aura_data table the next chapter unpacks in full.

Clients → gateway → app services → one database
CLIENTS BI tools & Portal/Admin UI Excel/Power Query, external partners GATEWAY NGINX gateway single entry · SSL/TLS · rate limit APP SERVICES Data API public /api, reporting reads Admin API /admin, master-data writes Import engine scheduled source writes Redis cache hot reads, session locks THE DATABASE aura_data one partitioned PostgreSQL table 379+ types, ~1,000 partitions every path reads & writes here see Chapter 04 — Flexible Storage

The edge: reaching systems AURA can't see

The architecturally interesting part isn't the database — it's the edge. AURA never bridges into the operator's network. Instead, a lightweight agent runs inside the customer's VDI (on the operator's VDI), pulls what AURA needs from systems like Well Data Master's Oracle database, and pushes results back out. AURA initiates nothing behind that boundary; the agent always calls out, never the other way around.

AURA ↔ VDI agent — pull-then-push, never a bridge in
AURA outside the customer network never bridges in polls out carries result back VDI agent inside the operator's network the operator's VDI always calls out Well Master Oracle read Well Files write

And the platform documents itself

The diagram above isn't a slide someone drew once and forgot — AURA's own admin console generates the same picture live, straight from its running services.

AURA Admin System Map, Overview tab: a layered architecture diagram drawn live by the platform itself. Layer 1 Clients (Web Browser, Mobile Client, API Client, External Partner) connect through Layer 2, an NGINX Reverse Proxy handling SSL/TLS, load balancing, and rate limiting, which routes to Layer 3 Application Services: Data API, Admin API, Import Service (Operations Reporting System sync, Analytics integration, scheduled jobs), and Background Jobs.
The same architecture, drawn by the platform itself. This isn't a slide — it's a live view generated by AURA's own admin console, proof the picture above matches what's actually running.
* Architecture facts from the platform's runtime topology documentation (services, ports, PostgreSQL partitioning, Redis, systemd, NGINX routing) and the VDI outbound-only pull-push agent model. Screenshot: AURA Admin System Map, captured 2026-07-14.
04 — Flexible Storage

Add a new data source with zero code and zero migration.

AURA's core architectural bet: one physical table, one JSONB column, config-driven ingestion. A brand-new Operations Reporting System vista can be live and serving BI reports in under an hour, because there is no Python to write and no schema to migrate — only JSON to edit.

Sources → the aura_data table → consumers

aura_data storage architecture — three bands
SOURCES THE TABLE CONSUMERS Ops Reporting API 15-min Analytics API continuous Well Archive warehouse Well Master · well master daily ART AFE Design · AFEs event aura_data PostgreSQL, LIST partitioned data_source data_type data  jsonb external_id last_updated 379+ types ~1,000 partitions /data/{data_type} generic endpoint BI Tools Domain endpoints

Config, not code — the 5-step workflow

Adding a new Operations Reporting System vista (example: miestacion) never touches Python. Edit JSON, pre-create the partition, deploy the config, trigger an import — the data is live the same minute.

1 Edit config JSON
2 Pre-create partition
3 Deploy config
4 Trigger import
5 Live via /data/{data_type}

No migration, no restart, no code review cycle. The data lands in aura_data(data_type='miestacion') and is queryable the same minute — automatically, through the generic endpoint, before anyone writes a single line of domain-specific code.

aura_data/config/config_ops_reporting.json step 1
{
  "endpoints": [
    ...
    {
      "path": "/api/vistas/miestacion/",
      "data_type": "miestacion",
      "id_field": "miestacionid",
      "last_updated_field": "lastmodificationdate",
      "initial_suffix": "pagina",
      "incremental_suffix": "modificados-desde",
      "query": { "size": 1000 },
      "limit": 1000,
      "categories": ["data_priority"] // HIGH_PRIORITY or optional
    }
  ]
}
AURA Admin source-configuration page, a healthcheck on data sources: 184 vistas, 190 configured, 0 unconfigured, 6 stale (not in API). A table lists each configured vista with its categories and a per-vista HEALTH column, with Verify All vs PROD, Verify All vs QA, and JSON Config controls.
184 vistas, 190 configured, 0 unconfigured, 6 stale. The config JSON from step 1 isn't a toy example — this is the live registry it belongs to, one row per data source, each mapped to its identity and freshness fields. The registry isn't just configuration — it continuously knows whether each source is healthy and in sync across environments, one click from a PROD/QA verification and the raw JSON.

The wow moment

A new Operations Reporting System vista live and serving BI reports — zero code changes, zero schema migration.

<1 hour

Five patterns that make this work

Identity

__aura_external_id

A stable, deterministic business key set on every record, built from source fields — enables BI delta-merge without re-querying the database.

stages_planned = int(asset_id) × 1000 + stage_number
Shape

Field templates

JSON mapping files normalize Operations Reporting System's Spanish field names to BI-friendly English, and switch between business and raw shapes per request.

cuenta → account_code  |  ?field_template=BI|raw
Units

Imperial → metric

Analytics delivers imperial units (feet, bbl/min); Operations Reporting System is metric-native. AURA normalizes everything to metric at the field level, so BI always sees one consistent unit system.

depth: ft (Analytics source) → m (API response)
Time

Timezone handling

Source timestamps arrive in Argentina time (ART); AURA normalizes to UTC in storage, and reporting uses a configurable 6 AM cutoff so a “day” matches the rig’s operational day — sync windows and Well Data Master write-backs are scheduled and formatted in ART (es-AR).

ART → UTC (store)  |  6AM cutoff → operational day
History

The Ledger

Rows are never overwritten or physically deleted. A new version supersedes the old one (status: active → superseded/deleted), and both the source system's timestamp and AURA's own storage timestamp are preserved — so any query can be replayed “as of” a moment, and any discrepancy audited.

status: active | superseded | deleted · last_updated + internal_last_updated
AURA Admin Data Editor grid on a populated table, Event Catalog: columns for event letter, conventional flag, code, type, and active, with search, sort, export, and per-row view, edit, and delete actions.
Every stored table, one editor. The flexible-storage model isn't just for ingestion — the same generic Data Editor lets an operator view, search, and correct any configured table without a bespoke admin screen per data type.

The Ledger in action — auditing time & money

When a BI number looks wrong at 7 AM, the ledger shows precisely which records hadn't arrived yet, when they did, and what they changed — because every record keeps its full timing history, not just its latest value.

Record Update Time Summary analysis tool for parte_operaciones_costos on one report date: 643 records total (418 additions, 225 edits) tracked against the 6 AM cutoff and the 6:30, 7:30, and 9:30 BI update windows, with records landing after the last window identified and their cost impact quantified.
643 records, timestamped both ways. This audit exists only because every record carries its full history: 418 additions and 225 edits to parte_operaciones_costos on one report date, checked against the 6 AM cutoff and the 6:30 / 7:30 / 9:30 BI windows — records arriving after the last window are flagged, with $318,396.78 in cost impact identified as landing “9:30 AM onwards.”
05 — Automation Core

One trigger. Many systems.

AURA is becoming the automation core of the operator's well-construction stack: an event in one system — an AFE created, a scheduled sync tick, a field update landing in Operations Reporting System — automatically configures, populates, or refreshes the others. No one re-types the same well into three different screens.

A · The AFE cascade

When the operator creates a new AFE, AURA fans that single event out to the Corporate ESB, Operations Reporting System, completions staging, notifications, and BI — work that used to mean manual entry into 3+ systems now happens in the background within 15–60 minutes.

Overview — one trigger, three parallel branches
3 branches, in parallel Operator creates an AFE AURA HTTP 201 FARO email + SMS the ESB → Ops Reporting Completions stages auto-load Ops sees the well in: Well Files · Completions dashboard · Cost dashboards 15–60 min, zero manual entry

Each branch, in detail

1 · FARO notify
AURA HTTP 201 Ops team email + SMS, immediate

FARO sends both an email and an SMS to the operations distribution list the moment AURA responds — no dashboard-watching required.

2 · the Corporate ESB → Operations Reporting System
the ESB SOAP DTOs, 2 endpoints ESB queues, forwards Ops Reporting ACK by email, not to AURA on FAILURE esb_sync_failure (5-min cooldown)

Operations Reporting System's async chain is fire-and-forget from AURA's point of view — Operations Reporting System ACKs the operator by email, not back to AURA. Any sync failure alerts immediately.

3 · Completions auto-load → BI
stages_planned ← Analytics design Themis BI next scheduled window

Only fires when the AFE carries stage design data; otherwise this branch is a no-op. Completions dashboards pick up the new well on the next refresh.

Before: manual entry into 3+ systems

  • Someone re-types AFE sections into the Corporate ESB by hand
  • Someone re-keys the well into Operations Reporting System's programa & tareas
  • Completions stages are copied in separately from the design feed
  • Every hand-off is a chance to mistype a section name or a depth
After: one AFE create. AURA's cascade replaces the manual entry with a single trigger — three systems configured in parallel, no re-typing, no missed hand-off. Hours to days saved per well.

B · Well Data Master read → create well (D1)

Five times a day, a VDI agent reads the operator's Well Data Master Oracle master and syncs it into AURA — keeping field, area, and well-status attribution current on all 6,575 active wells without anyone running a manual export.

Well Data Master Oracle → AURA well master
Well Master Oracle 6,575 active wells CATAMAPR / tecblora15 VDI agent master_sync_agent.py 6:27, 7:27, 9:27, 11:27, 12:27 ART 33 attrs mapped 23 direct + 10 FK-resolved POZO_ID ↔ UWI match AURA well master 6,550 updated + 25 inserted well_master_last_synced breadcrumb >30h stale → FARO alert
6,575
active wells synced per cycle
33
attributes mapped — 23 direct + 10 catalog FK-resolved
/day
business-hour cron: 6:27, 7:27, 9:27, 11:27, 12:27 ART
38s
execution time per full sync (Oracle read + AURA ingest)
Well Data Master remains authoritative — AURA only observes for D1, with a strict 1:1 POZO_ID↔UWI match and zero inactivations.

C · Update Well Data Master operation dates (D2 write-back)

Chained right after the D1 read on the same cron tick, AURA writes field-sourced dates and survey depths back into Well Data Master — guarded by an idempotency ledger and a no-downgrade rule so a stale or shallower survey can never overwrite a better existing value.

Operations Reporting System reales → ledger guard → Well Data Master — ledger-guarded
Ops Reporting reales dates + survey TVD/TMD 47–49 wells changed/cycle Ledger guard diff vs pushed_value no-downgrade (50m tolerance) Well Master API es-AR: "1000,09" not "1000.09" ledger-guarded push Push ledger ACK status=sent/skipped idempotent re-run Oracle read-back verify
AttributeFieldSource
327Inicio Perforación (spud date)Operations Reporting System-first COALESCE
329 / 330Inicio / Fin TerminaciónOperations Reporting System-first COALESCE
331 / 332TVD / TMD (survey depths)Deepest active survey station, 2dp round
A reversible production test validated the write path end-to-end — including catching and fixing a comma-decimal formatting edge case (a "3065.01" input corrupting to 306501) before any live write.

D · Well File System push for Daily Events Viewer

Every 15 minutes, a VDI agent pulls pending wells, events, and operations from AURA and pushes them into the Well File System — feeding the Daily Events Viewer, the daily drilling dashboard that consolidates reports, activities, and contractor performance.

AURA pending payload → VDI poll → Well File System → Daily Events Viewer
get_pending wells · events · operations 5 domains assembled VDI agent well_file_push_agent.py every 15 min Well Files platform POST / PATCH / 409 dedup operator platform Events Viewer daily drilling events view operator-internal Ops reads it daily
15min
push cadence — 4×/hour
5
domains pushed: wells, pads, events, event operations, news
5–20
new or changed events pushed per cycle

Proof: this is running, not a plan

Every automation in this chapter is backed by a live scheduled job on the AURA platform — not a diagram of an intention. The Services panel shows roughly 35 systemd services with real-time health, and System Status confirms the integrations themselves are reachable.

AURA Admin Services page listing roughly 35 systemd services with live green health dots — aura-api, aura-mat-worker, continuous import services, scheduled deletion jobs, backup and upload services, and monitors — alongside a live-tailing system log pane.
Services — live health, not a diagram. Roughly 35 systemd services power every automation in this chapter, each with a live green/red status dot and a tailing log pane underneath.
* Screenshot: AURA Admin portal, Services page, captured 2026-07-14. Integration-specific service names have been generalized; no personal data shown.
On the roadmap: two more automations are queued behind this chapter — telemetry-driven report pre-fill (auto-populating Operations Reporting System reports from Analytics detections) and well & event creation in the Corporate ESB directly from AFE triggers — both building on the automation core shown in this chapter.
06 — Systems & Integrations

12 systems, one platform

AURA doesn't just store data — it's the hub connecting Analytics, Operations Reporting System, Well Data Master, Well File System, FARO, Themis, TPO, Projects, and more. Click any card to see how data actually moves in and out.

Diagram nodes
External system AURA processing Destination / consumer

Real-Time Drilling & Completions Analytics

Inbound

REST API · daily timer + manual pull

Live
Direction
Inbound — AURA pulls from Analytics
Transport
REST API (/v1/assets, /v1/data, /v1/well-connections)
Trigger / cadence
analytics-import-wits.timer (daily) + manual via analytics_import.py
Analytics asset & WITS import
Analytics API /v1/assets, /v1/data config_analytics.json endpoint routing analytics_import.py pull + dedup (supersede_by_fields) aura_data assets, wits, stages Themis BI
  • Assets — pads, wells, rigs, equipment
  • WITS telemetry — bit depth, hole depth, pump pressure, RPM
  • Completion stages — stage number, fluid type, volume, proppant, mesh size
  • Transforms — imperial→metric unit conversion; Analytics asset_id → stable __aura_external_id

Analytics Fusion

Outbound

REST API POST · manual / on-demand

Live
Direction
Outbound — AURA pushes to Analytics
Transport
REST API POST (Analytics /integration endpoint)
Trigger / cadence
On-demand via sync_analytics_fusion.py; no schedule active
Well-plan report submission
AURA scheduler on-demand job AssembleWellPlanPayload archive + analytics + ops reales Analytics API POST well_plan Analytics Fusion ingests report
  • Well plans — well_id, surface/target depth, rig, trajectory
  • Report typewell_plan (completions/wireline reports planned)
  • Auth — 64-char ANALYTICS_FUSION_API_KEY bearer token

Well Operations Archive

Inbound

COPY-CSV / REST · manual import

Live
Direction
Inbound — AURA pulls well master (manual)
Transport
PostgreSQL COPY-CSV over SSH + REST GET
Trigger / cadence
Ad-hoc, manual via import_archive_pozo.py
Well master sync
Well Archive customer data source CSV export manual operator pull import_archive_pozo.py COPY binary-safe, UTC dates aura_data_archive well master lookups
  • Well master — uwi, well_name, pad_name, rig_name, surface location, depths
  • Pad coordinates — latitude, longitude, ground elevation
  • Transforms — CSV→COPY binary-safe encoding, UTC normalization; no unit conversion (both systems metric)

Operations Reporting System

Inbound

REST + live SQL · continuous 10–60 min timers

Live
Direction
Inbound — read (REST vistas + live SQL). The AFE→ESB write path reaches Operations Reporting System via Corporate ESB (see AFE→ESB card), not observed by AURA
Transport
REST vistas + live SQL pass-through; SOAP XML for writes
Trigger / cadence
ops-reporting-import-continuous-*.timer (10–60 min) + pre-BI windows (06:30/07:30/09:30/11:30 ART) + deletion sweep
Vista import + BI export
Ops Reporting vistas REST API config_ops_reporting.json endpoint → data_type mapping import_ops_reporting.py incremental window pull aura_data sections/dates programa, tareas BI pre-window export AFE → ESB write-back (see AFE→ESB card)
  • Well sections — name, casing type, start/end depth & date
  • Well dates & plan vistas — inicio/fin perforación & completación, programa, tareas
  • Transforms — Spanish section-name normalization, time-interval + depth-fallback section attribution (v2.6.50–53)
Monitoring: the import-failure monitor alerts automatically on three or more consecutive failures.

AFE → Corporate ESB

Outbound

SOAP/XML · event-driven on AFE create

Live (QA/PROD)
Direction
Outbound — AURA → Corporate ESB SOAP, then async to Operations Reporting System
Transport
SOAP/XML over HTTPS (Corporate ESB SOAP endpoints, port 443)
Trigger / cadence
Event-driven on AFE creation; wrapped in a BackgroundTask since v2.6.23
Section sync — protocol-level view
esb_client.py afeLines + tareas DTOs ESB SOAP endpoints AFE + WellPlanning (redundant) operator ESB queues, async Ops Reporting (async ACK) not observable by AURA on FAILURE → esb_sync_failure FARO alert (5-min cooldown)
  • Section name/ID flag — QA sends names (esb_section_by_name=true); PROD sends numeric IDs (customer-confirmed steady state)
  • Identity — deterministic format_id_novedad_origen, fresh id required per re-send (single-use on Operations Reporting System)
  • Business narrative — this is the protocol-level view of the same event chain shown in Chapter 05's AFE cascade

Well Data Master

Bidirectional

Oracle SQL + REST · VDI dispatch + daily cron

Live
Direction
Bidirectional — D1 read (Well Data Master→AURA), D2 write (AURA→Well Data Master)
Transport
SQL via SSH tunnel + VDI agent (read); Well Data Master REST CreateOrUpdate (write)
Trigger / cadence
D1: daily 07:00 ART cron + manual; D2: five daily crons (ART)
D1 — Well Data Master → AURA master sync
VDI agent master_sync_agent.py Well Master Oracle pozo master, catalogs well_master_columns.json + CATALOG_JOINS (FK resolve) aura_data_well_master 33 columns · 6,575 wells synced
D2 — AURA → Well Data Master writeback
compute_writeback_values 6 attrs from ops reales assemble_writeback_pending diff vs ledger, optional scope filter Well Master API comma-decimal, es-AR dates push ledger ACK record
  • D1 entities — POZO_ID equivalence, well attributes, 33 master columns (23 direct + 10 FK-resolved)
  • D2 entities — Inicio Perforación, Inicio/Fin Completación, TVD, TMD (2 depth attrs)
  • Transforms — comma-decimal es-AR depths (fixed a 100x inflation bug in v0.1.11), no-downgrade guard on depth overwrites

Well File System

Outbound

HTTP REST/SOAP · 15-min VDI poll

Live · 7 domains
Direction
Outbound — AURA → Well File System
Transport
HTTP REST/SOAP (Well File System API + VDI client.py)
Trigger / cadence
15-min poll via well_file_push_agent.py on the operator VDI
Pending-payload poll & push
aura_data wells, completions, programs /api/well_file/get_pending 7 payload domains the operator VDI agent client.py, 15-min poll Well Files platform POST/PATCH/409 dedup
  • Live domains (5) — wells, well_pads, events, event_operations, open_wells_news
  • Gated domains (2) — fractura, wireline — code merged, 24 unit tests pass
  • Transforms — dd-mm-yyyy date formats, Bloqueado/Desbloqueado status (native Operations Reporting System), Dirigido/Direccional trajectory

FARO *

Outbound

HTTPS REST · event-driven + digest

Live Optional module *
Direction
Outbound — AURA → FARO API
Transport
HTTPS REST (POST notifications.roderickc.com/api/v1)
Trigger / cadence
Event-driven (AFE create, import failures, sync alerts) + daily digest queues; 60-min cooldown per recipient (5-min for esb_sync_failure)
Commercial status
* Optional module — not yet acquired, fully functional and in active use today
Notification dispatch
AURA event AFE create, import fail, sync NotificationService format + dedupe FARO API template + address resolve Email (Gmail/SendGrid) SMS/WhatsApp (Twilio)
  • Recipients — distribution lists (@AURA_SUPPORT, @AURA_DATA_MANAGEMENT) + per-operator email
  • Notification types — email, sms, whatsapp; markdown body → HTML by FARO
  • Env behavior — subject line env-prefixed ([QA]) via resolve_subject
Gotcha: digest-type notifications return {queued}/{suppressed} instead of a notification_id — that's correct digesting, not a failure.

Themis BI

Inbound

HTTPS REST · on-demand + scheduled refresh

Live
Direction
Inbound — BI reads AURA data
Transport
HTTPS REST (AURA reporting endpoints)
Trigger / cadence
Themis tile refresh (typically 5–15 min) + ad-hoc dashboard clicks + pre-BI windows
Dashboard read path
Themis dashboard UI tile refresh / ad-hoc GET /api/reporting/* aggregate, filter, format PostgreSQL materialized views + live queries Themis render chart / table
  • Coverage — AFE summary, well status, drilling progress, completions, costs, NPT, pad performance
  • Materialized viewsreporting/auditoria, reporting/costos, reporting/time_summary
  • Transforms — OData filters → SQL WHERE clauses; daily/weekly aggregation; metric unit display

TPO Reports

Inbound

HTTPS REST + export · daily timer + manual

Live
Direction
Inbound — BI reads AURA data
Transport
HTTPS REST + scheduled export job
Trigger / cadence
Scheduled daily (~06:00 ART) + manual via /api/tpo/generate_report
KPI generation & export
aura_data well_activities, sections, wits /api/reporting/tpo generate.py — ROP, NPT% formulas Parquet / XLSX export archive Themis / external BI dashboard consumer
  • KPIs — rate of penetration (ROP), non-productive time (NPT) by category, section time vs plan
  • Formula — ROP = total_depth / on_bottom_time; NPT% = (total_time − on_bottom_time) / total_time
  • Depth-fallback — section attribution falls back to depth ranges, improving TPO accuracy

Projects

Outbound

HTTPS REST · manual + event-driven

Live
Direction
Outbound — AURA writes to Projects Tracker (fire-and-forget)
Transport
HTTPS REST (Projects Tracker API)
Trigger / cadence
Manual annotation + event-driven on new AFE (via FRD-004 secret delivery)
Task-tracking sync
AURA AFE creation event trigger FRD-004 secret drop if enabled PT API POST /api/v1/updates/note ProjectTracker ops task tracking
  • Task creation — well_id, well_name, rig, AFE details
  • Updates — status, notes, completion percentage
  • ConfigPROJECT_TRACKER_API_KEY in .env, sent as X-API-Key header

System Monitoring

Internal

PostgreSQL + systemd · 5–15 min timers

Live
Direction
Internal — AURA self-health
Transport
PostgreSQL logs, systemd journal, in-process metrics
Trigger / cadence
Continuous /health + endpoint-health-monitor.timer (15 min), import-failure-monitor.timer (15 min), pg-lock-monitor.timer (5 min)
Continuous health checks
systemd timer 5–15 min health / import-failure / pg-lock monitors check AURA service state on threshold breach FARO alert → AURA_SUPPORT
  • Tracked — API response time, error counts, import lag, DB connection pool, disk space
  • Backup monitor — hardened threshold (<150GB free OR ≥95% used)
07 — API Endpoints for BI / UTF

34 endpoints, one consistent contract

Built for the consumers that matter: BI dashboards and UTF template exports. Every endpoint — reporting, completions, drilling, AFE — shares the same auth model, the same query parameters, the same metadata envelope, and the same four output formats. Learn the pattern once, integrate any BI tool or script the same way.

34
public endpoints across 4 BI-facing domains
30+
admin endpoints (separate aura_admin_api)
98.6M
records served in the last 30 days — ~100K requests
3
auth methods — JWT, API key, OAuth 2.0
4
output formats — JSON, CSV, XLSX, Parquet
AURA Admin Usage Analytics for the last 30 days: 99,940 total requests, 98,585,278 records served, average 986 records per request, top endpoint cp_wbs_codigos. Charts show records served and requests over time, and a Top Endpoints table lists /api/reporting/cp_wbs_codigos with 25.7 million records served, costos_mensuales 17.5 million, hole_plan 10.6 million, time_summary 10.4 million, plan_tiempos 9.2 million, planes_pozos 7.2 million, cp_npt_subclase 5.0 million, and costos_diarios 1.9 million.
Not just designed for BI — used by it. Live usage analytics: 99,940 API requests served 98.6M records in the last 30 days (≈986 records per request, ≈3,300 requests a day), and the top endpoints are precisely the BI/UTF reporting feeds — cp_wbs_codigos (25.7M records), costos_mensuales (17.5M), hole_plan (10.6M), time_summary (10.4M).

The endpoint surface, by domain

Reporting carries most of the surface area — financial reports, operational timelines, and well metadata. Completions covers frac/wireline stage tracking and 10-second telemetry. The remaining domains are narrower, purpose-built endpoints.

Public endpoint count by domain (34 total)
* Reporting includes costos_mensuales, costos_diarios, eventos, time_summary, datos_generales, planes_pozos and 16 more. Analytics Completions covers stage tracking, frac/wireline milestones, and 10-second telemetry streams. A generic OData endpoint (/api/generic/data) additionally serves ad-hoc queries over any stored data type.

Auth, roles, and formats — one pattern for all 34

Pick the auth method that fits the client, request the format the consumer needs, and every endpoint responds the same shape.

  • JWT tokens — 24-hour lifetime, standard Bearer header, auto-refresh pattern documented
  • API keys — persistent, no rotation needed, best for service accounts
  • OAuth 2.0 — Google/Microsoft, HttpOnly cookies for browser dashboards
RoleAccessUse case
ReadOnlyRead data endpoints onlyBI analysts, dashboard viewers
DataManagerRead/write + exportsData management, ETL workflows
DeveloperAdmin tools + dev endpointsAPI development, integration work
SystemFull system accessSystem administrators
Output formats
  • JSON default, preserves types
  • CSV flattened, Excel-ready
  • XLSX auto-formatted columns
  • Parquet 5–10x compression, best for BI
Response-time tiers
  • Aggregated (70–100 rows) <100ms
  • Detail (200–300 rows) 0.5–1.2s
  • Historical (2,000+ rows) 1–2s
  • Large exports (10,000+) 5–15s

Anatomy of a request

One annotated example teaches the whole contract: standard query parameters, the __aura_* metadata envelope on every record, and a consistent top-level metadata block.

GET /reporting/costos_mensuales 200 OK
# Real request — costos_mensuales, formatted for BI consumption
GET /reporting/costos_mensuales?start_date=2026-03-01&output_format=parquet&field_template=BI
Authorization: Bearer <jwt_or_api_key>
  • start_date — filters to the operational window (Buenos Aires TZ, minus 6h)
  • output_format — one of json / csv / xlsx / parquet
  • field_template — BI (business-friendly names) or raw (database names)
response.json (trimmed)
{
  "data": [
    {
      "pozo": "W-1606",
      "mes": "2026-03",
      "monto_total": 184230.55,
      // stable BI merge key — integer, unique-per-row
      "__aura_external_id": 88213041,
      "__aura_data_source": "ops-reporting",
      "__aura_last_updated": "2026-07-14T06:02:11-03:00",
      "__aura_active": "active"
    }
  ],
  "metadata": { "count": 100, "total_records": 100, "timestamp": "2026-07-14T10:30:00-03:00" }
}
Interactive docs: Every endpoint is browsable and testable at /api/docs — Swagger UI generated from the OpenAPI 3.0 spec, with request/response schemas and copy-ready curl commands. 22 API doc pages in /docs/ cover authentication, parameter conventions, and domain-specific deep dives.
08 — Data Quality

Quality isn't a phase — it's a process that runs every 5 minutes

AURA ships with a testing and monitoring framework sized to match its data surface: 1,626 unit tests gate every commit, a smoke suite gates every deploy, a 3,500-test combinatorial sweep runs daily against every endpoint, and continuous monitors watch imports, locks, and health around the clock.

1,626
pytest unit tests across 102 files (600-test CI floor)
70
smoke tests — deploy gate, 2–4 minutes
3,500
full combinatorial tests — daily, 07:00 UTC
34/34
endpoints audited: __aura_external_id distinct == rowcount

Four cadences, one framework

Fast checks run on every commit; slower, wider checks run less often but cover more ground. Together they read as a single sentence: every commit, every deploy, every day, every five minutes.

Test & monitoring cadence bands

That continuous layer runs at two levels: inside AURA (import-lag, database-lock contention, endpoint health, and data-quality scans, all reporting to FARO) and outside it, where the RCI Monitor platform watches the host itself — services, disk, memory, load, and SSL expiry. A failed sync and a failing server both page the team before users notice either one.

AURA Admin Data Quality overview: 13 total automated tests, 10 with issues, 1,404 total issues, 9 high-severity. A per-test table lists named business-rule checks with live issue counts and last-scan times: FRAC sin Etapa (343), Service and Job Health (273), Unknown Account Codes vs chart of accounts (255), Partes sin AFE (195), Date Boundary Mismatches (116), Activity vs Event Dates (81), AFE Name Validation (68), Orphan Wells without Analytics equivalence (55), Analytics Schema Monitoring (16), and Endpoint Response Validation (2). Tabs above the table (Tests, Raw Capture, Validation, Temporal, TMD Classification, TMD Thresholds) show additional views beyond the overview.
Inside: 13 automated checks, running now. These are business-rule tests — a frac stage without a stage number, an operation without an AFE, a cost booked to a nonexistent account — not just technical checks, scanned continuously with severity and history. Issue counts demonstrate the checks are actually running against production data, not a claim that the data has a problem.
RCI Monitor platform watching the AURA production host: seven green service checks (SSL, Redis, PostgreSQL, Nginx, fail2ban, SSH, system), host vitals showing 64 CPUs, 29% disk, 29% RAM, and 41 days of uptime, 90-day CPU, load, memory, and disk trend charts with alert thresholds drawn in, a 200-record check history, and a live Telegraf agent.
Outside: the host, watched independently. RCI Monitor tracks the AURA server itself — seven service checks, 90-day resource trends with alert thresholds, and a live agent — so infrastructure failures surface even if AURA's own monitors can't report them.

The CI/CD gate pipeline

Every push and pull request runs the same sequence of gates, in order — all must pass before merge.

ci.yml — gate sequence
1 Lint
2 Syntax
3 Security
4 What's New
5 Unit floor (600)
6 DB Grants
7 Integration

Proof point: the __aura_external_id audit

Every BI consumer merges on __aura_external_id — it has to be an integer, unique per row, and stable across reloads. In July 2026, three reporting endpoints needed surrogate-key fixes; all 34 reporting and completions endpoints are now audited and verified distinct count == row count.

  • novedades_well_file — natural key from 3 fields, 880 rows verified unique
  • advance_curve — full envelope added, 3,836 rows verified unique
  • afe_x_avance — natural key from 7 fields, 17,175 rows verified unique
EndpointIssueVerified unique
novedades_well_file856/881 distinct IDs (UNION overlap)880 rows
advance_curveno external_id (chart data)3,836 rows
afe_x_avance28/10,418 distinct IDs17,175 rows
Security posture: Redis-based rate limiting, environment-specific CORS allowlist (no wildcard), HSTS on all HTTPS endpoints, obfuscated server header, and secret scanning built into every CI run.
09 — Data Governance

The data is governed, not just stored.

Governance in AURA is three disciplines working together: authoritative masters, enforced safeguards, and a complete audit trail — so the data can be trusted independent of whichever technology happens to move it.

A · Masters — one authoritative source per entity

Every governed entity has exactly one system of record. AURA doesn't compete with that authority — it observes it, mirrors it faithfully, and gives operators a single place to manage the master data everything else depends on.

Well Master

Well Data Master is authoritative. AURA maintains a strict 1:1 POZO_ID ↔ UWI mapping, synced across 6,575 wells with zero inactivations from the sync process — AURA observes, it doesn't overwrite.

Well Master · Rig Management

Rig Master

Rig identity and specs are curated the same way as wells — one editable master record per rig, referenced everywhere a rig is mentioned across drilling, completions, and cost data.

Rig Management

Catalogs & identity

Event catalogs, account codes, and every other lookup table are governed through the same Data Editor as production data. Identity is stable: __aura_external_id gives every record a deterministic business key (see Chapter 04).

Data Editor
* Source: platform automation documentation, Use Case 1 (Well Data Master read → create well, D1): 6,575 active wells, strict 1:1 POZO_ID↔UWI match, 0 inactivations confirmed. Cross-ref Chapter 04 for __aura_external_id.

B · Safeguards — the data protects itself

Where AURA is not the system of record, it writes back carefully — guarded, staged, and reversible — and where a field is sensitive, access can be locked down to that single field without touching anything else on the record.

AURA Admin Access Control Permissions screen: named permission sets (System_Full_Access, Admin_Access, Audit_Monitoring, ReadOnly_Dashboard, Developer_Standard, DataQuality_ReadOnly, DataManager_Standard) assigned to roles. For each resource, such as AFE Wells or Master Rigs, both resource-level operations (View, Read, Write, Edit, Delete) and a per-field matrix of the same operations plus Deny are shown, with bulk actions (All Read, All Edit, Deny All). The approved-amount field (monto_aprobado) on AFE Wells is flagged as a locked field.
Governance down to the field. Seven named permission sets control access at both the resource level and the individual field level. Here monto_aprobado (approved amount) is locked on AFE Wells — a data manager can edit the rest of the record, but that field stays closed. Deny always overrides a grant.

Write-back discipline

The one place AURA writes into another system's master data — Well Data Master's operation dates and survey depths — is guarded on every side:

  • No-downgrade guard: a shallower or older value never overwrites a better existing one (50m tolerance on depth checks)
  • Idempotent push ledger: every write is tracked; unchanged values are never re-sent
  • Dry-run first: every write path is validated in simulation before a single real value moves
  • Explicit sign-off gates: live writes wait on named approval before they activate
Read-only where AURA isn't the owner. Well Operations Archive is read-only on QA/PROD. Well Data Master's well master sync (D1) is observe-only — AURA never writes back into the well master itself, only into six specific operation-date and depth attributes, under all four guards above.
The Ledger, again: nothing in aura_data is overwritten or physically deleted. A status lifecycle (active → superseded/deleted) plus dual timestamps (source last_updated and AURA's own internal_last_updated) make every value replayable “as of” any moment — see Chapter 04 for the full mechanism and a live audit example.
* Source: platform automation documentation, Use Case 2 (operation-date writeback, D2): no-downgrade guard, idempotent push ledger, staged rollout gates. See Chapter 04 for the Ledger mechanism.

C · Audit — who did what, when

Every request is logged, every field-level change has history through the Ledger, and three dedicated admin views turn that raw log into something an operator can actually use.

769K
audit events logged in the last 7 days
100%
of API calls logged — every request, not a sample
field-level change history via the Ledger — never pruned
AURA Admin Audit Logs table: per-request rows with timestamp, user, action (get_api_call), resource (api_endpoint), details (GET 200), and source IP address, with search, date-range filter, and export controls.
Every request, one searchable log. Timestamp, user, action, resource, and source IP on every call — searchable, filterable by date range, and exportable. API-key and unauthenticated calls show as “No username” rows rather than a gap in the record.

Three more admin views turn that raw log into governance: Audit Performance tracks the audit system's own health, Usage Analytics shows who is calling which endpoints (Chapter 07 covers the volume side — 99,940 requests and 98.6M records served in 30 days), and API Traces gives a request-by-request drill-down when something needs investigating.

* Source: audit event count and admin navigation from the Chapter 02 dashboard capture; audit log screenshot captured 2026-07-14. Usage figures cross-referenced from Chapter 07.
10 — The Value

Why operators choose AURA

Everything in this presentation is running in production today, over 280M+ records at a major energy operator. This is what that buys — and how to see it against your own wells.

What an operator gets

Unify

One source of truth

Five source systems, one governed PostgreSQL model, one 95+ endpoint REST API. BI tools, engineers, and integrations stop reconciling and start querying.

Speed

New sources in hours, not projects

The flexible-storage registry turns a new data source into configuration, not code — a new vista can be live and serving BI reports in under an hour.

Automate

One trigger, many systems

An AFE created upstream reaches the ESB, daily reporting, completions staging, and the right inboxes in minutes — work that used to mean days of manual re-entry.

Trust

Data quality that watches itself

Business-rule tests, schema-drift alerts, and a ledger keeping full history on every record. Every failure reaches a human inbox automatically.

Govern

Governance built in

Corporate SSO, role-based access down to the individual field, and a fully audited admin surface — who saw what, who changed what, always answerable.

Deploy

Runs where you need it

Fully managed today, with a proven migration path onto an operator's own cloud infrastructure. No re-platforming, no lock-in surprise.

The numbers behind the claims

280M+
records under management in production
<1 hr
to onboard a new data source, config-only
99.9%
uptime SLA in production
100%
of admin actions audited, searchable, exportable

See it against your own wells

The fastest way to evaluate AURA is a walkthrough with your data in the loop — your source systems, your naming, your reporting pain. We will show you the path from scattered systems to one governed platform.