Building a Real-Time Gold Dashboard With 6 APIs and 14 Background Jobs

I wanted a single screen where I could see everything about gold: price, macro, sentiment, positioning, correlations, and AI analysis. Bloomberg Terminal does this for $24,000/year. I built my own for about $80/month.

The result is Aurum — a real-time XAUUSD intelligence dashboard running Next.js 15 on the frontend, FastAPI on the backend, DuckDB for analytics, and Redis for caching.

The data problem

Gold doesn't move in isolation. It responds to the dollar, real yields, Fed policy, geopolitical risk, physical demand, ETF flows, and speculative positioning. To trade it well, you need all of those data streams in one place.

No single API covers everything. So I wired up six:

The total data cost is just the EODHD subscription. Everything else is free or included in existing plans.

The 9 data domains

The dashboard organises gold intelligence into 9 sections:

1. Price action — Current spot, daily/weekly/monthly change, 52-week range, volume
2. Macro environment — Real yields, DXY strength, Fed funds rate, inflation expectations
3. Economic calendar — Upcoming events that move gold (NFP, CPI, FOMC, PMI)
4. COT positioning — Net speculative long/short, managed money, commercial hedgers
5. ETF flows — GLD and IAU holdings, weekly inflows/outflows
6. Correlations — Gold vs DXY, gold vs yields, gold vs SPX, rolling 30/60/90 day
7. Sentiment — News sentiment score, RSS headline analysis, fear/greed proxy
8. AI analysis — Claude reads all the data and writes a daily regime assessment
9. Technical levels — Support/resistance, moving averages, RSI, key price levels

Each domain has its own data pipeline, refresh schedule, and fallback source.

The 14 background jobs

FastAPI runs 14 scheduled jobs via APScheduler:

# Simplified scheduler config
jobs = {
    "price_update":        {"interval": "1m",  "source": "finnhub"},
    "economic_events":     {"interval": "6h",  "source": "eodhd"},
    "macro_data":          {"interval": "1h",  "source": "fred"},
    "cot_weekly":          {"interval": "sat",  "source": "cftc_socrata"},
    "cot_fallback":        {"interval": "sun",  "source": "eodhd"},
    "etf_flows":           {"interval": "4h",  "source": "eodhd"},
    "correlation_calc":    {"interval": "1h",  "source": "internal"},
    "sentiment_rss":       {"interval": "30m", "source": "rss"},
    "sentiment_score":     {"interval": "1h",  "source": "internal"},
    "ai_analysis":         {"interval": "4h",  "source": "claude"},
    "technical_levels":    {"interval": "15m", "source": "internal"},
    "cache_cleanup":       {"interval": "1h",  "source": "internal"},
    "health_check":        {"interval": "5m",  "source": "internal"},
    "db_maintenance":      {"interval": "24h", "source": "internal"},
}

The distinction between external (API calls) and internal (computed from existing data) matters for rate limiting. External jobs have retry logic and backoff. Internal jobs run fast because they're just DuckDB queries.

The gotchas

EODHD's default limit

Their /economic-events endpoint returns 50 results by default. Not 100, not "all" — 50. If you're fetching US economic events for the next month, 50 might only cover a few days. Always pass limit=1000 and country=US:

params = {
    "from": start_date,
    "to": end_date,
    "country": "US",
    "limit": 1000  # Default is 50!
}

I spent half a day wondering why my economic calendar was missing events. This was it.

COT data sources

The Commitment of Traders report is published weekly by the CFTC. Getting it reliably is harder than it should be.

Primary: CFTC Socrata API — Free, has full disaggregated data (managed money, swap dealers, producers). The API is REST-based and reliable, but the data lands on Tuesday for the previous Friday's report.

Fallback: EODHD — Has COT data but only net speculative positions. Less granular, but available faster. I use this when Socrata is slow or down.

async def fetch_cot():
    try:
        data = await cftc_socrata.get_gold_futures()
        if data and data.report_date >= last_friday():
            return data
    except Exception:
        pass
    # Fallback to EODHD (net spec only)
    return await eodhd.get_cot_summary("GC")

DuckDB for analytics

I chose DuckDB over PostgreSQL for the analytics layer because gold analysis involves a lot of time-series operations — rolling windows, correlations, regime detection. DuckDB handles these natively and runs in-process, so there's no network overhead.

PostgreSQL still handles the "system" data — job logs, user preferences, saved views. The two databases serve different purposes and shouldn't be merged.

The AI analysis layer

Every 4 hours, Claude reads the latest data from all 9 domains and writes a regime assessment:

Current regime: RISK-OFF ACCUMULATION
Confidence: 72%

Gold is trading at $2,847 with positive momentum. Real yields fell 8bps
this week while DXY weakened below 104. COT shows managed money adding
12,400 net longs. ETF inflows resumed after 3 weeks of outflows.

Key risk: FOMC minutes release Wednesday 2pm ET. If hawkish surprise,
expect pullback to $2,810 support.

This isn't trading advice — it's a structured summary that saves me 30 minutes of reading multiple sources every morning. The regime label (risk-on, risk-off, range-bound, accumulation, distribution) helps me quickly calibrate my bias for the day.

The stack

Frontend:  Next.js 15 + TypeScript + Tailwind + Recharts
Backend:   FastAPI + APScheduler + httpx
Analytics: DuckDB (in-process)
System DB: PostgreSQL (Railway)
Cache:     Redis (Railway)
AI:        Claude API
Deploy:    Vercel (frontend) + Railway (backend)

Total infrastructure cost: about $35/month (Railway for backend + database). Add the $80/month EODHD subscription and the full dashboard runs for ~$115/month. Bloomberg Terminal it is not. But for a solo trader focused on one asset, it covers what I need.

The dashboard doesn't make trading decisions for me. It organises information so I can make decisions faster. Most mornings, I open Aurum, read the AI summary, check COT positioning, glance at the correlation panel, and I'm ready. That used to take 45 minutes of bouncing between TradingView, the FRED website, and various news feeds. Now it takes about 5.