The guide · build blueprint

The ultimate AI stack.

You no longer need the most expensive flagship model to get top-tier results. The move is hybrid: route each task to the model best suited for it. Claude 4.6 writes the system architecture and does the deep engineering; Gemini 3.1 acts as the eyes and ears, processing massive amounts of live visual and geospatial data. Below is the full blueprint — how to bridge the two models, plus the actual file-by-file build for a real-time geospatial tracker.

The full build blueprint

🔒 Drop your email to reveal the full build

Get the full blueprint, free.

Drop your email and every file, prompt, and cost breakdown unlocks instantly, plus every other guide and resource in the library. One email a week, unsubscribe anytime.

# Example command inside your terminal
clink with gemini panoptic_analyzer to audit live_traffic_feed.jpg for vehicle coordinates

npm install -g @google/gemini-cli
export GEMINI_API_KEY=your_key_here

#!/bin/bash
output=$(gemini "$@" 2>&1)
echo "$output" | jq -r '.response' 2>/dev/null || echo "$output"

chmod +x ~/.claude/bin/gemini-clean

geospatial-tracker/
├── backend/
│   ├── main.py              # FastAPI app + WebSocket hub
│   ├── ingestion/
│   │   ├── opensky.py        # Live aircraft positions
│   │   ├── traffic_cams.py   # Public DOT camera feeds
│   │   └── satellite.py      # Sentinel/Planet tile fetcher
│   ├── analysis/
│   │   ├── gemini_client.py  # Gemini vision API wrapper
│   │   └── panoptic.py       # Detection orchestrator
│   ├── models/
│   │   └── schemas.py        # Pydantic models for all data
│   └── config.py             # API keys, polling intervals
├── frontend/
│   ├── src/
│   │   ├── App.tsx
│   │   ├── components/
│   │   │   ├── LiveMap.tsx    # Mapbox GL JS map layer
│   │   │   ├── PlaneLayer.tsx
│   │   │   ├── VehicleLayer.tsx
│   │   │   └── CameraPanel.tsx
│   │   └── hooks/
│   │       └── useWebSocket.ts
│   └── package.json
├── docker-compose.yml
└── .env

# backend/ingestion/opensky.py
import httpx
import asyncio
from models.schemas import AircraftPosition
OPENSKY_URL = "https://opensky-network.org/api/states/all"
async def fetch_aircraft(bbox: dict = None) -> list[AircraftPosition]:
    """
    Fetches all live aircraft positions from OpenSky.
    bbox: {"lamin": 45.0, "lomin": -125.0, "lamax": 50.0, "lomax": -115.0}
    Rate limit: 5 req/10s (anonymous), 1 req/5s (authenticated)
    """
    params = bbox or {}
    async with httpx.AsyncClient(timeout=10) as client:
        resp = await client.get(OPENSKY_URL, params=params)
        data = resp.json()
    aircraft = []
    for state in data.get("states", []):
        aircraft.append(AircraftPosition(
            icao24=state[0],
            callsign=(state[1] or "").strip(),
            origin_country=state[2],
            longitude=state[5],
            latitude=state[6],
            altitude=state[7],       # meters (barometric)
            velocity=state[9],       # m/s ground speed
            heading=state[10],       # degrees from north
            vertical_rate=state[11],
            on_ground=state[8],
            last_contact=state[4],
        ))
    return aircraft

# backend/ingestion/traffic_cams.py
import httpx
from datetime import datetime
# Example: Caltrans public traffic camera feeds
CAMERA_FEEDS = {
    "I-405_LAX": {
        "url": "https://cwwp2.dot.ca.gov/data/d7/cctv/image/i405-lax/i405-lax.jpg",
        "lat": 33.9425,
        "lon": -118.4081,
    },
    "I-5_Downtown": {
        "url": "https://cwwp2.dot.ca.gov/data/d7/cctv/image/i5-downtown/i5-downtown.jpg",
        "lat": 34.0522,
        "lon": -118.2437,
    },
}
async def capture_frame(camera_id: str) -> dict:
    """Downloads a single JPEG frame from a public traffic camera."""
    cam = CAMERA_FEEDS[camera_id]
    async with httpx.AsyncClient() as client:
        resp = await client.get(cam["url"])
        return {
            "camera_id": camera_id,
            "image_bytes": resp.content,
            "lat": cam["lat"],
            "lon": cam["lon"],
            "captured_at": datetime.utcnow().isoformat(),
        }

# backend/ingestion/satellite.py
import httpx
SENTINEL_WMS = "https://services.sentinel-hub.com/ogc/wms/{instance_id}"
async def fetch_satellite_tile(bbox: list, width: int = 1024, height: int = 1024) -> bytes:
    """
    Fetches a recent Sentinel-2 satellite tile for a bounding box.
    bbox: [min_lon, min_lat, max_lon, max_lat]
    Free tier: 30,000 requests/month
    """
    params = {
        "SERVICE": "WMS",
        "REQUEST": "GetMap",
        "LAYERS": "TRUE_COLOR",
        "BBOX": ",".join(map(str, bbox)),
        "WIDTH": width,
        "HEIGHT": height,
        "FORMAT": "image/jpeg",
        "CRS": "EPSG:4326",
        "TIME": "2026-02-01/2026-02-20",  # recent range
    }
    async with httpx.AsyncClient() as client:
        resp = await client.get(SENTINEL_WMS, params=params)
        return resp.content

# backend/analysis/gemini_client.py
import google.generativeai as genai
import json
import base64
from config import GEMINI_API_KEY
genai.configure(api_key=GEMINI_API_KEY)
PANOPTIC_SYSTEM_PROMPT = """You are an advanced geospatial analyst model.
Analyze the provided image and detect ALL visible objects in these categories:
- vehicles (cars, trucks, buses, motorcycles)
- aircraft (planes, helicopters)
- pedestrians
- infrastructure (bridges, intersections)
For each detected object, return:
1. category (string)
2. estimated_lat and estimated_lon (float) — infer from camera metadata provided
3. confidence (float, 0-1)
4. bounding_box (optional, [x1, y1, x2, y2] in pixel coords)
5. attributes (color, direction, estimated_speed if moving)
Return ONLY valid JSON. No markdown. No explanation."""
async def analyze_frame(
    image_bytes: bytes,
    camera_lat: float,
    camera_lon: float,
    camera_heading: float = 0,
    fov_degrees: float = 90,
) -> list[dict]:
    """
    Sends a camera frame to Gemini 3.1 for panoptic detection.
    Camera metadata helps Gemini estimate real-world coordinates.
    """
    model = genai.GenerativeModel("gemini-3.1-pro")
    context = f"""Camera metadata:
    - Position: ({camera_lat}, {camera_lon})
    - Heading: {camera_heading}° from North
    - Field of view: {fov_degrees}°
    - Image type: Traffic camera JPEG snapshot
    Use this metadata to estimate real-world lat/lon for each detected object."""
    response = model.generate_content(
        [
            PANOPTIC_SYSTEM_PROMPT,
            context,
            {"mime_type": "image/jpeg", "data": base64.b64encode(image_bytes).decode()},
        ],
        generation_config={"response_mime_type": "application/json"},
    )
    detections = json.loads(response.text)
    return detections if isinstance(detections, list) else detections.get("detections", [])

# backend/analysis/panoptic.py
import asyncio
from ingestion.traffic_cams import capture_frame, CAMERA_FEEDS
from ingestion.opensky import fetch_aircraft
from analysis.gemini_client import analyze_frame
async def run_detection_cycle() -> dict:
    """
    One full detection cycle:
    1. Pull aircraft data from OpenSky (structured API — no vision needed)
    2. Capture frames from all traffic cameras
    3. Send each frame to Gemini for panoptic detection
    4. Merge all results into a single GeoJSON payload
    """
    # Aircraft data is already structured — no Gemini needed
    aircraft = await fetch_aircraft()
    # Traffic cam analysis — this is where Gemini earns its keep
    camera_tasks = []
    for cam_id, cam_info in CAMERA_FEEDS.items():
        frame = await capture_frame(cam_id)
        camera_tasks.append(
            analyze_frame(
                image_bytes=frame["image_bytes"],
                camera_lat=frame["lat"],
                camera_lon=frame["lon"],
            )
        )
    all_detections = await asyncio.gather(*camera_tasks)
    # Flatten into unified GeoJSON
    features = []
    # Add aircraft as features
    for ac in aircraft:
        if ac.latitude and ac.longitude:
            features.append({
                "type": "Feature",
                "geometry": {"type": "Point", "coordinates": [ac.longitude, ac.latitude]},
                "properties": {
                    "category": "aircraft",
                    "callsign": ac.callsign,
                    "altitude": ac.altitude,
                    "velocity": ac.velocity,
                    "heading": ac.heading,
                    "source": "opensky",
                },
            })
    # Add Gemini detections as features
    for cam_id, detections in zip(CAMERA_FEEDS.keys(), all_detections):
        for det in detections:
            features.append({
                "type": "Feature",
                "geometry": {
                    "type": "Point",
                    "coordinates": [det["estimated_lon"], det["estimated_lat"]],
                },
                "properties": {
                    **det,
                    "source": f"camera:{cam_id}",
                    "source_model": "gemini-3.1-pro",
                },
            })
    return {"type": "FeatureCollection", "features": features}

# backend/main.py
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from fastapi.middleware.cors import CORSMiddleware
import asyncio
import json
from analysis.panoptic import run_detection_cycle
app = FastAPI(title="Geospatial Tracker")
app.add_middleware(CORSMiddleware, allow_origins=["*"], allow_methods=["*"], allow_headers=["*"])
connected_clients: list[WebSocket] = []
@app.websocket("/ws/live")
async def websocket_endpoint(ws: WebSocket):
    await ws.accept()
    connected_clients.append(ws)
    try:
        while True:
            await ws.receive_text()  # keep-alive
    except WebSocketDisconnect:
        connected_clients.remove(ws)
async def broadcast_loop():
    """Runs every 10 seconds — pulls data, analyzes, broadcasts."""
    while True:
        try:
            geojson = await run_detection_cycle()
            payload = json.dumps(geojson)
            for client in connected_clients.copy():
                try:
                    await client.send_text(payload)
                except:
                    connected_clients.remove(client)
        except Exception as e:
            print(f"Cycle error: {e}")
        await asyncio.sleep(10)  # adjust polling interval
@app.on_event("startup")
async def startup():
    asyncio.create_task(broadcast_loop())

// frontend/src/components/LiveMap.tsx
import { useEffect, useRef, useState } from "react";
import mapboxgl from "mapbox-gl";
import "mapbox-gl/dist/mapbox-gl.css";
mapboxgl.accessToken = import.meta.env.VITE_MAPBOX_TOKEN;
export default function LiveMap() {
  const mapContainer = useRef<HTMLDivElement>(null);
  const map = useRef<mapboxgl.Map | null>(null);
  const [stats, setStats] = useState({ aircraft: 0, vehicles: 0 });
  useEffect(() => {
    map.current = new mapboxgl.Map({
      container: mapContainer.current!,
      style: "mapbox://styles/mapbox/dark-v11",
      center: [-118.25, 34.05], // Los Angeles
      zoom: 10,
    });
    map.current.on("load", () => {
      // Add empty GeoJSON source — gets updated via WebSocket
      map.current!.addSource("detections", {
        type: "geojson",
        data: { type: "FeatureCollection", features: [] },
      });
      // Aircraft layer — larger icons, colored by altitude
      map.current!.addLayer({
        id: "aircraft-layer",
        type: "circle",
        source: "detections",
        filter: ["==", ["get", "category"], "aircraft"],
        paint: {
          "circle-radius": 8,
          "circle-color": [
            "interpolate", ["linear"], ["get", "altitude"],
            0, "#00ff88",       // ground level = green
            5000, "#ffaa00",    // mid-altitude = orange
            12000, "#ff0044",   // cruise altitude = red
          ],
          "circle-stroke-width": 2,
          "circle-stroke-color": "#ffffff",
        },
      });
      // Vehicle layer — smaller dots from camera detections
      map.current!.addLayer({
        id: "vehicle-layer",
        type: "circle",
        source: "detections",
        filter: ["==", ["get", "category"], "vehicles"],
        paint: {
          "circle-radius": 4,
          "circle-color": "#00d4ff",
          "circle-opacity": 0.8,
        },
      });
    });
    // WebSocket connection
    const ws = new WebSocket("ws://localhost:8000/ws/live");
    ws.onmessage = (event) => {
      const geojson = JSON.parse(event.data);
      // Update map source
      const source = map.current!.getSource("detections") as mapboxgl.GeoJSONSource;
      if (source) source.setData(geojson);
      // Update stats
      const features = geojson.features || [];
      setStats({
        aircraft: features.filter((f: any) => f.properties.category === "aircraft").length,
        vehicles: features.filter((f: any) => f.properties.category === "vehicles").length,
      });
    };
    return () => {
      ws.close();
      map.current?.remove();
    };
  }, []);
  return (
    <div style={{ position: "relative", width: "100vw", height: "100vh" }}>
      <div ref={mapContainer} style={{ width: "100%", height: "100%" }} />
      {/* HUD overlay */}
      <div style={{
        position: "absolute", top: 16, left: 16,
        background: "rgba(0,0,0,0.8)", color: "#0f0",
        padding: "12px 20px", borderRadius: 8, fontFamily: "monospace",
      }}>
        <div>✈ AIRCRAFT TRACKED: {stats.aircraft}</div>
        <div>🚗 VEHICLES DETECTED: {stats.vehicles}</div>
        <div style={{ fontSize: 10, opacity: 0.6 }}>LIVE • 10s refresh</div>
      </div>
    </div>
  );
}

# backend/models/schemas.py
from pydantic import BaseModel, Field
from typing import Optional
class AircraftPosition(BaseModel):
    icao24: str
    callsign: str = ""
    origin_country: str = ""
    longitude: Optional[float] = None
    latitude: Optional[float] = None
    altitude: Optional[float] = None
    velocity: Optional[float] = None
    heading: Optional[float] = None
    vertical_rate: Optional[float] = None
    on_ground: bool = False
    last_contact: Optional[int] = None
class Detection(BaseModel):
    category: str = Field(..., description="vehicle, aircraft, pedestrian, etc.")
    estimated_lat: float = Field(..., ge=-90, le=90)
    estimated_lon: float = Field(..., ge=-180, le=180)
    confidence: float = Field(..., ge=0, le=1)
    bounding_box: Optional[list[float]] = None
    attributes: dict = Field(default_factory=dict)
class DetectionResponse(BaseModel):
    """Validates Gemini's entire response before it hits your map."""
    detections: list[Detection]

from models.schemas import DetectionResponse
# After getting raw JSON from Gemini:
validated = DetectionResponse(detections=raw_json)
return validated.detections  # guaranteed clean data

# .env
GEMINI_API_KEY=your_gemini_key
MAPBOX_TOKEN=your_mapbox_token
# Terminal 1 — Backend
cd backend && uvicorn main:app --reload --port 8000
# Terminal 2 — Frontend
cd frontend && npm run dev

Open http://localhost:5173. You should see a dark map with live aircraft dots appearing within seconds, and vehicle detections populating as camera frames are analysed.

Cost reality check

Component	Cost
OpenSky API	Free (rate-limited)
Gemini 3.1 Pro (vision)	~$0.002/frame analysed
Sentinel Hub (satellite)	Free tier — 30k req/month
Mapbox	Free tier — 50k loads/month
Claude 4.6 (generating all this code)	~$0.30 total for the full project

Running 6 cameras at 10-second intervals = ~$3.10/day in Gemini API costs. That's a CIA-grade surveillance dashboard for the price of a coffee.

Key takeaway

The power isn't in either model alone — it's in the routing. Claude is your architect and engineer. Gemini is your analyst with superhuman vision. The MCP bridge (RLabs-Inc/gemini-mcp, eLyiN/gemini-bridge) is the nervous system connecting them. This is how real AI-native applications are built in 2026.

Credit for the geospatial video: Bilawal Sidhu (ex-Google PM) on X. Helpful resource: Altari's pre-built sales & marketing AI agents — run research, outreach, and content out of the box, no coding required.

Original guide by Artem Novitckii. More from him on Instagram, in the Skool community, and across his resource library.

The ultimate AI stack.

The full build blueprint

Get the full blueprint, free.

Why the Gemini + Claude combo is overpowered

How to connect them

Method 1: The clink CLI bridge (easiest for devs)

Method 2: Custom bash wrapper script (no extra dependencies)

Method 3: Enterprise integration via Composio MCP

Blueprint: recreating the geospatial tracker

Project structure

Step 1: Real-time data ingestion pipeline (Claude 4.6)

Step 2: Visual panoptic detection (Gemini 3.1)

Step 3: The WebSocket hub (Claude 4.6)

Step 4: The frontend map (Claude 4.6)

Step 5: Pydantic schemas — the glue that prevents chaos

Step 6: Run it

Cost reality check

Key takeaway