Jeronimo
Full-Stack / ML
compiling

Full-Stack / ML Engineer

Jeronimo

I design, build and ship complete systems end to end — ML & GPU pipelines, real-time backends, and the interfaces on top.

scroll to explore

Farmium — my company

The product I co-founded and engineer end to end.

A workflow-automation platform with the architectural depth of systems software — three faces of one product. Use the arrows to step through each panel.

Farmium — Platform

Co-Founder · Production platform

A multi-tenant workflow automation platform whose execution engine borrows tricks from CPU architecture

Farmium is a production SaaS for composing, versioning, and running complex automation workflows as directed acyclic graphs. The core is a half-million-line TypeScript/Vue platform whose parallel DAG engine does cost-aware speculative execution, Merkle-style computation caching, and Bayesian/evolutionary optimization — wrapped in real-time collaborative editing and a full observability stack.

NestJS · TypeScript · Vue 3 · DAG execution engine · CRDT (Yjs) · WebSockets · SHA-256 content-addressed cache · Bayesian/evolutionary optimization · Prometheus · Grafana · distributed tracing

Scale of the codebase

Half a million lines of typed code, organized into a 54-module backend and a 409-component frontend

The metrics panel captures the operating envelope this platform is engineered around. The backend is roughly 290K lines across 670 TypeScript files — 200+ services and 93 controllers over 110 database models grouped into 54 modules. The frontend adds about 200K lines across 409 Vue 3 components, including a collaborative visual editor. The DAG graph itself supports 60+ strongly-typed node kinds with static type inference and runtime type guards. These are the real dimensions the architecture has to stay coherent across, not a marketing rounding.

Execution engine

A parallel DAG engine with CPU-style speculative execution at its core

The detail panel goes deep on the heart of the platform: a parallel DAG execution engine spanning 152 files and roughly 68K LOC. It owns the full execution lifecycle — scheduling and running independent nodes concurrently, threading pipelines, evaluating conditional branches, and driving control flow like loops, switches, and approval gates. On top of that sits something unusual for a SaaS: cost-aware speculative execution. A 2-bit saturating-counter Branch Target Buffer, modeled directly on CPU branch predictors, learns which branches tend to be taken and pre-runs them — but only when the expected savings justify the cost, so it never speculates into work too expensive to discard.

Caching & optimization

A Merkle-style computation cache, then a search engine layered on top of the workflow itself

This detail panel covers how expensive work is never repeated, and how workflows tune themselves. Expensive node outputs are deduplicated through a Merkle-style computation cache: each key is a composable SHA-256 over workflowId, nodeType, codeVersion, config, and inputs, with deterministic canonicalization so identical work collapses to a single entry — even across different pipelines. Layered above it is an optimization system that treats a workflow as a search space, using A/B testing plus Bayesian and evolutionary search to tune node parameters against a multi-objective score balancing quality, cost, and time.

Real-time collaboration

A collaborative visual editor with IDE-grade inspection over live execution

The detail panel describes the front-of-house: the visual editor is collaborative in real time. CRDT-based editing (Yjs) lets multiple people shape the same workflow without conflicts, while WebSocket gateways stream live execution progress, per-node state transitions, and notifications back to every connected client. The frontend pairs this with git-style visual version control and diffing, a node-level debugging panel that previews each node's output, and a batch-processing UI — so authoring feels like drag-and-drop but ships with the inspection tools of an IDE.

Security & observability

Behavioral security intelligence and end-to-end observability across the whole estate

The final detail panel covers the operational backbone. A behavioral security engine profiles login activity to flag anomalies — impossible travel, brute force, credential stuffing — then risk-scores sessions and triggers automated responses. Billing stays correct under concurrency through atomic credit deduction with deadlock detection, retry, and idempotency. And everything is observable end to end: Prometheus metrics, Grafana dashboards, distributed tracing, error tracking, and centralized log aggregation across the 54-module backend, so failures are seen before they spread.

290KLOC

Backend (670 TS files)

200KLOC

Frontend (409 Vue 3 components)

200+

Services

Controllers

110

DB models

Backend modules

60+

Typed node kinds

68KLOC

DAG engine (152 files)

Speculative execution, predictor-style

The engine (152 files, ~68K LOC) drives node scheduling, pipelines, conditional branch evaluation, and full control flow including loops, switch nodes, and approval gates. Its speculative layer uses a 2-bit saturating-counter Branch Target Buffer, the same structure CPUs use to predict branches, to pre-run the branches most likely to be taken. Crucially it is cost-aware: it refuses to speculate when a branch is too expensive to run ahead or saves too little to be worth discarding. The whole graph is built from 60+ strongly-typed node kinds with static type inference plus runtime type guards, so type safety holds across the node graph at both compile and run time.

DAG engineSpeculative executionBranch Target BufferParallel schedulingType inference

Content-addressed cache plus multi-objective search

Cache keys are composable SHA-256 hashes over workflowId, nodeType, codeVersion, config, and inputs, canonicalized deterministically so two runs that are genuinely identical share one cached result and expensive outputs dedup cross-pipeline. On top sits a workflow optimization layer: A/B testing combined with Bayesian and evolutionary parameter search explores the configuration space, scored against a multi-objective function that weighs output quality against cost and execution time. The result is a platform that both avoids recomputing what it has already computed and actively converges workflows toward better quality/cost/time trade-offs.

Merkle cacheSHA-256 keysCross-pipeline dedupBayesian searchEvolutionary searchMulti-objective scoring

CRDT editing, live streaming, visual versioning

Concurrent editing is conflict-free via Yjs CRDTs, so several authors can edit one workflow simultaneously without stepping on each other. WebSocket gateways push execution progress, per-node state changes, and notifications to all clients in real time. Beyond authoring, the editor offers git-style visual version control with diffing to compare workflow revisions, a per-node debug panel that previews the actual output of each node, and a batch-processing UI for running workflows across many inputs at once — drag-and-drop authoring backed by the inspection surface of an IDE.

CRDT (Yjs)WebSocket gatewaysLive execution stateVisual versioningPer-node debug previewBatch UI

Anomaly detection, atomic billing, full-stack telemetry

The security intelligence engine does behavioral login-anomaly detection — impossible-travel, brute-force, and credential-stuffing patterns — feeding a risk score that drives automated response. On the billing side, credit deduction is atomic with deadlock detection, retry, and idempotency, so charges stay correct even under heavy concurrency. Observability is wired through the entire 54-module backend: Prometheus metrics, Grafana dashboards, distributed tracing, error tracking, and centralized log aggregation, giving a single coherent view of execution health from the node graph down to the infrastructure.

Anomaly detectionRisk scoringAutomated responseAtomic creditsDeadlock detectionIdempotencyMetrics + tracing + logs

Multi-Tenant B2B Integration API

Senior Research Engineer

A hardened B2B API other businesses build on top of

This is the integration surface external teams program against: every request is isolated to its tenant, authenticated, safe to retry, and impossible to silently lose. The panel at right frames the operational footprint, engineered to fail closed without dropping data rather than fail open under pressure.

TypeScript (NestJS 11), PostgreSQL, distributed cache, HMAC-SHA256, object storage, class-validator, end-to-end test suite

Senior Research Engineer

Per-tenant isolation enforced at the query layer, not just the edge

Tenancy is not a convention layered on at the perimeter: every record carries a tenant scope enforced down at the query layer, and inbound callers are authenticated with hashed keys compared in constant time. The trust boundary sits at the very front of the system.

Senior Research Engineer

Resilience: it stays up when the cache or a dependency degrades

Rate limiting and dependency isolation are written to degrade gracefully rather than collapse, and a downstream outage is contained before it can cascade across the API. Correctness and availability are preserved together, not traded off.

Senior Research Engineer

Idempotency, event sourcing and signed webhooks make retries safe

Because clients and networks retry, the same logical operation must land at most once in effect, and every state change must be both recoverable and observable. Inbound requests dedupe; outbound notifications are signed and replay-protected.

Senior Research Engineer

Distributed agents claim work lock-free, behind a full audit trail

A fleet of agents pulls jobs without lock contention, uploads bypass the API for large media, and the entire surface is wrapped in immutable audit logging and OWASP hardening. The path from request to effect is reconstructable after the fact.

35+

REST endpoints

Granular auth scopes

Job lifecycle states

Circuit-breaker states

7+jitter

Webhook retry attempts

HMACSHA-256

Key hashing + signed webhooks

Claimant per job (lock-free)

E2E

Test suite + OWASP hardening

Isolation by construction, keys hashed and compared in constant time

Every record carries a mandatory tenant scope that is enforced at the query layer, with uniqueness constraints applied per tenant, so two clients can never collide or read across the boundary even if they request the same logical resource. Authentication uses API keys hashed with HMAC-SHA256 over a server-side pepper, never stored in the clear, and verified with constant-time comparison to resist timing side-channels. A TTL-backed key cache with an in-memory fallback keeps the auth path fast under load, and revocation invalidates cached entries immediately so a withdrawn key stops working at once rather than lingering until expiry. Authorisation is carved into ten granular scopes, so a credential grants exactly the surface it needs and nothing more.

Tenant scoping at query layerHashed keys + pepper10 granular scopes

Degrade in place, isolate failure, never fail open

Rate limiting runs as a sliding window in the distributed cache and emits standard HTTP rate-limit headers, but if that cache drops away it degrades to an in-memory limiter so the API keeps enforcing limits and stays available instead of failing open. Each external dependency sits behind a custom three-state circuit breaker (CLOSED, OPEN, HALF_OPEN) driven by sliding-window error metrics: when a downstream service starts failing the breaker trips OPEN to isolate it before the failure cascades, then probes cautiously in HALF_OPEN and only recovers to CLOSED once the dependency proves healthy again. The pattern keeps a single sick downstream from dragging the whole surface down, and keeps the blast radius of any one outage contained.

Sliding-window rate limitIn-memory fallbackCLOSED / OPEN / HALF_OPEN

Dedupe on the way in, sign and persist on the way out

Operations are safe to retry by design: an idempotency key combined with automatic payload hashing makes a replayed submission resolve to the existing operation rather than spawning a duplicate, so clients can retry freely after a timeout. Underneath, append-only event tables and an immutable history record every state transition, giving full auditability and a recoverable record that nothing can silently erase. On the outbound side, webhooks are signed with HMAC-SHA256 and protected against replay by a timestamp, so receivers can verify a callback genuinely came from the platform. Delivery is at-least-once with capped exponential backoff plus jitter across seven attempts, and every attempt is persisted in a delivery history, so a transient receiver outage is retried without thundering-herd pile-ups and without losing the event.

Idempotency + payload hashingAppend-only event log7 retries + jitter

One claimant per job, every action on the record

A fleet of distributed agents pulls work with lock-free atomic claiming: a single conditional bulk update on job status guarantees exactly one claimant with no SELECT FOR UPDATE and no race across the fleet, so adding workers scales throughput without introducing claim contention. Jobs move through nine explicit lifecycle states, making progress legible at every step. Large media bypasses the request path entirely through direct-to-object-storage uploads via presigned URLs, multipart, or import-from-URL, each with transactional rollback and media dedup so a failed upload leaves no orphaned state. The whole surface is wrapped in immutable audit logging and OWASP hardening: security headers, CSP and HSTS, SSRF guards, strict DTO validation, and liveness and readiness health checks, all exercised by an end-to-end test suite so nothing meaningful happens off the record.

Lock-free atomic claim9 job statesImmutable audit + SSRF guards

Farmium — GPU Media

Co-Founder · GPU media

A serverless GPU media stack

A Python/CUDA/FFmpeg media engine on serverless GPU that turns raw images, audio and video into production-ready assets across multiple GPUs. Tiling and FP16/FP32 fallbacks keep work alive under OOM pressure.

Python · CUDA · FFmpeg · Real-ESRGAN · U2-Net · Stable Video Diffusion · CLIP · MediaPipe · serverless GPU · object storage

Co-Founder · GPU media

Super-resolution + segmentation

The pixel-level path: 4x super-resolution (Real-ESRGAN) upscales images and a U2-Net stage cuts subjects out of them. Both share an OOM-aware execution discipline so oversized inputs never blow GPU memory.

Co-Founder · GPU media

Video, generative + audio

The motion and synthesis layer: existing footage is reframed and re-composited straight through FFmpeg, while generative models produce new video and voice. Both adapt to whatever GPU resources are available at runtime.

Co-Founder · GPU media

Real-time captioning + multi-GPU orchestration

The throughput engine: a multi-model ensemble drives captioning at 30fps. The orchestration layer pools encoding sessions, fans I/O across hundreds of workers, and moves large files over a verifiable, retry-safe content-addressable protocol.

Co-Founder · GPU media

High-volume variation + media augmentation

Variation at catalog scale: one source is re-encoded, recomposited and reframed into many output shapes programmatically. One capability produces perceptually distinct variants while keeping visual fidelity intact, via frequency-domain optimization.

Super-resolution

30fps

Captioning throughput

200

Parallel I/O workers

10/GPU

Encoding sessions per GPU

FP16/FP32

OOM-aware precision switching

SHA-256

Content-addressed transfer

Learned models (Real-ESRGAN, U2-Net)

DCT/wavelet

Perceptual augmentation

Pixel-perfect upscaling and cutouts

The 4x super-resolution path uses Real-ESRGAN with tile-based processing: very large images are split into tiles so even oversized inputs never exceed GPU memory, with the tiles reassembled into a seamless result. The pipeline switches between FP16 and FP32 on the fly, trading memory headroom against output quality as the input demands — and falling back rather than failing when a GPU is constrained. Alongside it, a U2-Net semantic segmentation stage with alpha-matting produces per-pixel subject cutouts clean enough to composite straight onto new backgrounds, isolating foreground subjects from arbitrary scenes without manual masking.

Real-ESRGAN 4xtile-based processingU2-Net segmentationalpha-mattingFP16/FP32 switchingOOM-aware fallback

Reframing, generation and voice

Video is reframed with content-aware cropping piped directly through FFmpeg — no intermediate files ever hit disk — with background blur and SVG logo compositing layered into the same pass, so a single source becomes correctly-framed, branded output in one streamed operation. On the generative side, the stack produces video via Stable Video Diffusion and synthesizes or clones voice with TTS. The backend adapts automatically to available GPU resources, selecting an execution path that fits whatever hardware the serverless layer assigns rather than assuming a fixed device profile.

FFmpeg direct pipingcontent-aware reframebackground blurSVG logo compositingStable Video DiffusionTTS synthesis/cloningadaptive GPU backend

30fps captioning on a multi-GPU fabric

Captioning runs on a multi-model ensemble — CLIP for semantics combined with MediaPipe face mesh — sustained at 30fps. Underneath, the orchestration layer pools encoding sessions per hardware type (up to ~10 per datacenter GPU), fans work out to roughly 200 parallel I/O workers, and caches CUDA graphs to cut per-frame launch overhead. Large-file movement rides a content-addressable transfer protocol: SHA-256 verification on every object, atomic upload-then-confirm against a manifest, and idempotent retry with backoff — so a file is either fully present and verified or safely retried, never half-written or silently lost.

CLIP + MediaPipe ensemble30fpsper-hardware session pooling (~10/GPU)~200 parallel I/O workersCUDA graph cachingSHA-256 verificationatomic upload-then-confirmidempotent retry + backoff

Content variation at scale

For high-throughput catalogs the stack generates large numbers of media variants programmatically — re-encoding, recompositing and reframing a single source into many output shapes for different placements and formats. Among these is a perceptual media augmentation engine: it produces perceptually distinct variants of an image or clip while keeping visual fidelity intact, using frequency-domain (DCT/wavelet) optimization to vary the signal where it matters least to a viewer. It sits as one capability within the broader variation toolkit, sharing the same multi-GPU orchestration and content-addressable transfer as the rest of the pipeline.

programmatic variant generationbatch re-encode/recomposite/reframeperceptual augmentationDCT/wavelet frequency-domainfidelity-preservinghigh-throughput catalog

Selected client & research work

Engagements and independent research — applied ML, scientific tooling, and robustness. Client domains stay at the capability level.

Asynchronous Multi-Modal Sensor Fusion Platform

Senior ML Research Engineer

Asynchronous Multi-Modal Sensor Fusion

A real-time fusion engine that ingests four independent sensing modalities at mismatched rates and reconciles them into a single coherent state estimate. The live panel shows the end-to-end pipeline: heterogeneous streams arriving asynchronously, time-aligned, and fused into one track without forcing a common clock.

Python, C++, PyTorch, ONNX Runtime, embedded edge inference, real-time DSP, Kalman filtering, CI/CD

Senior ML Research Engineer

The numbers that define the envelope

The platform is engineered to run inside a tight latency and footprint budget while combining many estimators. The right panel collects the headline figures that bound the system: modality count, explainability latency, estimator breadth, and the deployed model size on edge hardware.

Senior ML Research Engineer

Asynchronous fusion across four modalities

Four independent modalities never arrive on the same schedule. The right panel details how the engine treats time as a first-class input: each stream carries its own timestamps and rates, and the fusion core aligns them on demand instead of forcing a lowest-common-denominator clock.

Senior ML Research Engineer

Dual-band CNN with explicit calibration

One modality is driven by a learned model rather than a hand-tuned filter. The right panel describes the dual-band convolutional network and the calibration layer that keeps its confidence honest, so downstream fusion can trust the weights it assigns to that channel.

Senior ML Research Engineer

Sub-25ms explainability in the loop

Every fused decision ships with a reason, not after the fact but inside the real-time budget. The right panel explains how explainability is treated as a hard latency constraint — under 25 milliseconds per decision — so operators see why a state was produced as it is produced.

Senior ML Research Engineer

Twenty-plus estimators under one orchestrator

Fusion is not a single algorithm but an ensemble. The right panel breaks down the estimator stack — direction, time-difference, spatial, and recursive-tracking families — and how a coordinating layer arbitrates between more than twenty of them at runtime.

Senior ML Research Engineer

8.5MB: the whole stack runs at the edge

All of the above fits on constrained hardware. The right panel covers the deployment story — an 8.5MB model footprint, optimized inference, and the packaging discipline that lets the full fusion stack run on edge devices rather than in a datacenter.

4async

Sensor modalities fused

8.5MB

Edge model (FP16)

66ms

p99 latency, 5 tracks

25ms

Attribution latency (under)

70x

Calibration error reduction

20+

Classical estimators

Under per-model size budget

2band

Dual-band SE-CNN inputs

Four streams, one coherent state

The core problem is temporal: each of the four modalities samples the world at a different cadence, with independent jitter and drop-out behaviour, and none can be assumed synchronous. The engine assigns every measurement its own timestamp and routes it through a per-modality intake stage that handles rate conversion, gating, and out-of-order arrival. A central estimator then performs time-aligned fusion, predicting the shared state forward to each measurement's instant and updating incrementally, so a late or sparse stream degrades gracefully rather than stalling the pipeline. The result is a single continuously-maintained estimate that stays valid even when one or more modalities momentarily disappear, with no global synchronisation barrier and no buffering penalty imposed by the slowest sensor.

asynchronoustime-alignment4-modalitiesgraceful-degradation

Learned perception, calibrated outputs

A convolutional network processes two complementary bands jointly, exploiting the structure shared across them while preserving each band's distinct signal characteristics. Rather than emit raw scores, the network is paired with an explicit calibration stage so that the confidence it reports corresponds to real reliability — a precondition for principled fusion, since the central estimator weights each modality by how trustworthy it claims to be. Calibration is validated as a first-class metric, not an afterthought: a miscalibrated channel that is overconfident would silently dominate the fused state, so the pipeline measures and corrects this before the model's output ever reaches the estimator. The dual-band design and calibration together turn a learned component into a well-behaved sensor that the rest of the system can reason about quantitatively.

CNNdual-bandcalibrationconfidence

Explainability as a real-time requirement

Explainability is engineered as part of the inference path, bounded to under 25 milliseconds per decision, so the reasoning behind each fused output is available at the same cadence as the output itself. The design avoids post-hoc methods that would blow the time budget; instead the contributing factors — which modalities drove the estimate, how much each was weighted, and where confidence concentrated — are extracted directly from the fusion step. Holding XAI inside such a tight envelope is a systems-engineering result, not just a modelling one: it required structuring the estimator so that attribution is a cheap by-product of fusion rather than a separate, expensive analysis pass. This keeps the system auditable in operation, turning every decision into something an operator can interrogate without slowing the pipeline.

XAIexplainability<25msauditable

An orchestrated estimator ensemble

The fusion stack composes more than twenty distinct estimators spanning several families: subspace direction estimation (MUSIC-style), time-difference-of-arrival (TDOA), spatial beamforming, and recursive state tracking (Kalman-family) filters. No single estimator is robust across all operating conditions, so an orchestration layer runs the appropriate subset for the current context, cross-checks their outputs against one another, and feeds reconciled results into the central state estimate. This ensemble design is what makes the system resilient: when one technique degrades — geometry-poor conditions for direction finding, low SNR for TDOA — others compensate, and the orchestrator reweights accordingly. Managing twenty-plus estimators as a coherent, runtime-switchable stack rather than a pile of standalone scripts is the engineering backbone that lets the platform stay accurate across a wide range of inputs.

MUSICTDOAbeamformingKalmanensembleorchestration

Datacenter-grade fusion on an edge budget

The deployed model footprint is 8.5MB, small enough to run the complete fusion stack on constrained edge hardware with no cloud dependency in the decision loop. Reaching that footprint while preserving accuracy, sub-25ms explainability, and twenty-plus estimators required deliberate engineering across the whole pipeline: portable inference runtimes, careful operator selection, and a build-and-packaging process that keeps the learned components compact without retraining penalties. Running at the edge means the asynchronous four-modality fusion, the calibrated dual-band network, and the estimator ensemble all execute locally and in real time, so latency, availability, and data locality are all controlled rather than outsourced. The 8.5MB figure is the tangible proof that the system was designed for the field, not the lab — a full-capability platform that fits where it actually has to run.

edge8.5MBONNXreal-timeon-device