B2B Prospecting Platform

The frontend is a React 18 SPA with no build step: JSX is transpiled in the browser and the API itself serves the page. I chose this so the deploy would be a single container, with no frontend pipeline. The cost shows up in a slower first load and in the lack of typing and component tests.

The split between the two databases follows who owns the data. The remote PostgreSQL is shared with other systems and says what already exists. SQLite stores what only matters to this application, so it sits on a Docker volume, outside the container, to survive a rebuild.

Login is OIDC with PKCE against the company SSO, with a secure cookie by default. Roles (user and admin) and monthly credits live in the platform itself, because the SSO says who the person is, not what they can spend. The test suite runs in a CI workflow.

The order is fixed. The SQLite cache answers if it has a record less than 120 days old. Without one, the lookup goes to the official registry API. If the official one fails, and the admin allows it, three public providers are tried in sequence. The rule set (active, head office, size, founded 3 or more years ago, legal nature other than sole proprietor) is applied after the response, the same at every layer, so the choice of provider does not change the verdict.

The price of the cache is staleness: a company whose status changes within the window is only seen on the next trip to the source. I accepted that because the same company is looked up many times by different users, and every trip to the source pays latency and rate limits.

Gemini researches the described market and suggests economic-activity codes (CNAE). From those the platform extracts companies, runs each one through the registry rule set and removes what already exists in the source of truth. Secondary CNAEs come in only if the user ticks the option. Generation runs in the background, and lists are saved: a reload does not lose the work.

Each user has a lock against concurrent generation, so two of their generations never compete for the same credits and the same companies. When generation runs out of new companies, it falls back to the rescue and moves deals that already exist in the CRM, instead of creating duplicates. That path is best effort: if it fails, generation delivers what it has and does not crash.

The import is asynchronous and gets a job id as soon as it starts. Rows are deduplicated and sent in throttled batches, so as not to hit the CRM’s rate limit. Transient failures are retried with exponential backoff. Progress goes to the browser over SSE.

Everything a job created is tied to its id, and rollback undoes only that job. What failed, and what was blocked for already existing in the source of truth, can be exported to a file for fixing and resending, instead of getting lost in the log.

What passes goes to the two-phase move. Phase A writes the tag, the platform waits a configurable interval, and phase B moves the deal and clears the owner. I split the phases after seeing the CRM send to deduplication, and delete, deals that got the tag and the move in the same write.

The idempotency key combines run and deal. A deal stuck between phases is recognized, and the resume resends only phase B. The credit is charged once per moved deal, guaranteed by a unique index on the credit ledger. One limit remains open: one of the pipelines silently reverts category changes made through the API, even while returning success. That led to the rule of taking a snapshot of every field before any write, so changes can be reverted with values that were read, not inferred.

27.4k deals were extracted, 14.8k of them with a known label. The split is temporal, 80/10/10 ordered by creation date, because a random split would leak the future into training. Categorical variables use target encoding with smoothing. Imbalance is handled with scale_pos_weight instead of SMOTE, the output goes through isotonic calibration, and every prediction comes with a SHAP explanation.

On the test set: ROC-AUC of 0.797, PR-AUC of 0.095 and precision@10 of 0.239. The PR-AUC looks low, but the positive class is rare, and the number has to be read against that base rate. A new model is only promoted in the registry if it wins on PR-AUC and keeps calibration error at or below 0.05. For concept drift, the training window dropped from 365 to 180 days, with a 60-day half-life on the weights.

Enrichment joins the deals against a 71-million-row table. The first version filtered with ANY() over a text array and fell into a full scan. Switching to a JOIN with unnest made the query 167× faster.

The first version of the similar-company search used all of the target company’s CNAEs as the seed. In a real case, one secondary transport code was enough for 87.5% of the candidates to be trucking companies. Today the seed is the primary CNAE, expanded through the official hierarchy with exact, group or sector scope, and secondary codes come in only as a user option.

Broad searches hit the 120 s timeout for two reasons: counts with no state filter and a secondary-CNAE index used even when it was not needed. Now the index is only used when the search asks for it, a pre-check rejects a scope that is too large with HTTP 422, and counts have a low cap: above it the screen shows “≥N” instead of waiting for the exact number.

On the map, import by city uses the exact city and state, with no spelling normalization, so the credit count matches what the map showed. The sales analysis tabs were ported from a colleague’s Streamlit apps, with a numeric check against the originals.