HR Automated Candidate Sourcing & Pipeline Sync

Context

A recruitment agency placing medical sales representatives, QA and regulatory-affairs specialists for pharmaceutical companies. Candidates arrived from several job boards, referral inboxes and LinkedIn outreach. Recruiters copied each profile by hand into the ATS, then again into the CRM the agency shared with its pharma clients.

The operational bottleneck

• Duplicate and stale profiles. The same candidate existed three or four times with different statuses, so clients occasionally received a profile that had already been placed elsewhere.
• Status drift. Interview outcomes recorded in the ATS reached the client CRM only when someone remembered to update it.
• Compliance exposure. Personal data was scattered across spreadsheets and mailboxes with no retention policy, a real risk in a regulated industry.

The cost was not the software; it was recruiter hours and slower submittals in a market where the first qualified shortlist usually wins the mandate.

Architecture

1. Ingestion workers (Python). Scheduled jobs poll job-board APIs; an inbound-mail parser extracts CVs and contact data from referral emails.
2. Normalisation & deduplication. Emails and phone numbers are canonicalised (E.164), then matched against existing profiles with deterministic keys and a fuzzy name/employer score for edge cases.
3. System of record. A PostgreSQL schema with an append-only audit trail of every status transition and its source.
4. Bi-directional sync. ATS webhooks push status changes in real time; the CRM is updated through its REST API using the OAuth 2.0 client-credentials flow, with token caching and automatic refresh.
5. Nightly reconciliation. A diff job compares all three systems and repairs anything a missed webhook left behind.

def candidate_key(email: str | None, phone: str | None) -> str | None:
    """Deterministic identity key: the same person always maps to one profile."""
    if email:
        return "e:" + email.strip().lower()
    if phone:
        return "p:" + to_e164(phone)
    return None  # falls through to fuzzy matching + human review queue

Security & data protection

• Role-based access control: recruiters see their own pipelines; client contacts see only submitted candidates.
• Sensitive fields encrypted at rest; secrets kept outside the codebase and rotated.
• An automated retention policy anonymises inactive profiles after the agreed period.

Results

The agency stopped re-typing candidate data altogether. Statuses now propagate from the ATS to the client CRM within minutes, duplicates are caught at ingestion instead of in front of the client, and recruiters spend the reclaimed time on sourcing and interviews: the work that actually closes placements.