💼 Yojaka AI (Agentic Profile Matching Engine)¶

Production-Grade AI Recruiter & Profile Matching Engine built with LangGraph 1.x Native Commands, HyDE Query Synthesis, Parent-Document Chunking, Air-Gapped Local Inference (Ollama / vLLM), Calibrated Margin Routing, Parallel Dual-Rubric Subgraphs, Layout-Aware Section Parsing, Anthropic Contextual Retrieval, Two-Stage Cross-Encoder Reranking, Hybrid RAG, Dynamic Skill Expansion, Zero-Disk In-Memory Upload, and Model Context Protocol (MCP).

💡 What is Yojaka (योजक)?
In classical Sanskrit, योजक (Yojaka) derives from the root युज् (yuj) — meaning to connect, unite, align, or orchestrate. Rather than treating candidate vetting as a cold keyword gatekeeper, Yojaka AI operates as an intelligent orchestrator: parsing unstructured human potential, dynamically expanding semantic equivalences, and cascading through structured reasoning to match talent with purpose.

⚡ Core Highlights & Capabilities¶

• 🎯 3-Stage Cascading Funnel (O(N) → O(K)): Coarse hybrid vector/lexical retrieval (Stage 1) → structured LLM profile audit (Stage 2) → grounded decision synthesis with tailored interview questions (Stage 3). Eliminates rate-limit bottlenecks and token exhaustion.
• 🧬 Advanced RAG & Air-Gapped Local Inference (ADR-019): Hypothetical Document Embeddings (HyDE) query synthesis, hierarchical small-to-big parent-document chunk mapping (150–250 token child chunks linked to 1,000–1,500 character parent blocks), pre-retrieval faceted metadata filtering, and zero-cloud air-gapped local model inference via Ollama and vLLM.
• 🔀 Calibrated Margin Routing & Subgraphs (ADR-018): Mathematical embedding margin gating (\(\Delta = \text{Top1} - \text{Top2} \ge 0.12\)) dispatching confident queries in <2ms at $0 token cost, native LangGraph Command(goto=...) destination routing, concurrent dual-rubric candidate evaluation, and isolated typed subgraphs.
• 📑 Layout-Aware Section Document Parsing: Parses PDF visual layout blocks (pymupdf), DOCX paragraph runs, and text delimiters into canonical sections (SUMMARY, EXPERIENCE, SKILLS, EDUCATION), preserving multi-role work histories without mid-sentence truncation.
• 🧠 Anthropic Contextual Retrieval Prepending: Situates isolated resume chunks with 50–80 word document metadata banners before dense embedding and BM25 indexing, preserving pristine candidate text for recruiter UI display.
• ⚖️ Two-Stage Hybrid Retrieval & Cross-Encoder Reranking: Combines dense vector cosine similarity with BM25 Okapi lexical scoring, followed by fine-grained cross-encoder/ms-marco-MiniLM-L-6-v2 reranking with Sigmoid score calibration and Reciprocal Rank Fusion (RRF).
• 🔒 Zero-Disk In-Memory Upload & Ephemeral PII Isolation: Ingests .pdf, .docx, and .txt files directly in memory via io.BytesIO layered through CompositeVectorStore with session-scoped in-memory vector stores (ADR-016).
• 🖥️ Dual Presentation Layer: Clean <75-LOC Streamlit conductor (app.py) featuring st.write_stream typewriter streaming and live tool execution tracing, coupled with a headless FastAPI sidecar (api/) exposing REST and Server-Sent Events (SSE) endpoints.
• 🔌 Model Context Protocol (MCP) Dual Gateway: Hot-swap between in-process tool execution and FastMCP JSON-RPC stdio servers (ADR-001).

🏛️ System Architecture¶

📚 Documentation & Deep Dive: For comprehensive interactive dataflow diagrams, mathematical scoring formulations, and security specifications, explore Technical Architecture or visit the Online Documentation Site.

🎯 3-Stage Cascading Screening Funnel¶

Candidate evaluation cascades across 3 tiers to optimize LLM token consumption O(N) → O(K):

🧩 Supported Capabilities & Tech Stack¶

⚡ 60-Second Quickstart¶

1. Clone & Setup Environment¶

# Clone the repository
git clone https://github.com/shashankch/yojaka-ai-profile-matching-engine.git
cd yojaka-ai-profile-matching-engine

# Create and activate virtual environment (Python 3.10+)
python3 -m venv .venv
source .venv/bin/activate

# Install in editable mode with dependencies
pip install -e .

2. Configure Credentials¶

cp .env.example .env

💡 For the complete configuration reference, see Architecture Section 14 and .env.example.

3. Generate Mock Data & Ingest¶

# Generate synthetic multi-format candidate resumes (34 profiles: PDF, DOCX, TXT)
python -m agentic_profile_matching.generate_dataset

# Chunk, enrich, and vector-index candidate profiles into ChromaDB (Idempotent)
python -m agentic_profile_matching.resume_rag

4. Launch Application¶

Option A: Interactive Streamlit Recruiter Dashboard

streamlit run src/agentic_profile_matching/app.py

Option B: Headless FastAPI Gateway Sidecar

python -m agentic_profile_matching.api.main
# Or via uvicorn directly:
uvicorn agentic_profile_matching.api.app:app --host 0.0.0.0 --port 8000

🐳 Docker Deployment¶

# Start Streamlit UI, Redis Broker, and Celery Background Worker
docker compose up -d

# View real-time logs
docker compose logs -f

🧪 Testing & Automated Quality Gates¶

# Run complete unit and integration test suite (127 tests)
pytest tests/ -v

# Run RAG Evaluation Benchmark Suite (Recall@K, MRR & Faithfulness)
pytest tests/eval/ -m eval -v

# Run Ruff linter and code formatter checks
ruff check src/ tests/
ruff format --check src/ tests/

📚 Technical Documentation & Resources¶

• 🌐 Online Documentation Site (MkDocs Material): Full interactive documentation with instant search, dark mode, high-res C4 diagrams, and ADR catalog hosted on GitHub Pages.
• 🏛️ System Architecture & Technical Specifications: Deep dive on dataflow sequences, mathematical formulations, state transitions, and distributed scaling.
• 📐 Architecture Decision Records (ADRs 001–019): Complete catalog of formal design decisions, evaluated alternatives, and trade-offs.
• 🗺️ Implementation Roadmap: Phased milestones (Completed Phases 1–19 and Future Milestones Phases 20–22).
• 🛡️ Engineering Conventions: Architectural patterns, Pydantic V2 schemas, error boundaries, and type safety rules.
• 🤝 Contributing Guidelines: Local developer setup, branching conventions, and quality gates.
• 📝 Changelog: Semantic versioning release history.
• 📄 License: Apache License 2.0.

⚖️ License¶

This project is licensed under the Apache License 2.0 — see the LICENSE file for details.