VP of Engineering

Key Responsibilities:

• Define and execute the engineering roadmap with a strong emphasis on Databricks capabilities, LLMs, and Vector Search technologies.

• Architect scalable and high-performance data pipelines using Databricks, Delta Lake, and Unity Catalog, ensuring optimal data governance and analytics.

• Lead the integration of LLMs and vector databases to enhance entity resolution, semantic search, and intelligent data enrichment.

• Drive collaboration with Databricks' tools and features to implement real-time data processing workflows and AI/ML solutions.

• Oversee the design and development of AI-powered MDM solutions, ensuring compliance with business rules such as match and merge, survivorship, and anomaly detection.

• Build and mentor a high-performing engineering team, fostering a culture of innovation and collaboration.

• Collaborate with internal stakeholders, clients, and partners to deliver scalable solutions that align with business goals.

  
  

Qualifications:

• Educational Background:

  • Master’s or Ph.D. in Computer Science, Artificial Intelligence, Data Science, or related fields.

• Experience:

  • 12+ years of engineering leadership experience in AI/ML or data platform-driven organizations.

  • Proven expertise in deploying LLMs (e.g., GPT, BERT) and vector search technologies (e.g., Pinecone, Milvus, Weaviate).

  • Hands-on experience with Databricks workflows, including Delta Lake, Unity Catalog, and Machine Learning pipelines. Candidates with direct exposure to working in or with Databricks teams are highly preferred.

  • Strong background in MDM processes and data engineering best practices.

  • Demonstrated success in building scalable, cloud-native architectures on AWS or Azure.

• Technical Skills:

  • Advanced knowledge of AI/ML frameworks such as PyTorch, TensorFlow, and Hugging Face.

  • Proficiency in Databricks features, including Auto Loader, SQL Analytics, and real-time processing pipelines.

  • Expertise in vector database solutions and tools for building LLM-based applications.

  • Familiarity with compliance standards like HIPAA and GDPR.

• Soft Skills:

  • Strategic thinker with the ability to align engineering initiatives with broader business objectives.

  • Strong leadership and communication skills, capable of engaging with clients, partners, and stakeholders.

    
    

Preferred Qualifications:

• Direct experience working within Databricks or significant collaboration with their ecosystem.

• Industry expertise in healthcare or life sciences, retail etc.

• Familiarity with Databricks Machine Learning and tools for scaling AI/ML models.