Role Summary: You'll develop hybrid search systems integrating lexical and semantic search with Reciprocal Rank Fusion (RRF), implement knowledge graph systems using Neo4J and Graphiti, and build feature engineering pipelines for social media data. You'll gain exposure to GraphRAG and modern search evaluation methodologies.
Key Responsibilities
Hybrid Search & Semantic Systems
- Develop advanced search algorithms combining lexical and semantic methodologies
- Implement RRF techniques and vector-based semantic search using modern embedding techniques
- Design scalable search architectures for complex query processing
Knowledge Graph & GraphRAG
- Build and maintain knowledge graph systems using Neo4J and Graphiti
- Implement Graph-based RAG for enhanced search intelligence
- Design graph schemas and develop graph traversal algorithms
Feature Engineering & Data Processing
- Design feature engineering pipelines for social media data
- Develop topic clustering algorithms for content categorization
- Implement real-time feature generation for dynamic content
Search Infrastructure
- Build advanced indexer systems using Elasticsearch
- Design and optimize search index schemas for diverse content types
- Develop comprehensive evaluation frameworks and A/B testing infrastructure
Microservices & APIs
- Design scalable microservices using saga pattern for distributed search
- Develop FastAPI-based high-performance search APIs
- Implement event-driven architectures for real-time updates
Software Best Practices
- Write clean, well-documented Python code following OOP principles
- Implement comprehensive testing strategies
- Participate in code reviews and agile practices
Minimum Qualifications
Technical Skills:
- Programming: Proficiency in Python, ideally with some exposure to AI/ML libraries and search-related tools.
- Information Retrieval: Familiarity with search algorithms, ranking strategies, and how relevance is measured.
- Machine Learning & Semantic Search: Experience or working knowledge of vector databases, embeddings, and semantic search is a plus, but not strictly required.
- Graph Databases: Familiarity with graph database concepts (e.g., Neo4j) is beneficial.
- Search Engines: Some hands-on experience with Elasticsearch or similar platforms is preferred.
- APIs: Comfortable working with modern APIs; experience using FastAPI is an advantage.
Technology Stack You'll Work With
- Backend Framework: FastAPI
- Databases: MongoDB, PostgreSQL, Neo4J, Redis
- Search Engine: Elasticsearch
- Knowledge Graph: Graphiti
- Analytics: Metabase
- Orchestration: Apache Airflow
- AI/ML Tools: Vector databases, embedding models, semantic search frameworks
