Develop and support AI-driven applications focused on knowledge base ingestion, including extracting, processing, and storing structured and unstructured data (e.g., Excel files, PDFs, images, and database records).
Build and maintain data pipelines for machine learning models, ensuring data quality, consistency, and efficient processing.
Implement document parsing, text preprocessing, and chunking strategies to prepare data for retrieval-augmented generation (RAG) systems.
Design and support workflows for retrieving relevant data and injecting context into prompts for AI applications.
Assist in developing and integrating backend services using Python frameworks such as Django.
Collaborate with cross-functional teams to test, deploy, and improve AI models and pipelines.
Support containerized deployments and environments using Docker and Kubernetes.
Contribute to continuous improvement of AI systems, including reducing hallucinations and improving response accuracy.
Qualifications
Minimum Qualifications:
Bachelor's degree in Computer Science, Information Technology, Data Science, or any related field.
years of experience in AI/ML, data engineering, or backend development with exposure to production-level AI systems.
Strong understanding of AI/ML concepts, including embeddings, vector databases, and RAG architecture design.
Proficient in Python and relevant libraries such as pandas, numpy, openpyxl, requests, and PyPDF.
Hands-on experience in building and optimizing data pipelines for structured and unstructured data.
Strong experience in data preprocessing, document parsing, chunking strategies, and normalization techniques.
Experience working with vector databases and similarity search mechanisms.
Familiar with Docker and Kubernetes for containerized deployment and scaling.
Experience with Dify platform or similar AI workflow/orchestration tools is an advantage.
Strong understanding of prompt engineering, context injection, and techniques to minimize hallucinations in AI systems.
Experience with API development and system integration using frameworks such as Django or FastAPI.
Familiar with CI/CD pipelines, version control (Git), and Agile methodologies.
Strong problem-solving, analytical, and communication skills, with the ability to work across cross-functional teams.
