Lead Gen-AI Engineer

Required Skills
● Solution Prototyping & Testing: Rapidly test various models and approaches to determine the best fit for the project without extensive over-investment. Stay updated on LLM advancements (e.g., GPT-3.5 to GPT-4).
● Prompt Engineering: Write, test, and refine prompts, including creating effective prompt chains and managing prompt versions for optimal results.
● Model & Prompt Evaluation: Analyze model and prompt performance to ensure reliability. Develop robust testing methodologies, incorporating training and test sets for thorough evaluation.
● Performance Monitoring: Build pipelines to monitor model responses and track performance metrics in production for ongoing optimization.
● RAG Architecture Implementation: Design and maintain retrieval-augmented generation (RAG) architectures. Work with data pipelines, vector databases, and search algorithms. Manage document chunking and embedding selection for optimal context window usage.
● Data Extraction: Extract data from various sources (e.g., PDF files) as part of the RAG process.
● Software Development Best Practices: Employ CI/CD, test-driven development, and sound architectural principles in all GenAI solutions.
● Evaluation Scripts & Load Testing: Develop tests for the random nature of LLM outputs and conduct load testing to ensure system robustness, even when reliant on third-party APIs.
● ETL & Traditional Data Management: Build data pipelines to manage and integrate traditional relational data where necessary.