Job Title / Designation: AI Engineer (LLM Infrastructure & Agentic AI)
Functional Area: Enterprise Software / AI Infrastructure
Industry: IT Software
Location of Job: Nairobi, Kenya
Employment Type: Full-Time
Salary Offered: Competitive / Negotiable
Qualification: Bachelor’s Degree in Computer Science, Software Engineering, IT, or related technical field

Role Overview

We are looking for a technically strong AI Engineer to join our infrastructure and AI team in Nairobi. This role sits at the intersection of GPU systems engineering, LLM serving, retrieval-augmented generation (RAG), and agentic AI, running on our self-managed Kubernetes GPU cluster.

In this role, you will deploy, optimize, and maintain open-source LLM inference engines on NVIDIA GPU hardware, build and tune RAG pipelines, and work with agentic AI runtimes including Hermes and OpenClaw within our Kubernetes cluster. Everything we build is open-source, self-hosted, and production-grade. You will own end-to-end components from model deployment to agentic pipeline integration.

General Responsibilities / Duties

LLM Hosting & Serving

  • Deploy and maintain Ollama (llama.cpp backend) as the primary inference engine on Sybyl GPU servers.

  • Manage GGUF-format model quantization (Q4_K_M, Q8_0) for efficient inference, optimizing for throughput, latency, and VRAM efficiency.

  • Tune inference parameters: context length, GPU memory fraction, and batch size for throughput and latency targets.

  • Monitor GPU utilization, VRAM allocation, and inference performance; resolve bottlenecks.

  • Evaluate alternative open-source engines (vLLM, llama.cpp standalone, LMDeploy, TGI) for future hardware upgrades.

  • Manage model registry and version control for hosted models across Qwen, Mistral, and LLaMA families.

Agentic AI Systems

  • Deploy and maintain Hermes (Nous Research) on CPU worker nodes, connected to Ollama via OpenAI-compatible API.

  • Deploy and integrate OpenClaw agentic framework within the stack.

  • Build multi-agent pipelines: tool-use, function calling, agent chaining, and multi-step reasoning.

  • Debug agentic runtime behavior and tune prompting strategies for reliability and latency.

RAG Pipeline Engineering

  • Design, implement, and maintain production RAG pipelines using LangChain and LlamaIndex.

  • Manage and tune vector stores (ChromaDB, FAISS) for document retrieval at scale.

  • Integrate web search via SearXNG into retrieval pipelines.

  • Implement chunking strategies, embedding model selection, hybrid search (dense + BM25), and re-ranking.

  • Evaluate and improve retrieval accuracy, latency, and hallucination rates across pipelines.

Kubernetes & GPU Infrastructure

  • Manage GPU workloads on Kubernetes: node taints/tolerations, GPU resource requests, and namespace management.

  • Work with NVIDIA GPU Operator, CUDA 12.x drivers, and GPU monitoring tooling.

  • Author and maintain Kubernetes manifests and Helm charts for all AI services.

  • Contribute to cluster reliability: Longhorn storage, Traefik + MetalLB ingress, and networking.

  • Maintain Ubuntu 22.04 LTS and NVIDIA driver 550/560 environments on GPU nodes.

Open-Source Stack Stewardship

  • Maintain LibreChat as the production AI front-end and SearXNG for search augmentation.

  • Evaluate new open-source tools and models; produce written integration recommendations.

  • Contribute upstream to open-source projects where appropriate.

Required Skills & Experience

Core Requirements:

  • Bachelor’s degree in Computer Science, Software Engineering, IT, or equivalent practical experience.

  • Strong Python programming skills with clean, production-ready code.

  • Practical experience deploying and serving open-source LLMs with Ollama and llama.cpp.

  • Solid understanding of GGUF, GPTQ, and AWQ quantization formats and trade-offs.

  • Hands-on RAG pipeline experience using LangChain or LlamaIndex.

  • Working knowledge of vector databases (ChromaDB, FAISS, or similar).

  • Kubernetes fundamentals: kubectl, namespaces, deployments, resource limits, and node taints.

  • Solid background in Docker and container-based deployment workflows.

  • Linux systems administration: Ubuntu, systemd, shell scripting, and GPU driver management.

  • NVIDIA CUDA basics: GPU memory management, driver compatibility, and CUDA version constraints.

  • Clear written and verbal technical communication skills.

Add-on and Preferred:

  • Experience with Hermes, OpenClaw, LangGraph, CrewAI, AutoGen, or similar agentic frameworks.

  • Helm chart authoring and Kubernetes-native deployments.

  • Experience with LiteLLM or similar LLM proxy/gateway solutions.

  • Experience with SearXNG, web scraping, or search-augmented RAG.

  • Experience with Traefik ingress, MetalLB, or Longhorn storage.

  • Experience deploying and configuring LibreChat or similar open-source front-ends.

  • Knowledge of multi-GPU tensor parallelism and batch inference.

  • Contributions to open-source AI/ML repositories on GitHub.

  • Exposure to East African enterprise data contexts (fintech, legal, or enterprise SaaS).

Architecture & Tech Stack Notes

Sybyl operates an open-source-first engineering stack. The team does not build on or depend on proprietary commercial model APIs (OpenAI, Anthropic, Google Gemini), cloud-managed ML services (AWS SageMaker, Azure ML, Vertex AI), or proprietary managed cloud vector databases. Candidates must have hands-on experience working directly with local open-weight models and self-hosted infrastructure.

    Upload Your CV: