Senior AI Quality Assurance (QA) Engineer

At TWG Group Holdings, LLC (‘TWG Global’), we are at the cutting edge of innovation and business transformation, leveraging data and AI to manage a diverse portfolio that spans Investments, Securities, Insurance, Finance, Corporate Lending, Merchant Banking, as well as Sports, Media, and Entertainment. Data is one of our most valuable assets, and our commitment to AI allows us to transform that data into a powerful competitive advantage. We are pioneering a range of data and AI services that provide real-time intelligence to both our customers and workforce, enabling them to make informed decisions within interactive, cloud-native business applications.  

As we evolve into an AI-first, model-driven, and cloud-native company, we prioritize responsible data and AI practices that protect our customers and workforce while ensuring compliance within a regulated environment. Our goal is to lead by example, acting with the highest standards of ethics, responsibility, and transparency. This commitment drives our development of cutting-edge AI/ML business applications across our diverse product and service offerings.  

In this role, you will be at the heart of our data and analytics transformation, collaborating closely with management teams to achieve our strategic objectives and operational excellence. Our decentralized structure empowers each business unit to operate autonomously while benefiting from the strategic guidance and support of our central AI Solutions Group.  

Through our pioneering partnerships with major data and AI vendors, you will have the opportunity to work on transformative projects that will revolutionize marketing, distribution, customer acquisition, and internal processes. These initiatives are designed to enhance productivity, reduce costs, and create high-demand products and services. The advanced data systems we develop will grant management unparalleled access to real-time performance data, enabling swift, informed decision-making and agile business adjustments.  

Moreover, you will play a crucial role in leveraging our strategic relationships and equity positions in leading tech startups. These collaborations are not just about staying ahead of the competition—they are about creating true competitive advantages across our businesses. You will contribute to projects that are reshaping industries, from harnessing cloud-based supercomputing capabilities to driving key initiatives with major universities that are shaping the future of technology.  

At TWG Global, your work will directly contribute to our ambitious goal of generating sustained growth and superior returns, with expected annual equity returns exceeding 20%, including growing dividends. Join us as we push the boundaries of technology and innovation, delivering significant value across our portfolio while making a profound impact on the industries we serve.  

TWG Global is seeking a dedicated Senior AI Quality Assurance (QA) Engineer to join our team focused on commercial-grade AI products. This role is critical in ensuring the reliability, accuracy, and performance of our AI solutions through systematic testing and human-in-the-loop (HIL) processes.

The ideal candidate will have hands-on experience with AI products, model evaluation, and test automation. 

What you’ll do: 

• Design, develop, and execute comprehensive test plans for complex AI use cases and scenarios. 
• Validate the accuracy of the AI agents and application, ensuring consistent and high-quality outputs. 
• Conduct release, system, and performance testing to maintain commercial-grade standards across product versions. 
• Perform in-house model evaluation, including preparation and review of training and evaluation data. 
• Develop and maintain a regression test case library for ongoing use case validation. 
• Collaborate with data scientists and engineers to optimize testing processes and resource allocation. 
• Design and implement automated test cases, integrating HIL where automation is insufficient. 
• Benchmark AI agent performance against competitors and industry standards. 

• 2–5 years of experience in AI product testing, QA, or related fields. 
• Proven experience with AI products, prompt engineering, and LLM model evaluation. 
• Strong skills in data manipulation and test data preparation. 
• Proficiency in designing and executing automated tests. 
• Excellent analytical, problem-solving, and communication skills. 
• Bachelor’s degree in Computer Science, Engineering, or a related field (preferred). 

Compensation:  

The expected annual salary range for this role is $190,000. A discretionary bonus may be provided as part of the compensation package. 

Position Location:

This position will work out of the TWG office on a hybrid schedule in Santa Monica, CA. Consideration for a different working location will be considered on a case-by-case basis.

TWG is an equal opportunity employer. All applicants will be considered for employment without attention to age, race, color, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status.