Dataiku is The Universal AI Platform™, giving organizations control over their AI talent, processes, and technologies to unleash the creation of analytics, models, and agents. Providing no-, low-, and full-code capabilities, Dataiku meets teams where they are today, allowing them to begin building with AI using their existing skills and knowledge.

Internship goal

Identify and implement an industrial use case for converting an agentic system that uses a Large Language Model (LLM) into one that uses a Small Language Model (SLM), leveraging Dataiku's platform to create a real-world example for our customers.

Detailed description

Agents are being increasingly experimented with and integrated into critical business processes. As their use becomes more widespread, there is a growing demand for improved efficiency, both in terms of performance and cost. Additionally, data security is a primary concern. Companies are looking to host their own Large Language Models (LLMs) rather than depend on third parties to ensure their sensitive information remains secure.

While state-of-the-art LLMs simplify the development of agents with their strong reasoning and interpolation skills, creating reliable agents with smaller LLMs (SLMs) is a more complex challenge. It often requires advanced techniques like fine-tuning or meticulous prompt optimization to achieve consistent results. However, this effort is worthwhile. Recent research has shown how to reliably convert agentic systems that use LLMs into systems that use SLMs, which is the exact application we want to develop at Dataiku.

Dataiku offers a comprehensive platform for building, evaluating, and fine-tuning agents. The main goal of this internship is to identify a practical, industrial use case where converting to an SLM-based agent makes sense. You will then implement this case, creating a tangible example that our customers can use for inspiration.

During this internship, you will:

• Get familiar with Dataiku, its Agent and LLM mesh infrastructure.
• Research state-of-the-art techniques for converting LLMs agentic systems into SLMs ones.
• Experiment on some industrial use-cases how algorithms perform and evaluate their efficiency.
• Collaborate with the Data Science and the broader Solutions team to identify technical challenges and industrial context.
• Develop a solution or demo that leverages this technique on an example that resonates with the industry.
• Contribute to increasing Dataiku’s credibility as the platform of choice for their Agentic AI use-cases.

Stack

• Python #LI-Onsite #LI-FR1

What are you waiting for!

At Dataiku, you'll be part of a journey to shape the ever-evolving world of AI. We're not just building a product; we're crafting the future of AI. If you're ready to make a significant impact in a company that values innovation, collaboration, and your personal growth, we can't wait to welcome you to Dataiku! And if you’d like to learn even more about working here, you can visit our Dataiku LinkedIn page.

Our practices are rooted in the idea that everyone should be treated with dignity, decency and fairness. Dataiku also believes that a diverse identity is a source of strength and allows us to optimize across the many dimensions that are needed for our success. Therefore, we are proud to be an equal opportunity employer. All employment practices are based on business needs, without regard to race, ethnicity, gender identity or expression, sexual orientation, religion, age, neurodiversity, disability status, citizenship, veteran status or any other aspect which makes an individual unique or protected by laws and regulations in the locations where we operate. This applies to all policies and procedures related to recruitment and hiring, compensation, benefits, performance, promotion and termination and all other conditions and terms of employment. If you need assistance or an accommodation, please contact us at: reasonable-accommodations@dataiku.com

Protect yourself from fraudulent recruitment activity

Dataiku will never ask you for payment of any type during the interview or hiring process. Other than our video-conference application, Zoom, we will also never ask you to make purchases or download third-party applications during the process. If you experience something out of the ordinary or suspect fraudulent activity, please review our page on identifying and reporting fraudulent activity here.