Staff Data Engineer

Job summary
Permanent contract
Paris
Salary: Not specified
Skills & expertise
Cybersecurity knowledge
Motivational skills
Interviewing
AWS Batch
GitHub
+15
Apply

GitGuardian
GitGuardian

Interested in this job?

Apply
Questions and answers about the job

The position

Job description

About GitGuardian

GitGuardian is a global cybersecurity scale-up. The company is based in Paris, New York City, Boston.
Our early investors include Scott Chacon, co-founder of GitHub, and Solomon Hykes, co-founder of Docker, alongside leading US and European VC firms. We raised our Series C in early 2026.
GitGuardian leads the way in securing the credential layer, helping organizations protect the secrets that let code, machines, and AI agents access systems with broad detection, identity context, and remediation at scale. Already trusted by 600K+ developers worldwide and by 50 of the Fortune 500!

About your team and your mission

You will join the Public Intelligence squad, which builds the engine behind GitGuardian's mission to protect companies from secrets leaked in public. We collect public data at scale, from every public commit pushed to GitHub to public Docker Hub images, and find the secrets in it. AI agents then work out which company each leak belongs to and how critical it is. Customers see these leaks in their GitGuardian workspace, and the most critical ones help our sales team start conversations with new prospects.

We ingest about 12.5 million public GitHub commits a day and store nearly 400 million secret occurrences in ClickHouse. The systems run in production, and AI agents already analyze every public secret we find. They grew fast, one project at a time, and we now have a real job ahead of us: rethink how we collect, store and process public data, so the platform is faster, cheaper, more reliable and ready for new sources.

As the squad's Staff engineer, you will own this architecture end to end, from the collectors that read public sources to the real-time processing that feeds our agents and our product. The role sits between software and data. Our hardest problems are as much about collecting and processing a huge, rate-limited stream reliably as about how the data is stored and modeled.

You will report to Alexis, Senior Engineering Manager, and work with a squad of 5 engineers (software, data and AI) and a Product Manager.

Your impact will go beyond the squad. Public data flows through systems owned by other squads, and the architecture you design will shape their work too:

  • Secrets Detection builds the engine that finds secrets in the data we collect, and the analyzers our agents use to assess the impact of a leak.

  • Incidents builds the incident experience of the GitGuardian platform. Public incidents are a shared object between our squads: we qualify them and compute their risk score, and customers handle them in the same product as every other incident.

  • SRE runs the infrastructure our collectors and pipelines rely on.

  • Data & AI runs the company's data platform, which our pipelines share today.

Key challenges
  • Design the next generation of our collectors. The GitHub public feed is huge, rate-limited and noisy. Our collectors must keep up with it, measure how much of it they cover, absorb traffic spikes, and recover from an incident without overloading the rest of the pipeline. The same design then has to work for every new source.

  • Move public data processing to real time. Part of our processing still waits for daily batches. You will design the event-driven processing that takes a commit from GitHub to a qualified incident in minutes (scanning, attribution, agent analysis), with clear latency targets and back-pressure when volume spikes.

  • Build a single source of truth for public data. Move public data and its transformations out of Snowflake into an S3 + Iceberg data lake that every GitGuardian service can rely on.

  • Simplify a multi-service architecture. The platform grew one project at a time: collectors, queues and workers, a Django product backend, Dagster pipelines, ClickHouse and Snowflake. You will define the target architecture and lead the migration step by step, without stopping production or losing data.

  • Make the platform operable. Define what healthy means for each part of the pipeline (coverage, latency, data quality), and build the monitoring and alerting that go with it.

Your responsibilities

  • Own the end-to-end architecture of the public data platform: collection, real-time processing, storage, and delivery to our agents and product.

  • Stay hands-on: write production code on the most critical parts, starting with the collectors, the real-time processing and the migrations.

  • Turn the squad's priorities into a technical strategy and roadmap, together with the Engineering Manager and the Product Manager.

  • Lead design workshops, and make decisions the squad can build on.

  • Drive the architecture decisions that span several squads, and get them aligned.

  • Mentor the squad's engineers and raise the technical bar through design reviews and knowledge sharing.

  • Lead the squad's software factory. AI agents write a growing share of our code, and you will define how we build with them: the skills, tools and conventions that make them reliable, so the whole squad ships faster without lowering the bar.

Technical environment

  • Languages: Python, SQL

  • Backend and processing: Django, Celery, RabbitMQ, Redis

  • Orchestration: Dagster, AWS Batch

  • Storage and analytics: S3, Iceberg, ClickHouse, Snowflake, PostgreSQL

  • Ingestion: Vector

  • Deployment: Docker, Kubernetes, Terraform, ArgoCD, GitLab CI

  • Cloud: AWS, OVH

  • AI: LangGraph, LiteLLM, Claude Code

  • Observability: Coralogix, Sentry, OpenTelemetry

About you

If you think you match at least 70% of these criteria, please apply!

Here's what we consider essential for success in this role:

  • 7+ years building and operating large-scale, data-intensive systems in production, as a data engineer or a backend engineer.

  • Strong software engineering: you have designed and run high-throughput collection or ingestion services, and you know their hard parts well: queues and workers, back-pressure, retries and idempotency, deduplication, rate-limited external APIs.

  • Production experience with real-time, event-driven processing (RabbitMQ, Kafka or similar) and the latency targets that come with it.

  • Strong data architecture: data lakes on object storage, analytical databases at scale (ClickHouse or similar), and data modeling for pipelines.

  • Python expert.

  • Proven experience redesigning a production architecture step by step, migrating without downtime or data loss.

  • An operations mindset: you design observability and alerting together with the system, and you know how to run an incident.

  • Technical leadership: you align several engineers on a target architecture, mentor them, and explain your decisions in writing.

  • Deliberate use of AI coding agents: you give them context and a spec, read what they produce, and catch what they get wrong.

  • Fluent English in an international environment.

The following skills would strengthen your application but aren't required:

  • Apache Iceberg or other open table formats.

  • Large-scale crawling.

  • AWS, Kubernetes and Terraform in production.

  • LLM agents in production pipelines.

  • An interest in cybersecurity and in how secrets leak.

  • Experience in high-growth startups or scale-ups.

The interview process

1. Video call with a recruiter

To discover your professional project and evaluate if there could be a mutual match

2. Interview with your future manager (1h)

Purpose: To know more about you and your achievements, and to present the squad.

Skills Assessed: Ownership, communication, motivation, and your experience as a technical leader.

3. Technical interview with the team (2h)

Purpose: System design discussions on collection and real-time processing, and a hands-on exercise on public GitHub data.

Skills Assessed: Software and data architecture, distributed systems, data modeling, and how you reason about trade-offs.

4.1 Final interviews with Pierre, VP of Engineering, and Eric, Co-founder and CEO

Purpose: To know more about your ambitions, and to share our company's vision for the next couple of years.

4.2 References check

You can start thinking about two contacts who can attest to your previous or current professional experiences. These contacts should be as recent as possible, and we will call them at the end of the process.

Want to know more?

These job openings might interest you!

These companies are also recruiting for the position of “Data / Business Intelligence”.

  • Mistral Ai

    AI Scientist - Physics Models

    Mistral Ai
    Mistral Ai
    Permanent contract
    Paris
    No remote work
    Artificial Intelligence / Machine Learning, IT / Digital
    280 employees

  • ADVANCED SCHEMA

    Consultant Data & IA

    ADVANCED SCHEMA
    ADVANCED SCHEMA
    Permanent contract
    Paris
    Occasional remote
    IT / Digital, SaaS / Cloud Services
    240 employees

  • Scaleway

    Datacenter IT Manager

    Scaleway
    Scaleway
    Permanent contract
    Paris
    A few days at home
    IT / Digital, SaaS / Cloud Services
    600 employees

  • Swan

    Senior Data Analyst

    Swan
    Swan
    Permanent contract
    Paris
    A few days at home
    Salary: €65K to 72K
    Banking, IT / Digital
    350 employees

  • Skapa

    Lead Data & AI Engineer

    Skapa
    Skapa
    Permanent contract
    Paris
    A few days at home
    Design, IT / Digital
    12 employees

  • Thélio

    Consultant expérimenté Data & AI Governance (F/H)

    Thélio
    Thélio
    Permanent contract
    Paris
    Occasional remote
    Salary: €45K to 65K
    IT / Digital, Strategy
    100 employees

See all job openings
Apply