OnCrawl relies on data to prove SEO is no dark magic. SEO is pure data science and our customers rely on it to make predictable decisions in terms of SEO campaigns.

As Data Engineer, you will join our gang of data nerds to help us provide our customers with insightful, SEO-minded, analysis jobs. Our jobs are designed to scale at the billion web pages scale and massive amounts of data are processed daily to compute very high level metrics such as near duplicate pages, similarity heat maps, flow diagrams of InRank juice, and many more. We run our data platform using Spark, but other solutions can be used depending on the purpose.

Your missions:

• Write state-of-the art data processing jobs crunching millions of web pages and millions of web-server log events per hour
• Work on structured and unstructured data
• Get familiar with Spark, Elasticsearch and the data warehouse system to achieve stability and scalability
• Create new ways to explore and visualize reports in real time

• Java
• Spark or Hadoop MR or Apache Hive
• Experience writing scalable jobs running 24/7, or willing to learn
• Elasticsearch, MongoDB
• Python
• Git, Continuous integration