|
Описание: |
In Competera, we are building a place where optimal pricing decisions can be made easily. We believe that AI technologies will soon drive all challenging decisions and are capable of helping humans be better.
We are now looking for a Senior Data Engineer to improve our data processing pipelines from performance, cost and correctness standpoints.
You could be a perfect match for the position if
You want to:
* Migrate BigQuery workloads to Spark 3.5 on Databricks.
Re-engineer our daily-TB batch jobs into Delta Lake pipelines that run faster and cost less.
* Turn full reloads into true incremental processing.
Build CDC / MERGE logic so we scan only the data that changed and deliver fresh features within minutes.
* Add quality gates & observability from day one.
Instrument every stage with custom metrics, data-drift alerts and cost reports the product team can read.
* Set up monitoring & slot-second cost dashboards.
Expose processing-time, SLA and $-per-feature charts so we can make data-driven trade-offs.
* Pair with Data Scientists and Product Managers.
Work side-by-side with Data Scientists and Product Managers from idea to release, instead of simply passing datasets back and forth.
* Continuously tune for scale.
Dozens of terabytes move through the platform daily; you’ll experiment with partitioning, Z-ORDER, and Photon to keep latency low as volume grows.
You have:
* 5+ years of experience in a data engineer role.
* Strong knowledge of SQL, PySpark, Python, Airflow.
* Experience with integration platforms (Apache NiFi / Fivetran / Airbyte).
* Ukrainian — fluent, English — intermediate +.
Pleasant extras:
* Databricks, GCP, BigQuery, Kafka, data modeling patterns, data quality approaches and tools.
Soft skills:
* Product mindset.
* Ability to work in a fast-paced environment.
* Willingness to take ownership of a feature and guide it through all stages of the development lifecycle.
* Proactivity, openness, and a desire to dive deep into the domain and learn new approaches and tools.
You’re gonna love it, and here’s why:
* Rich innovative software stack, freedom to choose the best suitable technologies.
* Remote-first ideology: freedom to operate from the home office or any suitable coworking.
* Flexible working hours (we start from 8 to 11 am) and no time tracking systems on.
* Regular performance and compensation reviews.
* Recurrent 1-1s and measurable OKRs.
* In-depth onboarding with a clear success track.
* Competera covers 70% of your training/course fee.
* 20 vacation days, 15 days off, and up to one week of paid Christmas holidays.
* 20 business days of sick leave.
* Partial medical insurance coverage.
Drive innovations with us. Be a Competerian.
Відгукнутись на вакансію |
