Sr Staff Data EngineerApply Now Apply Later Job ID 524879BR Location Burbank, California, United States Business Studio Entertainment Brand The Walt Disney Studios Date posted 02/07/2018
The Senior Staff Data Engineer works in a small multi-disciplined team creating next generation video consumption platforms. We expect you to be up to date on the happenings in the data community, passionate about what you do and connected to the open source community. You will participate in overall system design, come up with webtailored solutions that emphasize reuse and good design patternsApply Now Apply Later
- Build and optimize performance of Hadoop/Spark batch jobs.
- Build and optimize performance of Spark, Kafka, Cassandra, ELK, and whatever else makes sense for realtime pipelines.
- Design and architect high quality data-lake, data-warehouse, and data-marts data models.
- Enable and implement Data Science workflows and advanced machine learning althorithms.
- Build and optimize performance of ElasticSearch cluster and relevance
- Build data pipelines orchestration.
- Become and stay an expert in current and emerging technologies and tools
- Contribute to Open Source solutions and communities wherever you can
- Collaborate with other software engineers and crossfunctional teams
- Evangelize technologies, solutions, and best practices
- Contribute new ideas to a larger community of highcaliber professionals
- Balance resources, requirements, and complexity
- Very passionate about coding (If you have a Github profile, that’s awesome! Would love to check it out! Otherwise we will ask you to code before or during the interview).
- String understanding of distributed systems and distributed computation.
- Strong working knowledge in at least 2 of: Scala, Java, Python, or Go-Lang
- Demonstrated working knowledge with data Apache Hadoop / Spark ecosystem like Spark, Hive, Presto, Oozie, Pig, Hue, Zeppelin
- Demonstrated working knowledge of data modeling
- Unit, Integration, and Load testing
- Developing REST APIs.
- Ant, Maven, SBT, and/or Gradle
- Docker containers building and deployment.
- Excellent communication and collaboration skills
- GraphQL knowledge
- Kubernetes knowledge
- Apache Spark MLlib
- Apache Spark GraphX
- Amazon AWS or other cloud Services