Job Description - Data Engineer (19WD35491)
Data Engineer19WD35491Americas-United States of America-California-San Rafael
Location: San Rafael, CA- United States
Job ID: 19WD35491
Position OverviewAutodesk is seeking a Data Engineer with a minimum of 2-3 years’ experience building data pipelines to join our Enterprise Data Engineering & Visualization (EDEV) team. The EDEV Team is chartered with building robust and resilient pipelines that create the master data sets that serve as the building blocks for all data scientists and data analysts throughout the organization. In addition, this team establishes and promotes the data engineering standards and best practices across Autodesk and help to lead the effort to make data an integral part of decision making at Autodesk.
The ideal candidate will be responsible for designing, developing, implementing and documenting data solutions for complex ETL data pipelines, and lending technical expertise within the group. The ideal applicant will be able to thrive in a highly collaborative workplace and actively engage in the development process.
- Map and automate data flow from a variety sources including desktop, web, and mobile product feeds and business systems
- Develop workflows to run Apache Spark applications using Oozie, AWS Lambda and AWS Glue
- Deploy solutions via automated CI/CD processes to execute data pipelines running in batch
- Work with programming languages used for data manipulation, including: Scala, PySpark, Spark SQL, R, Java
- Design, develop, execute and document software solutions to address complex data collection, processing, transformation and reporting issues
- Collaborate with peer organizations, dev ops, support organizations on technical issues and provide guidance
- Interpret and translate business needs to technical requirements
- Work with team to troubleshoot code level problems quickly and efficient on as need basis
- Develop, refine and educate the data community on coding standards and best practices
- Participate in code and document reviews
- Bachelor’s Degree or relevant experience in the field of Computer Science, Mathematics or Statistics
- 2 to 3 years of experience in big data technologies including Spark, Hadoop, RedShift, Vertica, Snowflake, Hive, Lambda, Glue, S3
- Proficiency in executing analysis using SQL, Hive, R, Spark Dataframes, Scala, and/or Python
- Strong SQL skills
- Fluency with one or more of the scripting languages: Python, Java, Scala, etc.
- Working knowledge and experience with basic and complex data structures
- Good communication skills and ability to explain complex topics to a non-technical audience
- Knowledge and experience with large data sets, event streams and distributed computing (Hive/Hadoop, Spark etc.)
- Experience working with Agile Scrum Teams
- Experience in Version control management - tracking and coordinating engineering work
- Proven experience of finding bugs and vulnerabilities with automated tools
- Expertise in data research/analysis with a focus on data quality and consistency
- Experience with algorithms, distributed storage & compute to solve complex business problems
- An advanced school degree
About AutodeskWith Autodesk software, you have the power to Make Anything. The future of making is here, bringing with it radical changes in the way things are designed, made, and used. It's disrupting every industry: architecture, engineering, and construction; manufacturing; and media and entertainment. With the right knowledge and tools, this disruption is your opportunity. Our software is used by everyone - from design professionals, engineers and architects to digital scientists, students and hobbyists. We constantly explore new ways to integrate all dimensions of diversity across our employees, customers, partners, and communities. Our ultimate goal is to expand opportunities for anyone to imagine, design, and make a better world.
At Autodesk, we're building a diverse workplace and an inclusive culture to give more people the chance to imagine, design, and make a better world. Autodesk is proud to be an equal opportunity employer and considers all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender, gender identity, national origin, disability, veteran status or any other legally protected characteristic. We also consider for employment all qualified applicants regardless of criminal histories, consistent with applicable law.
To all recruitment agencies: Autodesk does not accept unsolicited headhunter and agency resumes. Autodesk will not pay fees to any third-party agency or company that does not have a signed agreement with Autodesk, Inc.