Big Data Technical Lead
Cars.com is seeking an experienced Hadoop/Big Data Technical Lead to join our organization in providing leadership for the overall, design, development and deployment of a Hadoop based platform. Developing conceptual, logical and physical target-state architecture, engineering and operational specs for Big Data solutions across all lines of businesses by effectively applying software engineering principles, patterns and best practices. The candidate will collaborate with business users and application teams across the Cars.com organization to build consensus and to promote adoption of Big Data solutions effectively. Additionally, provides technology insight and expertise on industry trends
The ideal candidate will be able to:
- Utilize and mentor developers in Big Data Technologies such as Hadoop and Spark, Kafka and In-memory applications using Java, SQL, Pig, Hive, Impala, Scala, based on the concepts of Lamda architecture
- Function as a hands-on team Technical Leader that provides day to day direction and solutions to achieving set goals.
- Analyze multiple sources of structured and unstructured data to propose and design data architecture solutions for scalability, high availability, fault tolerance, and elasticity.
- Understand business taxonomy and provide data analysis, design, modeling, and implementation leadership to project teams, database development group(s), IT, and the enterprise and ensure that analytical solutions are aligned with our architectural direction.
- Develop conceptual, logical and physical design for various data types and large volumes.
- Architect, design and implement high performance, large volume data integration processes, database, storage, and other back-end services
- Implement security, encryption best practices for big data environments.
- Create and maintain data flow diagrams, entity relationship diagrams, system diagrams, business and logical systems integration as it pertains to Analytics/Business Intelligence and big data systems.
- Find and implement solutions to integrate event-based, semi-structured and 3rd party data sources (e.g. E-commerce transaction data) with enterprise relational models and strategies for transactional and analytical systems to effectively share data.
- Partner with our Enterprise Architecture team to ensure that the defined architectures align with other architecture domains (e.g. infrastructure, EDW, security, etc).
- Ability to perform data related benchmarking, performance analysis and tuning
- Solid experience with NoSQL and MPP/columnar database technologies
- Demonstrated ability to architect and model mission critical BI, OLAP, OLTP, ETL, NoSQL, MPP, In-memory and batch Hadoop solutions leveraging multiple data technologies (Oracle, Teradata, Informatica; Hive, HBase; Impala, Solr, Spark)
- Operational expertise in the use of SQL/HQL/NoSQL
- Demonstrated experience translating business and technical requirements into comprehensive data reporting strategies and analytic solutions.
- Extensive background and expertise in developing and managing data technologies, technical operations, reusable data services, and related tools and technologies.
- Excellent communication and interpersonal skills as well as leadership and management abilities
- 10+ years of IT experience; 4 focused in areas of Big Data architecture and operation.
- 5+ years architecting big data solutions with a major RDBMS platform (Oracle, Teradata)
- 4+ Experience with Hadoop ecosystem frameworks such as: MapReduce, Hive, Yarn, HBase, Pig Latin, Storm, Kafka, Flume HCatalog, Spark, Oozie, Zookeeper, Java and others
- 2 years functioning as a Technical Leader/Manager role in a Big Data environment.
- Working knowledge of setting up, configuring and optimization of Hadoop clusters
- Experience with one or more of Hadoop-capable BI tools such as Datameer, Platfora, Splunk, and similar
- Experience with cloud and platform services, including self-provisioning, dynamic resource allocation and usage metering
- Ability to lead large scale, long-term data projects and work effectively leading cross-functional teams
- Experience executing or managing data operations is a plus