Miramar, FL
Date Added: May 17, 2023
Software Resources has an immediate job opportunity for a Sr. Data Solutions Engineer with a major corporation in Miramar, FL.
7+ month contract
Job Description:
The Senior Data Solutions Engineer is responsible for building, managing, and optimizing complex reusable enterprise data pipelines effectively and in a timely manner through the development lifecycle to be used by internal consumers, such as business/data analysts and data scientists. Additionally, this role will operate in a fast pace environment working closely with the Data Science and Business teams on Revenue Management initiatives which concentrates in building data solutions, curated data to enable intelligent pricing automation, which relies on machine-learning and data-derived business rules. The Senior Engineer would use both technical and analytical skills to understand and solve business problems using available resources and current technology stack, while ensuring data governance and data security compliance.
1. Create and maintain technical design documentation
2. Gather and document business requirements, data mapping and designing
3. Create, build, and maintain complex data pipelines from disparate sources that meet functional / non-functional business requirements
4. Create, maintain, and reuse existing ETL/ELT processes, employing a variety of data integration and data preparation tools
5. Mentor engineers in finding optimal and efficient solutions for designing, preparing, and storing data for analytical and operational use cases.
6. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing pipelines for greater scalability, etc.
7. Work with stakeholders including Product, Data and Business teams to assist with data-related technical issues and support their data needs
8. Create datasets for: (1) operational reports, key performance indicators/metrics, or other insights into current organizational activities, (2) analytics and data science to provide the ability to uncover the answers to major questions that help organizations make objective decisions and/or gain a competitive edge
9. Write, debug and implement complex queries involving multiple tables or databases across platform(s)
10. Collaborate with the Enterprise Architecture team to ensure alignment on data standards and processes
11. Work with data and analytics experts to strive for greater functionality in data systems
12. Manages and develops data processing and assigns customers to marketing segments in the CDP; using demographic data, behavioral data, and intent signals and works with data scientists to incorporate machine learning outcomes into the overall customer segmentation model
Required Skills:
1. Significant experience in using best practices in designing, building and managing data pipelines that require data transformations as well as metadata and workload management
2. Significant experience in working with large, heterogeneous datasets in building and optimizing data pipelines, pipeline architectures and integrated datasets using traditional and new data integration technologies (such as ETL, ELT, data replication, change data captures, message-oriented data movement, API design, stream data integration and data virtualization)
3. Significant experience with streaming technologies (Kafka, Pubsub, Kinesis) and log-based architectures and experience writing batch and stream processing jobs (i.e. Apache Beam, Google Cloud DataFlow, Apache Spark, Apache Storm)
4. Significant experience in performing root cause analysis on internal and external data and processes to identify issues and opportunities for improvement
5. Expert level knowledge with programming languages including SQL, PL/SQL, T-SQL
6. Expert level knowledge with relational SQL databases such as Oracle and SQL Server
7. Significant experience with a scripting language & streaming technologies: Python, Java, Scala, Kafka, etc.
8. Experience with NoSQL databases are a plus
9. Experience supporting and working with cross-functional teams in a dynamic environment
Required Education:
Bachelor of Science in Computer Science, Information Technology or equivalent
Required Years of Experience:
5+ years of experience in a data/cloud engineering role
5+ years of experience working and creating datasets for a data warehouse
5+ years of experience with ETL development tools, Informatica or Azure Data Factory (ADF) preferred
3+ years of cloud experience, Azure preferred
Desired Skills:
1. Clear understanding of data modeling patterns (relational and dimensional)
2. Proven ability to collaborate with technical peers
3. Capable of working independently as well as part of a team
4. Strive to provide orientation and direction to junior engineers requiring their expertise
5. Experienced with continuous integration and continuous deployment practices
6. Ability to approach complex problems with creativity and display analytical and problem-solving skills
7. Display curiosity in understanding the data for the specific area of responsibility
8. Knowledge and experience working with agile methodologies and tools (such as Jira) a plus
Don't delay. Join the Software Resources team today!
Software Resources specializes in connecting talented IT professionals with challenging job opportunities that transform jobs into careers.
To meet our clients hiring needs, we continuously source talented IT Professionals with all levels of expertise and in all disciplines. We offer world class major medical, dental and vision benefits, 401(k) with match, short term disability, Life Insurance and AD&D. You, our future employees, can make a tremendous difference to our company and our clients. Please apply to this job and experience the Software Resources difference. You can view all of our jobs at https://www.softwareresources.com/careers/
7+ month contract
Job Description:
The Senior Data Solutions Engineer is responsible for building, managing, and optimizing complex reusable enterprise data pipelines effectively and in a timely manner through the development lifecycle to be used by internal consumers, such as business/data analysts and data scientists. Additionally, this role will operate in a fast pace environment working closely with the Data Science and Business teams on Revenue Management initiatives which concentrates in building data solutions, curated data to enable intelligent pricing automation, which relies on machine-learning and data-derived business rules. The Senior Engineer would use both technical and analytical skills to understand and solve business problems using available resources and current technology stack, while ensuring data governance and data security compliance.
1. Create and maintain technical design documentation
2. Gather and document business requirements, data mapping and designing
3. Create, build, and maintain complex data pipelines from disparate sources that meet functional / non-functional business requirements
4. Create, maintain, and reuse existing ETL/ELT processes, employing a variety of data integration and data preparation tools
5. Mentor engineers in finding optimal and efficient solutions for designing, preparing, and storing data for analytical and operational use cases.
6. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing pipelines for greater scalability, etc.
7. Work with stakeholders including Product, Data and Business teams to assist with data-related technical issues and support their data needs
8. Create datasets for: (1) operational reports, key performance indicators/metrics, or other insights into current organizational activities, (2) analytics and data science to provide the ability to uncover the answers to major questions that help organizations make objective decisions and/or gain a competitive edge
9. Write, debug and implement complex queries involving multiple tables or databases across platform(s)
10. Collaborate with the Enterprise Architecture team to ensure alignment on data standards and processes
11. Work with data and analytics experts to strive for greater functionality in data systems
12. Manages and develops data processing and assigns customers to marketing segments in the CDP; using demographic data, behavioral data, and intent signals and works with data scientists to incorporate machine learning outcomes into the overall customer segmentation model
Required Skills:
1. Significant experience in using best practices in designing, building and managing data pipelines that require data transformations as well as metadata and workload management
2. Significant experience in working with large, heterogeneous datasets in building and optimizing data pipelines, pipeline architectures and integrated datasets using traditional and new data integration technologies (such as ETL, ELT, data replication, change data captures, message-oriented data movement, API design, stream data integration and data virtualization)
3. Significant experience with streaming technologies (Kafka, Pubsub, Kinesis) and log-based architectures and experience writing batch and stream processing jobs (i.e. Apache Beam, Google Cloud DataFlow, Apache Spark, Apache Storm)
4. Significant experience in performing root cause analysis on internal and external data and processes to identify issues and opportunities for improvement
5. Expert level knowledge with programming languages including SQL, PL/SQL, T-SQL
6. Expert level knowledge with relational SQL databases such as Oracle and SQL Server
7. Significant experience with a scripting language & streaming technologies: Python, Java, Scala, Kafka, etc.
8. Experience with NoSQL databases are a plus
9. Experience supporting and working with cross-functional teams in a dynamic environment
Required Education:
Bachelor of Science in Computer Science, Information Technology or equivalent
Required Years of Experience:
5+ years of experience in a data/cloud engineering role
5+ years of experience working and creating datasets for a data warehouse
5+ years of experience with ETL development tools, Informatica or Azure Data Factory (ADF) preferred
3+ years of cloud experience, Azure preferred
Desired Skills:
1. Clear understanding of data modeling patterns (relational and dimensional)
2. Proven ability to collaborate with technical peers
3. Capable of working independently as well as part of a team
4. Strive to provide orientation and direction to junior engineers requiring their expertise
5. Experienced with continuous integration and continuous deployment practices
6. Ability to approach complex problems with creativity and display analytical and problem-solving skills
7. Display curiosity in understanding the data for the specific area of responsibility
8. Knowledge and experience working with agile methodologies and tools (such as Jira) a plus
Don't delay. Join the Software Resources team today!
Software Resources specializes in connecting talented IT professionals with challenging job opportunities that transform jobs into careers.
To meet our clients hiring needs, we continuously source talented IT Professionals with all levels of expertise and in all disciplines. We offer world class major medical, dental and vision benefits, 401(k) with match, short term disability, Life Insurance and AD&D. You, our future employees, can make a tremendous difference to our company and our clients. Please apply to this job and experience the Software Resources difference. You can view all of our jobs at https://www.softwareresources.com/careers/