Technical Architect/Manager- Data Engineering

Job Title: Technical Architect/Manager- Data Engineering

Job Summary:-

We are looking for a highly skilled and motivated Technical Architect/Managerto spearhead the development of scalable and efficient data engineering solutions. The ideal candidate will possess deep expertise in Python, PySpark, AWS services, and streaming data platforms, with a proven ability to integrate complex data sources and develop distributed data processing frameworks. This role requires a strong technical leader who can guide the team, solve complex challenges, and deliver optimal solutions that align with client requirements. Wealth management experience is an added benefit.

Key Responsibilities:

• Provide hands-on technical leadership to the team in designing and implementing data engineering solutions.
• Lead by example in adopting best practices for coding, testing, and deployment.

• Design and develop robust ETL pipelines using AWS Glue, Lambda, and other AWS services to process large volumes of data efficiently.
• Implement complex data transformations and integrate data from multiple sources such as APIs, databases, and streaming platforms.

• Design and implement streaming data pipelines using Kafka, AWS Kinesis, or similar technologies.
• Build scalable frameworks to handle real-time data ingestion and processing.

• Develop distributed data processing frameworks to ensure performance and scalability in handling large datasets.
• Optimize the performance of data processing jobs for both batch and real-time workloads.

• Provide optimal data engineering solutions aligned with client requirements and business objectives.
• Collaborate with architects to design scalable and secure data solutions leveraging AWS cloud services.

• Utilize AWS services (e.g., S3, Glue, Lambda, Kinesis, DynamoDB) to build efficient and scalable cloud-based solutions.
• Stay updated with the latest AWS services and features to continuously improve system performance and cost efficiency.

• Work closely with clients, business analysts, and other stakeholders to understand requirements and translate them into technical solutions.
• Communicate progress, challenges, and solutions effectively to both technical and non-technical stakeholders.

Qualifications:

• Education:
• Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
• Experience:
• 10+ years of experience in data engineering, with at least 3+ years in an Architect role.
• Strong hands-on expertise in Python, PySpark, and AWS services for data processing and integration.
• Proven experience designing and developing streaming data solutions using Kafka, AWS Kinesis, or similar technologies.
• Solid experience working with APIs and integrating data from diverse sources.
• Extensive knowledge of distributed data processing frameworks and best practices.
• Skills:
• Strong problem-solving and solution-oriented mindset to deliver optimal results.
• Excellent knowledge of data integration techniques and cloud-based architecture.
• Proficient in implementing complex data transformations and scalable data workflows.
• Exceptional team leadership and mentoring abilities.
• Strong communication skills for effective stakeholder collaboration.

Preferred Qualifications:

• AWS Certified Solutions Architect certification or equivalent.
• Familiarity with Terraform or CloudFormation for AWS infrastructure as code.
• Wealth Management domain experience.

Why Join Us?

• Be a key player in building cutting-edge data engineering solutions for large-scale projects.
• Work with a talented and collaborative team in a dynamic environment.
• Competitive salary, benefits, and opportunities for career growth.