Data Engineer I or II

DATA ENGINEER I or II

The Enterprise Decision Support and Data Governance (EDGE) Innovation team is seeking a highly skilled Data Engineer who will also serve as a Data Subject Matter Expert (SME). In this role, you will play a crucial part in developing and maintaining the enterprise curation layer within the EDGE. You will leverage your extensive experience in data engineering to ensure the data within EDGE is accessible, reliable, and high performing for our utilization by our Innovation, data & analytics teams throughout the enterprise.

This position follows a hybrid work schedule. Employees are required to be in the office four (4) days per week.

This position may be filled as an Engineer I (GSE05) or Engineer II (GSE06). Individuals who demonstrate advanced skills in terms of work experience, education, and certifications may be eligible for the more advanced position.

SUMMARY 

The Data Engineer will collaborate with business and technology stakeholders to understand requirements, define data models, and implement Gas data assets. They will manage data governance and maintain a data catalog. The role includes supporting machine learning model creation, training, operation, and automation. Additionally, they will enforce data standards and collaborate with various teams to deploy optimized data solutions, plan capacity, and ensure secure data storage and usage.

MAJOR JOB RESPONSIBILITIES: 

Functional Expertise:

• Provide key knowledge, insight, and informational guidance related to the integration of data and design of the holistic data landscape in support of the EDGE organization’s objectives
• Translate Decision Support Requirements related to KPI’s, Indices, and other analytical insights into Executable strategies, approaches and platform designs or other means of information delivery
• Produce & Manage all Decision Support Solution Design Frameworks and their supporting Documentation 
• Identify and propose new capabilities and improvements to Decision Support capabilities
• Guide, Advise and Support other teams on implementation issues related to application of data related capabilities and approaches
• Perform expert oversight and guidance for delivery of assigned Decision Support Solutions
• Support Business Information Architects in the communication and interpretation of information and analysis to business leadership

Responsibilities

• Support EDGE Project related assignments as required
• Coordinates with the TO to identify approaches and solutions to data implementation problems related to the supporting EDGE solutions platforms
• Develop and enhance the data management processes within the curation layer of the EDGE platform, ensuring scalable and accurate data handling and retrieval.
• Create data models and construct efficient data pipelines, ETL processes, and workflows using Databricks, optimizing data ingestion transformation, and storage to support analytical needs.
• Collaborate with data scientist to refine data collections methods while building and tuning Databricks jobs and Apache Spark application for improved performance and scalability.
• Implement best practices for data management practices including partitioning, indexing, and compression to increase query efficiency.

Business Acumen:

• Evaluates and recommends approaches and strategies on deployment and application of the tools and technologies adopted by Southern Company with the TO in the development and delivery of solution strategy, platforms, and applications
• Develop and strengthen data standards and policies to maintain data integrity and consistency across the EDGE.
• Adhere to and implement data governance protocols to uphold high-security standards and clear data lineage.
• Proactively monitor data systems to identify and resolve issues that could impact data accuracy.

Engagement:

• Collaborates with other team members, TO and business managers to resolve complex business issues and problems
• Develops and maintains productive working relationships with key EDGE and TO managers and staff
• Engage with data scientists, analysts, and business stakeholders, to precisely understand their data needs and translate these into Databricks-based technical solutions.
• Offer technical guidance and support, enabling the seamless introduction and adoption of advanced data solutions.
• Remain abreast of the latest data engineering trends, particularly in Databricks and other cloud-based technologies.

DATA ENGINEER I 

Education, Certifications/Licenses:

Required: 

• Bachelor's degree in related discipline (e.g., Computer Science, Information Management or equivalent) or equivalent related work experience

Preferred:

• Master's degree in related discipline (e.g., Computer Science, Information Management or equivalent) or equivalent related work experience

Related Work Experience:

Required:

• 1 - 3 years data-oriented solution architecture development experience utilizing Both On-Premises and Cloud (Azure) modern data architecture platforms and patterns. 
• 3 years of experience developing context diagrams as well as conceptual, logical, physical architecture, data model and ER diagrams as well as data orchestration diagrams. 
• 2 years hands-on experience designing and implementing data models for structured and unstructured datasets utilizing tools such as ERWIN, SQL Developer, Python, SSMS, Visual Studio and Visio.
• Ability to deliver results in a cross functional team environment

Preferred:

• Expert knowledge and hands-on experience working with SQL. 
• Hands on experience working with relational databases such as Oracle, SQL Server as well as Azure Data Lake, Synapse Data Warehouse. 
• Experience in Data Governance processes and tools such as Informatica Data Catalog

Specific Skills & Knowledge: 

Required: 

• Advanced proficiency in MS Office Tools
• Experience in Data Quality processes and tools such as DQS 
• Hands-on experience developing big data technologies such as Distributed file Systems, HIVE and Spark  
• Ability to provide guidance on app/data performance, high availability and disaster recovery
• Experience with designing re-usable solution components and perform data model integrity checks
• Experience working in a medium to large enterprise setting and collaborating multiple cross-functional teams 
• Experience working on concurrent engineer assignments on multiple projects. 
• Significant knowledge of business requirements and solutions design methodologies, tools, and approaches
• Significant knowledge of graphical modeling approaches, tools, and repositories
• Significant knowledge of Logical and Physical Data Modelling approaches, tools, and methodologies, and supporting software platforms

DATA ENGINEER II

Education, Certifications/Licenses:  

Required: 

• Bachelor's degree in related discipline (e.g., Computer Science, Information Management or equivalent) or equivalent related work experience

Preferred:

• Master's degree in related discipline (e.g., Computer Science, Information Management or equivalent) or equivalent related work experience

Related Work Experience:

Required:

• 3 - 5 years data-oriented solution architecture development experience utilizing Both On-Premises and Cloud (Azure) modern data architecture platforms and patterns. 
• 3- 5 years of experience developing context diagrams as well as conceptual, logical, physical architecture, data model and ER diagrams as well as data orchestration diagrams. 
• 2- 4 years hands-on experience designing and implementing data models for structured and unstructured datasets utilizing tools such as ERWIN, SQL Developer, Python, SSMS, Visual Studio and Visio.
• Ability to deliver results in a cross functional team environment

Preferred:

• Expert knowledge and hands-on experience working with SQL. 
• Hands on experience working with relational databases such as Oracle, SQL Server as well as Azure Data Lake, Synapse Data Warehouse. 
• Experience in Data Governance processes and tools such as Informatica Data Catalog

Specific Skills & Knowledge: 

Required: 

• Advanced proficiency in MS Office Tools
• Experience in Data Quality processes and tools such as DQS 
• Hands-on experience developing big data technologies such as Distributed file Systems, HIVE and Spark  
• Ability to provide guidance on app/data performance, high availability and disaster recovery
• Experience with designing re-usable solution components and perform data model integrity checks
• Experience working in a medium to large enterprise setting and collaborating multiple cross-functional teams 
• Experience working on concurrent engineer assignments on multiple projects. 
• Significant knowledge of business requirements and solutions design methodologies, tools, and approaches
• Significant knowledge of graphical modeling approaches, tools, and repositories
• Significant knowledge of Logical and Physical Data Modelling approaches, tools, and methodologies, and supporting software platforms