Weather Data / ML Engineer

Get AI-powered advice on this job and more exclusive features.

BAM is seeking a highly skilled and experienced Weather Data Engineer/ML Engineer to develop and operate a proprietary weather analytics platform and bridge the gap between data science and software engineering. Weather data and AI-driven modeling are at the core of our strategy to deliver a forecasting edge and drive Sharpe improvement for our trading business. The role will focus on productionizing, scaling, and maintaining AI-based weather forecast models in real-world systems.

As a key member of our team, you will design, implement, and maintain cloud-native (AWS) data pipelines and infrastructure for ingesting, processing, and serving real-time and historical weather data—including raw observations, climate simulation data, and ML weather prediction model outputs. You will play a critical role in supporting our Commodity and Equity teams AI efforts, primarily supporting the ML ops for development and deployment of proprietary, decorrelated forecasting models, and enabling advanced analytics and research.

You will collaborate closely with portfolio managers, analysts, meteorologists, data scientists, technologist and AI researchers to integrate new weather datasets, support signal postprocessing and debiasing, and drive innovation in weather analytics. You will also work with central BAM resources to ensure seamless integration and operational excellence.

Responsibilities

• ML Ops: Design, implement, and maintain scalable machine learning infrastructure and automated model deployment pipelines to ensure robust, reliable, and efficient ML operations.
• Build data pipelines: Design, implement, and maintain scalable, cloud-native (AWS) data pipelines for ingesting, processing, and storing real-time and historical weather data, including raw observations (e.g., satellite, radar, sensor networks) and climate simulation data (e.g., CMIP6).
• Enable data access: Develop and maintain robust APIs and data services to enable efficient access to weather data and AI model outputs for analytics, modeling, and visualization.
• Deploy models: Integrate trained models into live applications and ensuring they operate reliably at scale; ensure model backfill completeness and consistency.
• Monitoring and maintain models: Tracking model performance, detect drift, retraining as needed, optimize for efficiency, and troubleshooting issues including model blending, debiasing, and finetuning workflows.
• Collaborating with cross-functional teams: Work closely with portfolio managers, financial analysts, meteorologists, data scientists, and AI researchers to onboard, profile, and optimize new weather datasets and support research projects.
• Ensure data quality: Implement and automate data quality validation, monitoring, and alerting to ensure high reliability and availability of all weather data feeds.
• Optimize for performance and cost: Ensuring models are efficient, scalable, and cost-effective in production environments.
• Champion best practices: version control, CI/CD, automated testing, code review, refactoring, documentation and broad knowledge sharing.

Requirements

• Academic Degree: Degree in Computer Science, Atmospheric Science, Engineering, or a related field with a computational focus.
• Minimum Experience: 5+ years of hands‑on development experience building and supporting production data systems.
• Python 3 Proficiency: Advanced programming skills in Python 3 with experience in data analysis and geospatial libraries (pandas, numpy, GeoPandas).
• AWS: Hands-on experience with AWS services including S3, EC2, Lambda, RDS, and geospatial services like Amazon Location Service.
• Snowflake: Proven experience with Snowflake data warehouse, including data modeling, optimization, and geospatial functions.
• PostGIS: Strong expertise in PostGIS for spatial database operations, spatial indexing, and complex geospatial queries.
• System architecture knowledge: Strong understanding of system architecture and the full technology stack (software, OS, CPU/memory, local/network storage, networking, etc.).
• Experience within collaborative software development environments: version control, CI/CD, automated testing, code review, and refactoring.
• Database knowledge: Strong knowledge of one or more relevant database technologies (e.g., Postgres, Redshift, Snowflake).
• Weather data and workflow experience: Experience working with weather, climate, or environmental datasets (e.g., GRIB, NetCDF, HDF5, CSV, JSON); familiarity with weather data sources and formats (e.g., NOAA, ECMWF, GFS, satellite, radar, sensor networks); familiarity with weather modeling, forecasting, or analytics workflows.

Beneficial

• Proficient in one or more object-oriented programming languages (e.g., Java, C#).
• Experience with distributed computing frameworks (e.g., Spark, Dask, Slurm).
• Experience with event-driven, asynchronous architectures and messaging technologies (e.g., Kafka, RabbitMQ).
• Experience with orchestration and container technologies (e.g., Airflow, Kubernetes, Docker).
• Experience with monitoring and alerting tools (e.g., CloudWatch, Prometheus, Grafana, Sentry/OTel).
• Experience with dashboarding, uncertainty quantification, and supporting research analytics.

Seniority level

Mid-Senior level

Employment type

Full-time

Job function

Engineering and Finance