| --- |
| license: mit |
| tags: |
| - reinforcement-learning |
| - multi-agent |
| - time-series |
| - diffusion-model |
| - energy-management |
| - smart-grid |
| --- |
| # SolarSys: Scalable Hierarchical Coordination for Distributed Solar Energy |
|
|
| SolarSys is a novel **Hierarchical Multi-Agent Reinforcement Learning (HRL)** system designed to manage energy storage and peer-to-peer (P2P) trading across large communities of solar-equipped residences. This repository contains the full source code for the SolarSys system, including the trained policies, the custom Gym environment, and the hierarchical diffusion model used for data generation. |
|
|
| --- |
|
|
|
|
| ## System Architecture |
|
|
| The core of SolarSys is a two-level decision hierarchy: |
|
|
| 1. **Low-Level (Intra-Cluster):** Individual households use a **MAPPO** agent to make instantaneous decisions (charge, discharge, local P2P trade, grid trade) based on local meter readings and price signals. |
| 2. **High-Level (Inter-Cluster):** Cluster Managers use a **Mean-Field** policy to coordinate bulk energy transfers between clusters, ensuring the overall system remains balanced against grid constraints. |
|
|
|
|
|  |
|
|
| --- |
|
|
| ## Data Generation Framework |
|
|
| To enable large-scale simulation with realistic temporal dynamics, SolarSys includes a **Hierarchical Diffusion Model** for generating synthetic, long-duration energy profiles that maintain both long-term (seasonal/monthly) and short-term (daily/hourly) characteristics. |
|
|
| * **Model:** Hierarchical Diffusion U-Net |
| * **Input:** Location based housing dataset(eg. Ausgrid dataset, newyork dataset) |
| * **Output:** High-resolution time series for Grid Usage and Solar Generation (kWh). |
|  |
|
|
| To ensure all necessary libraries are available, install dependencies using the provided `requirements.txt` file: |
|
|
| ```bash |
| pip install -r requirements.txt |
| --- |
| |
| ## Repository Structure |
| |
| The project is organized into core modules and data folders. |
| |
| ```tree |
| SolarSys_Hugging_face/ |
| ├── assets/ |
| ├── requirements.txt # Project Dependencies |
| ├── Dataset/ # Energy Data (Raw, Processed, and Synthetic) |
| │ ├── Data/ |
| │ │ ├── Ausgrid_processed_data.zip # Processed data (Australia) |
| │ │ ├── Ausgrid_raw_data/ |
| │ │ └── Pecan_street_processed_data.zip # Processed data (US) |
| │ └── Generated_Data.zip # Example synthetic data (2000 agents) |
| ├── Data_generation_tool_kit/ # Hierarchical Diffusion Model Code |
| │ ├── Hier_diffusion_energy/ |
| │ │ ├── hierarchial_diffusion_model.py # H-Diffusion U-Net Architecture |
| │ │ └── global_scaler.gz # Global normalization scaler, please delete and rerun if had problem |
| │ ├── dataloader.py # |
| │ ├── train.py # Diffusion model training script |
| │ └── generate.py # Script to generate long-term sequences |
| ├── Models/ # Trained RL Policies for trial, please retrain your model for best results |
| │ ├── 100agents_10size/ |
| │ │ └── models/ |
| │ │ ├── inter_ep10000.pth # High-Level (Inter-Cluster) policy |
| │ │ └── low_clusterX_ep10000.pth # Low-Level (Intra-Cluster) policies |
| │ └── (Additional scaling folders) # e.g., 5agents_5size/, 1000agents_20size/, etc. |
| ├── SolarSys/ # SolarSys Implementation (MAPPO + MeanField) |
| │ ├── Environment/ |
| │ │ ├── cluster_env_wrapper.py # Vectorized environment wrapper |
| │ │ └── solar_sys_environment.py # Core Gym environment definition |
| │ ├── mappo/ # MAPPO policy definition (Intra-Cluster) |
| │ │ └── trainer/ |
| │ │ └── mappo.py # MAPPO algorithm |
| │ ├── meanfield/ # MeanField policy definition (Inter-Cluster) |
| │ │ └── trainer/ |
| │ │ └── meanfield.py # MeanField algorithm core |
| │ ├── cluster.py # Inter-Cluster Coordinator logic |
| │ ├── training_freezing.py # Main SolarSys HRL training script |
| │ └── cluster_evaluation.py # Evaluation script for SolarSys |
| └── Other_algorithms/ # Baselines |
| ├── HC_MAPPO/ # Hierarchical Cluster MAPPO |
| │ ├── Environment/ |
| │ ├── HC_MAPPO_train.py # Training script for HC-MAPPO |
| │ └── HC_MAPPO_evaluation.py # Evaluation script for HC-MAPPO |
| └── Flat_System/ # Flat Baselines |
| ├── maddpg/ # Multi-Agent DDPG |
| │ ├── maddpg_train.py |
| │ └── maddpg_evaluation.py |
| ├── mappo/ # Flat MAPPO |
| │ ├── mappo_train.py |
| │ └── mappo_evaluation.py |
| ├── meanfield/ # Flat MeanField Actor-Critic (MFAC) |
| │ ├── meanfield_train.py |
| │ └── meanfield_evaluation.py |
| ├── PG/ # Policy Gradient (PG) |
| │ ├── pg_train.py |
| │ └── pg_evaluation.py |
| └── solar_sys_environment.py # Copy of the flat environment for baselines |
