--- title: NeuralPOM canonical_url: "https://www.modelscope.cn/datasets/chiyodamomo01/NeuralPOM" md_url: "https://www.modelscope.cn/datasets/chiyodamomo01/NeuralPOM.md" repository: chiyodamomo01/NeuralPOM last_updated: 2026-11-03 license: "Apache License 2.0" storage_size: "4.9 TB" downloads: 4869 stars: 1 --- # NeuralPOM > NeuralPOM - chiyodamomo01 在 ModelScope 开源的数据集。NeuralPOM · Ocean Reference Ensembles chiyodamomo01/NeuralPOM 是 ModelScope 魔搭社区上的数据集，存储大小 4.9 TB，采用 Apache License 2.0 许可。 - **Repository**: chiyodamomo01/NeuralPOM - **License**: Apache License 2.0 - **Storage size**: 4.9 TB - **Downloads**: 4869 - **Stars**: 1 - **Last updated**: 2026-11-03 Source: https://www.modelscope.cn/datasets/chiyodamomo01/NeuralPOM --- ![NeuralPOM reference ensembles: differentiable ocean dynamics and staged learning](assets/neuralpom-reference-banner.png) # NeuralPOM · Ocean Reference Ensembles **Fine-grid-generated reference data for three-dimensional ocean forecasting.** [中文说明](README_zh.md) · [arXiv](https://arxiv.org/abs/2611.02460) · [Paper PDF](https://arxiv.org/pdf/2611.02460) · [Code](https://github.com/ChiyodaMomo01/NeuralPOM) · [Quick start](#quick-start) · [Data guide](docs/DATA_GUIDE.md) · [Benchmark protocol](docs/BENCHMARK.md) · [Citation](#citation) NeuralPOM couples differentiable Princeton Ocean Model dynamics with conservation-constrained neural state updates. This data repository contains the **explicit mode-split** and **semi-implicit free-surface** reference ensembles used to train and evaluate the system in an idealized wind-driven double-gyre basin. Fine-grid simulations are conservatively restricted to the stored forecast grid, retaining daily numerical states and subdaily supervision. [1, §3.1 and Appendix B.1] > **Access status: private staging.** Transfer and final acceptance are in progress. The machine-readable [transfer record](metadata/transfer_progress.json) and [release record](metadata/release.json) carry the current status. This page prepares the dataset for reproducible use; it does not announce a public release. | At a glance | Archive specification | |---|---| | Reference ensembles | **2 numerical frameworks × 8 members** | | Common coverage | **9,000 file days per member**, 0000–8999; additional explicit tails are recorded separately | | Generation → storage | **354 × 322 × 16 → 90 × 82 × 16**; 15 active vertical layers | | Daily state | **45 NPZ fields**, including current and historical numerical levels | | Intraday supervision | **12 NPZ fields**, every 20 minutes during the **first four hours** of each day | | Planned contents | **2,004,264 NPZ files**, carried by **3,090 TARs + 16 rolling restarts** | | Packaging | Original NPZ bytes inside uncompressed, 100-file-day TAR blocks; one `FILES.jsonl` per TAR | Counts and coverage come from the [upload plan](metadata/plan.json); grid and sampling definitions follow [1, §3.1] and [sampling metadata](metadata/sampling.json). The paper's **5.4 TB** describes the historical saved-source inventory, including metadata and temporary files; the accepted transfer total will be reported separately. ## Authors and affiliations

Ruiqi Shu1,2,†  ·  Guangliang Liu1,3,4,†  ·  Ruijian Gou5,†  ·  Tianrun Gao6  ·  Haomin Wen7  ·  Hao Li8,7,2,*  ·  Xiaomeng Huang1,*

1. Department of Earth System Science, Tsinghua University, Beijing, China 2. Shanghai Academy of Artificial Intelligence for Science, Shanghai, China 3. State Key Laboratory of Physical Oceanography, Institute of Oceanographic Instrumentation, Shandong Academy of Sciences, Qingdao, China 4. Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China 5. Department of Ocean Big Data and Artificial Intelligence, Laoshan National Laboratory, Qingdao, China 6. School of Computing and Data Science, The University of Hong Kong 7. Shanghai Innovation Institute, Shanghai, China 8. Artificial Intelligence Innovation and Incubation Institute, Fudan University, Shanghai, China † Equal contribution. * Correspondence: Hao Li (lihao_lh@fudan.edu.cn) and Xiaomeng Huang (hxm@tsinghua.edu.cn). ## Abstract Coarse-grid ocean circulation models rely on empirical subgrid parameterizations that struggle to capture mesoscale eddies, whereas purely data-driven neural forecasters accumulate trajectory drift, violate tracer conservation, and damp small-scale variability over extended integrations. Here we present NeuralPOM, a hybrid ocean forecast model that couples differentiable explicit mode-split and semi-implicit free-surface implementations of the Princeton Ocean Model (POM) to a conservation-constrained neural corrector. The three-dimensional primitive-equation solver advances the ocean state, while the neural network periodically updates the physical fields at high-frequency (1 h) or low-frequency (1 day) intervals under volume, heat, and salt conservation constraints. Using a fine-grid reference dataset exceeding 100,000 model days in a wind-driven double-gyre basin, we introduce a three-stage training procedure comprising offline single-step pretraining, multi-step differentiable training through the ocean solver, and multi-day closed-loop rollout training. Over 100-day forecasts, NeuralPOM reduces errors across all prognostic fields relative to coarse-grid physical closures and direct neural forecasters, preserving mesoscale kinetic energy while improving eddy phase alignment. Controlled ablations show that closed-loop training suppresses multi-day drift, whereas backpropagating gradients through the differentiable solver stabilizes high-frequency coupling. Finally, because high-frequency tracer updates can introduce density anomalies via the equation of state that excite spurious baroclinic pressure gradients, we introduce Density-Normal Damping (DND) to attenuate the density-modifying (diapycnal) tracer component while retaining the density-compensated (isopycnal) correction. ## The reference ocean ![Developed double-gyre circulation: vorticity, sea-surface elevation, temperature, zonal and meridional velocity, and salinity](assets/double-gyre-reference.png) *An instantaneous fine-grid explicit reference snapshot at model day 10,298, on the 352 × 320 wet interior. From the paper's “Developed double-gyre circulation” figure. This illustrates the generating simulation; the downloadable trajectory fields are stored on the 90 × 82 grid.* [1, Appendix D] The archive supports **hybrid model training**, **full-state neural forecasting**, and **100-day forecast evaluation**. Use each numerical framework's own reference ensemble for its corresponding comparison. [1, §§3–4] ## Quick start ### 1. Start with a small, verified sample Install the SDK and NumPy in your own environment: ```bash python -m pip install modelscope-hub==0.4.6 numpy ``` For private access, provide **your own authorized token** through `MODELSCOPE_API_TOKEN` in your environment. Use a fixed, accepted repository commit for published experiments; `master` is suitable for inspecting the current staging snapshot. ```python from pathlib import Path from modelscope_hub import HubApi import io import tarfile import numpy as np api = HubApi() revision = "master" # Replace with the accepted 40-character commit for reproducibility. archive = api.download_file( "chiyodamomo01/NeuralPOM", "dataset", "samples/implicit/sample_implicit_m0_d1000.tar", revision=revision, local_dir="./neuralpom_sample", ) # Inspect a member directly, without unpacking the archive. entry = "member_0_gpu0/reference_data_lowres_consistent/ref_Day1000_seg02.npz" with tarfile.open(archive, "r:") as tar: with tar.extractfile(entry) as stream: payload = stream.read() with np.load(io.BytesIO(payload), allow_pickle=False) as state: print(state.files) print(state["u"].shape) # (90, 82, 16) print(float(state["time"])) # Approximately 1000 + 1/24 model days. ``` The initial two-core sample passed download, original-byte comparison, ZIP CRC, field, shape and clock checks. Full acceptance of all planned objects is a separate stage. See [sample validation](metadata/sample_validation.json). ### 2. Select the blocks you need Download [the selection helper](tools/download_reference.py), then preview the block selection: ```bash python download_reference.py \ --core implicit --members 0 \ --start-day 1000 --end-day 1001 --kind both \ --revision master --output ./neuralpom_data --list-only ``` Replace `--list-only` with `--extract` to download and safely extract the selected archives. The helper fetches the initial daily checkpoint as well as the selected reference blocks. See the [download and time guide](docs/DATA_GUIDE.md#selective-downloads) for endpoint semantics and fixed-version examples. Avoid cloning or downloading the entire multi-terabyte repository just to inspect a sample. ## What is in the repository? ```text NeuralPOM/ ├── README.md # English dataset card ├── README_zh.md # Chinese reading guide ├── paper/NeuralPOM_Report.pdf # Manuscript snapshot used for this card ├── docs/ # Data, benchmark and release guides ├── assets/ # Dataset banner and paper figures ├── samples/{implicit,explicit}/ # Small download-validated examples ├── {implicit,explicit}/ │ ├── manifest.json │ └── members/member_0/ # Members 0–7 │ ├── provenance.json │ ├── checkpoints/day0000-0099.tar │ └── reference_20min_first4h/day0000-0099.tar ├── generation_restarts/ # One final fine-grid restart per member ├── shared/geometry/ # Registered grid, masks and provenance ├── shared/forcing/ # Reference forcing description ├── metadata/ # Coverage, variables, splits and progress └── tools/download_reference.py # Select by framework, member and time ``` Inside each TAR, the original `member_i_gpui/...` paths are retained. **Separate the explicit and implicit extraction roots** to prevent same-named source files from overwriting one another. The selection helper does this automatically. ### Time, variables and geometry - **Checkpoint file day D represents nominal model time D + 1.** A forecast starting at model day 1000 uses `checkpoint_Day0999.npz`. - **Reference segment s represents nominal time D + (s + 1)/72**, where `s` runs from 0 to 11. Segments **02, 05, 08, 11** give the 1, 2, 3 and 4 hour endpoints. - **Model days have no calendar epoch.** Intraday targets do not cover the remaining 20 hours of a day. - Arrays retain the original staggered-grid conventions, temperature/salinity offsets and numerical buffers. Consult the [data dictionary](metadata/variables.json), [sampling metadata](metadata/sampling.json), and [geometry provenance](shared/geometry/provenance.json) before interpreting them as physical fields. The [data guide](docs/DATA_GUIDE.md) explains field groups, source paths, extraction and the documented manuscript–geometry discrepancy. ## Benchmark protocol | Role | Members | Model-time selection | Use | |---|---|---|---| | Training | 0–6 | Starts within [1000, 7000) | Fit model parameters and training statistics | | Validation | 7 | Starts within [7000, 8000) | Model selection | | Development evaluation | 7 | 16 origins, 8000 + 32r, r = 0…15 | 100-day forecast comparisons and development diagnostics | | Reserved held-out origins | 7 | 8800 and 8900 | Kept separate from development decisions | Stage-specific admissible starts and end-boundary exclusions are enumerated in the [training/validation CSV](metadata/splits/training_validation_starts.csv). The [16-origin CSV](metadata/splits/development_16.csv) and [reserved-origin CSV](metadata/splits/heldout_2.csv) preserve the archive's registered roles. The manuscript describes the 16-origin forecast comparisons in §3.2; those development comparisons are not relabeled as independent blind tests here. ## Connection to NeuralPOM ![NeuralPOM physical–neural coupling and the three-stage training procedure](assets/neuralpom-framework.png) *Paper Fig. 1: constrained neural updates coupled to differentiable ocean dynamics, followed by single-step pretraining, multi-step differentiable training and closed-loop rollout training.* [1, Fig. 1] Daily checkpoints retain numerical histories needed to initialize the physical solver; intraday fields provide targets for short forecast intervals. The paper studies conservation-constrained velocity and tracer increments and **Density-Normal Damping (DND)**, which attenuates the density-modifying component of temperature–salinity corrections. The reference data remain the original simulation outputs. [1, §§2, 3 and 5] **Scope:** this repository transfers numerical reference data and supporting metadata; executable model code, trained weights, forecast outputs and training caches are separate artifacts. See [release and provenance](docs/RELEASE.md). ## Citation **[1]** Shu, R., Liu, G., Gou, R., Gao, T., Wen, H., Li, H. and Huang, X. (2026). *New Wine in an Old Bottle — NeuralPOM: A Hybrid Ocean Forecast Model with Differentiable Dynamics and Staged Learning*. arXiv:2611.02460 [physics.ao-ph]. [arXiv](https://arxiv.org/abs/2611.02460) · [PDF](https://arxiv.org/pdf/2611.02460) · [Repository PDF](paper/NeuralPOM_Report.pdf) · [BibTeX](CITATION.bib) ```bibtex @misc{shu2026neuralpom, title = {New Wine in an Old Bottle -- NeuralPOM: A Hybrid Ocean Forecast Model with Differentiable Dynamics and Staged Learning}, author = {Shu, Ruiqi and Liu, Guangliang and Gou, Ruijian and Gao, Tianrun and Wen, Haomin and Li, Hao and Huang, Xiaomeng}, year = {2026}, eprint = {2611.02460}, archivePrefix = {arXiv}, primaryClass = {physics.ao-ph}, url = {https://arxiv.org/abs/2611.02460} } ``` When reporting results, record the **dataset repository, accepted commit, selected frameworks/members/blocks and split role**, in addition to citing the manuscript. A DOI or accepted release identifier will be added when assigned; none is implied by this staging version. ## License and access The repository's existing **Apache License 2.0** metadata is retained during private staging. Final public data licensing and release scope require the author's release decision. Upstream POM software licensing and the paper are separate from the numerical-data license. See [release policy](docs/RELEASE.md#license-and-access). For scientific correspondence, use the contacts printed in the [manuscript](paper/NeuralPOM_Report.pdf). For a reproducibility report, include the repository commit and affected archive path, without credentials or private source-system paths.