AminiTech/Amini_FM_V0

Amini FM V0 is a compact open-weights transformer model for learning representations of satellite pixel time series. It accepts 512 time positions and 10 numerical channels per pixel.

The model is part of Amini's work on self-supervised representation learning for Earth observation. It adapts PatchTST to numerical spectral-band and index sequences, learning through reconstruction of randomly masked temporal patches. The released checkpoint contains an encoder and a patch reconstruction head.

Model details

Property	Value
Developer	AminiTech / Amini.ai
Architecture	PatchTSTForPretraining
Model size	Approximately 0.6M
Input shape	[batch_size, 512, 10]
Encoder layers	3
Hidden width	128
Attention heads	4
Feed-forward width	512
Patch length / stride	12 / 12
Cross-channel attention	Disabled
Normalization	batchnorm
Internal scaling	std
Masking	Random patches; configured ratio 0.4
Reconstruction objective	Mean squared error
CLS token	Enabled
Weight format / precision	Safetensors / float32
Exported Transformers version	4.38.2

Architecture details refer to this checkpoint configuration. Context length counts positions after preprocessing; it does not specify a calendar duration.

Uses

• Generate pixel time-series embeddings for similarity search and clustering.
• Train downstream land-use/land-cover classifiers with labeled data.
• Explore temporal patterns in agricultural and environmental data.

Classification, regression, and forecasting require an appropriate downstream model and task-specific evaluation. The reconstruction head does not produce land-cover labels or future forecasts. The prediction_length field in the configuration does not establish a trained forecasting capability.

Input and preprocessing

Provide finite float32 values with shape [batch_size, 512, 10], preserving chronological order and consistent channel definitions.

An optical preprocessing recipe documented for this model family uses the following channel order:

features = [
    "Green", "Blue", "RED", "NIR", "SWIR1", "SWIR2",
    "NDMI", "NDWI", "CI", "NDVI",
]

It sorts observations chronologically, groups them by pixel, standardizes each pixel's channels with StandardScaler, and linearly interpolates each sequence to 512 positions. This external preprocessing is followed by the model's internal standard-deviation scaling.

Keep feature definitions, channel order, and scaling consistent when adapting this preprocessing recipe to your data.

Training and related research

The exported configuration records self-supervised random patch masking and MSE reconstruction. The mask ratio is a configuration value; the realized fraction depends on the discrete number of patches.

Related research describes Sentinel-2 median biweekly composites from 2020-2024 and 50,000 chips across Africa. The companion study presents experiments on the compact model family. ICLR 2025 ML4RS paper

Usage

Install PyTorch, Transformers with PatchTST support, and Safetensors. The example below uses synthetic input to demonstrate the API. Real inputs require the preprocessing described above.

Extract embeddings

import torch
from transformers import PatchTSTModel

MODEL_ID = "AminiTech/Amini_FM_V0"
REVISION = "1261b01085f22f58bc70984aa32e5b041e206105"

encoder = PatchTSTModel.from_pretrained(
    MODEL_ID,
    revision=REVISION,
    use_safetensors=True,
    do_mask_input=False,
).eval()

past_values = torch.randn(
    1, encoder.config.context_length, encoder.config.num_input_channels,
    dtype=torch.float32,
)
observed_mask = torch.ones_like(past_values, dtype=torch.bool)

with torch.inference_mode():
    hidden = encoder(
        past_values=past_values,
        past_observed_mask=observed_mask,
    ).last_hidden_state

    # Shape: [batch, channels, patch_tokens_including_cls, hidden_width]
    patch_hidden = hidden[:, :, 1:, :]  # Exclude the prepended CLS token.
    per_channel_embeddings = patch_hidden.mean(dim=2)
    pixel_embeddings = per_channel_embeddings.mean(dim=1)

print(pixel_embeddings.shape)  # torch.Size([1, 128])

Loading the bare encoder omits the reconstruction head and may produce an unused-weight warning. This example averages patch embeddings and then channels; pooling should match the downstream training protocol. Channel-level or CLS-token embeddings can also be used.

Run masked reconstruction

from transformers import PatchTSTForPretraining

model = PatchTSTForPretraining.from_pretrained(
    MODEL_ID, revision=REVISION, use_safetensors=True,
).eval()

with torch.inference_mode():
    outputs = model(
        past_values=past_values,
        past_observed_mask=observed_mask,
    )
    reconstruction_loss = outputs.loss

The pretraining class applies random input masking even in evaluation mode. The encoder example explicitly disables masking for repeatable embeddings. Missing values must be represented consistently with the applicable preprocessing; the all-ones mask above is for fully populated synthetic input.

Evaluation and limitations

The usage examples have been checked with the released weights on CPU using PyTorch 2.6.0 and Transformers 4.48.3; this checks loading and forward-pass behavior, not downstream accuracy.

• Clouds, missing observations, interpolation, sensor differences, and feature scaling can affect embeddings.
• Transfer across regions, seasons, sensors, and acquisition conditions requires evaluation on representative data.
• Random pixel splits can contain spatially related examples; geographic and temporal holdouts are useful when assessing generalization.
• Reconstruction loss alone does not establish downstream classification, crop-yield prediction, calibrated change detection, or causal relationships.

License

A license has not been specified for this checkpoint. Apache-2.0 metadata on other Amini releases does not establish the license for these weights.

References

• Self-Supervised Representation Learning on Remote Sensing Pixel Time Series with Patch-Based Masking, ICLR 2025 ML4RS Workshop.
• Hugging Face PatchTST documentation.