Feature Extraction
Transformers
Safetensors
audio_embeddings
audio
custom_code
self-supervised-learning
audio-embeddings
best-rq-2
audioset
Instructions to use ltuncay/BEST-RQ-2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ltuncay/BEST-RQ-2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="ltuncay/BEST-RQ-2", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("ltuncay/BEST-RQ-2", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Download audio-embeddings/src/models/components/masking.py from ltuncay/BEST-RQ-2: direct link, hf CLI and curl.
- Browser
- Download file 2.33 kB
-
https://huggingface.co/ltuncay/BEST-RQ-2/resolve/main/audio-embeddings/src/models/components/masking.py
- Command line
-
hf download hf://ltuncay/BEST-RQ-2/audio-embeddings/src/models/components/masking.py
-
curl -L -o masking.py https://huggingface.co/ltuncay/BEST-RQ-2/resolve/main/audio-embeddings/src/models/components/masking.py
2.33 kB
| import torch | |
| import numpy as np | |
| from typing import Tuple, Optional | |
| class MaskingGenerator: | |
| """ | |
| Generates masks for the input patches. | |
| Args: | |
| input_size (tuple[int, int]): Input image size (H, W). | |
| patch_size (tuple[int, int]): Patch size (H, W). | |
| mask_ratio (tuple[float, float]): Range of mask ratio (min, max). | |
| """ | |
| def __init__( | |
| self, | |
| input_size: Tuple[int, int] = (128, 256), | |
| patch_size: Tuple[int, int] = (16, 16), | |
| mask_ratio: Tuple[float, float] = (0.4, 0.6), | |
| ): | |
| self.height, self.width = input_size | |
| self.patch_h, self.patch_w = patch_size | |
| self.num_patches_h = self.height // self.patch_h | |
| self.num_patches_w = self.width // self.patch_w | |
| self.num_patches = self.num_patches_h * self.num_patches_w | |
| self.mask_ratio = mask_ratio | |
| def __call__( | |
| self, | |
| batch_size: int, | |
| device: torch.device = torch.device("cpu"), | |
| grid_size: Optional[Tuple[int, int]] = None, | |
| ) -> torch.Tensor: | |
| """ | |
| Generate masks for a batch. | |
| Args: | |
| batch_size (int): Batch size. | |
| device (torch.device): Device to place the masks on. | |
| grid_size (Optional[Tuple[int, int]]): Grid size (H, W) if different from init. | |
| Returns: | |
| torch.Tensor: Masks [B, N] (boolean, True=masked). | |
| """ | |
| masks = [] | |
| for _ in range(batch_size): | |
| mask = self._generate_mask(grid_size) | |
| masks.append(mask) | |
| return torch.stack(masks).to(device) | |
| def _generate_mask( | |
| self, grid_size: Optional[Tuple[int, int]] = None | |
| ) -> torch.Tensor: | |
| """ | |
| Generate a single mask. | |
| """ | |
| if grid_size is not None: | |
| num_patches_h, num_patches_w = grid_size | |
| num_patches = num_patches_h * num_patches_w | |
| else: | |
| num_patches = self.num_patches | |
| mask = torch.zeros(num_patches, dtype=torch.bool) | |
| target_masked = int(num_patches * np.random.uniform(*self.mask_ratio)) | |
| # Random Permutation Masking | |
| if target_masked > 0: | |
| perm = torch.randperm(num_patches) | |
| mask_indices = perm[:target_masked] | |
| mask[mask_indices] = True | |
| return mask # Already flattened | |