Download gap-text2sql-main/mrat-sql-gap/seq2struct/models/abstract_preproc.py from c4ai-usp/gap-text2sql: direct link, hf CLI and curl.
- Browser
- Download file 1.38 kB
-
https://huggingface.co/c4ai-usp/gap-text2sql/resolve/refs%2Fpr%2F1/gap-text2sql-main/mrat-sql-gap/seq2struct/models/abstract_preproc.py
- Command line
-
hf download hf://c4ai-usp/gap-text2sql@refs/pr/1/gap-text2sql-main/mrat-sql-gap/seq2struct/models/abstract_preproc.py
-
curl -L -o abstract_preproc.py https://huggingface.co/c4ai-usp/gap-text2sql/resolve/refs%2Fpr%2F1/gap-text2sql-main/mrat-sql-gap/seq2struct/models/abstract_preproc.py
1.38 kB
| import abc | |
| class AbstractPreproc(metaclass=abc.ABCMeta): | |
| '''Used for preprocessing data according to the model's liking. | |
| Some tasks normally performed here: | |
| - Constructing a vocabulary from the training data | |
| - Transforming the items in some way, such as | |
| - Parsing the AST | |
| - | |
| - Loading and providing the pre-processed data to the model | |
| TODO: | |
| - Allow transforming items in a streaming fashion without loading all of them into memory first | |
| ''' | |
| def validate_item(self, item, section): | |
| '''Checks whether item can be successfully preprocessed. | |
| Returns a boolean and an arbitrary object.''' | |
| pass | |
| def add_item(self, item, section, validation_info): | |
| '''Add an item to be preprocessed.''' | |
| pass | |
| def clear_items(self): | |
| '''Clear the preprocessed items''' | |
| pass | |
| def save(self): | |
| '''Marks that all of the items have been preprocessed. Save state to disk. | |
| Used in preprocess.py, after reading all of the data.''' | |
| pass | |
| def load(self): | |
| '''Load state from disk.''' | |
| pass | |
| def dataset(self, section): | |
| '''Returns a torch.data.utils.Dataset instance.''' | |
| pass | |