HarborDataset
"# HarborDataset\n\n```python\nfrom modal_dojo import HarborDataset\n```\n\nA dataset loaded from Harbor tasks.\n\n## Constructor\n\n```python\nHarborDataset(*, split: Literal['all', 'train', 'eval'] = 'train', dataset_name: str = '', path: str | None = None, task_root: str = '', task_glob: str = '*', task_names: list[str] | None = None, instruction_path: str = 'instruction.md', label_metadata_path: str | None = None, test_data_dir: str | None = None, prompt_template: str = '{instruction}', system_prompt: str = '', train_size: int | None = None, eval_size: int | None = None, train_repeats: int = 1, eval_repeats: int = 1, shuffle_tasks: bool = False, shuffle_seed: int = 0, always_download: bool = False) -> None\n```\n\n## `apply_chat_template`\n\n```python\napply_chat_template() -> bool\n```\n\nWhether to apply the model's chat template to the input.\n\n## `cache_key`\n\n```python\ncache_key() -> str | None\n```\n\n## `input_key`\n\n```python\ninput_key() -> str\n```\n\nPrompt column name.\n\n## `label_key`\n\n```python\nlabel_key() -> str\n```\n\nGround-truth column name, or `None` when rows carry no label.\n\n## `load`\n\n```python\nload(split: Literal['all', 'train', 'eval'] = 'all') -> Any\n```\n\n## `output_format`\n\n```python\noutput_format() -> str\n```\n\nThe on-disk format written by `write()`, either `parquet` or `jsonl`.\n\n## `rows`\n\n```python\nrows() -> Iterable[DatasetRow]\n```\n\nLoad raw examples.\n\n**Returns**\n\nAn iterable collection of raw examples.\n\n## `validate_written`\n\n```python\nvalidate_written(path: str) -> None\n```\n\nValidate the materialized file format and required columns.\n\n## `write`\n\n```python\nwrite(path: str) -> None\n```\n\nMaterialize training data at `path`.\n"
from modal_dojo import HarborDatasetA dataset loaded from Harbor tasks.
Constructor
Section titled “Constructor”HarborDataset(*, split: Literal['all', 'train', 'eval'] = 'train', dataset_name: str = '', path: str | None = None, task_root: str = '', task_glob: str = '*', task_names: list[str] | None = None, instruction_path: str = 'instruction.md', label_metadata_path: str | None = None, test_data_dir: str | None = None, prompt_template: str = '{instruction}', system_prompt: str = '', train_size: int | None = None, eval_size: int | None = None, train_repeats: int = 1, eval_repeats: int = 1, shuffle_tasks: bool = False, shuffle_seed: int = 0, always_download: bool = False) -> Noneapply_chat_template
Section titled “apply_chat_template”apply_chat_template() -> boolWhether to apply the model’s chat template to the input.
cache_key
Section titled “cache_key”cache_key() -> str | Noneinput_key
Section titled “input_key”input_key() -> strPrompt column name.
label_key
Section titled “label_key”label_key() -> strGround-truth column name, or None when rows carry no label.
load(split: Literal['all', 'train', 'eval'] = 'all') -> Anyoutput_format
Section titled “output_format”output_format() -> strThe on-disk format written by write(), either parquet or jsonl.
rows() -> Iterable[DatasetRow]Load raw examples.
Returns
An iterable collection of raw examples.
validate_written
Section titled “validate_written”validate_written(path: str) -> NoneValidate the materialized file format and required columns.
write(path: str) -> NoneMaterialize training data at path.